James Gillum (Gillam) was born on June 4, 1826, in Albemarle County, Virginia. A slave, he belonged to Nathan S. Gillum of Pike County, Missouri, for whom he worked as a farmer and teamster. At some point, Gillum married an unnamed woman who died in 1866, with whom he had an unnamed child.

Gillum enlisted as a private in the Union army on August 30, 1864, in Bowling Green, Missouri, and mustered in the same day at Benton Barracks in St. Louis. His service record describes him as 5 feet, 8 1/2 inches tall, with crisp hair, black eyes, and a black complexion. He served first in Company I of the 18th USCT Infantry Regiment but later transferred to Company F. He achieved the rank of first sergeant on October 10, 1864, commissary sergeant on October 6, 1865, and quartermaster sergeant on November 10, 1865. Gillum served throughout Tennessee during the war. He saw action at the Battle of Nashville on December 15-16, 1864, in which the Union army repulsed General John Bell Hood's attempt to capture the state capitol. The 18th USCT then joined the rest of the Union forces in Middle Tennessee to pursue Hood to the Tennessee River that winter. Gillum mustered out on February 21, 1866, in Huntsville, Alabama.

Following the war, Gillum returned to Missouri. He briefly lived in St. Louis before moving to Quincy, Illinois, where he remained for the rest of his life. He worked as a preacher and a peddler. Gillum married his second wife, Annie Smith, on January 29, 1875, in Adams County. She died on April 25, 1883. Together they had one child: Nellie G. Brown, born on September 26, 1875. Gillum died of lung disease in Quincy on June 9, 1887. Following his death, his daughter Nellie applied for a pension, but she never received one due because she was unable to prove that Gillum died from a disease caused by his service.