Inman Horner Payne was born on August 28, 1822, in Fauquier County, Virginia, to George Payne and Catherine Edmonds. His father died on December 29, 1822, and his mother followed on November 9, 1836. Payne married Mary Anne Massie on January 30, 1845, and they had at least nine children together: Mary, born around 1847; George H., born around 1851; John M., born around 1853; Inman H., born around 1854; Bernard W., born 1859; Julian, born around 1861; Markham, born around 1865; Lena, born around 1867; and Thomas, born around 1868. They lived in Fauquier County, and he worked as a farmer.

Payne joined the Democratic Party, and he helped organize a dinner to Vice President George M. Dallas in 1846. He played an active role in local Democratic politics for the next several years. In 1858, he opened a grocery and hardware store with several of his neighbors, advertising that they had a “Railroad to our door, and ample Warehouse, Wagon yard, Platforms, &c.” By 1860, he owned at least $32,920 in real estate and $14,812 in personal property, including at least seven slaves.

During the Civil War, Payne spent eighteen months working in local Confederate hospitals. He may also have served as a commissary captain. He applied for a presidential pardon in August 1865, and President Andrew Johnson promptly granted him one. By 1870, after years of war and economic upheaval, the value of his real estate had plummeted to $19,000, and he owned only $4,100 in personal property.

In 1894, one reporter noted, Payne and his wife “moved from their country seat to the Warren Green Hotel, where they are receiving as cordially as in their honeymoon half a century ago.” Four years later, a local newspaper publicized their 53^rd wedding anniversary, observing that they were “remarkably active and free from bodily infirmities.” The writer predicted that “long years of married bliss are yet in store for them both.” Mary died in Warrenton on October 6, 1900. Payne spent the next few years making “annual pilgrimage to his numerous children,” travelling as far west as Montana and as far south as Louisiana. He died in Hamilton, Montana, on October 10, 1905, while visiting his daughter Lena.