Skillman Doughty was born around 1826 in New York to Benjamin Doty and Jane Lewis. His father was a farmer who owned $5,000 of real estate by 1850. He grew up and attended school in Romulus, New York, and by 1850, he was working as a laborer. He married Phebe Smith around 1851, and they had at least three children: Seward, born around 1852; Edmund, born around 1854; and Julia, born around 1857. He worked as a farmer in New York, and by 1860, he owned $200 of personal property.
He was drafted during the Civil War, but he arranged for English immigrant James York to serve in his place. By 1870, his personal estate had grown to $2,855. A decade later, he owned 64 acres of land worth about $5,000. In 1879, his farm produced 200 bushels of barley, 677 bushels of hay, and 110 bushels of wheat. He died of a “fever” in Seneca County, New York, on December 19, 1885.