Harriet Richmond (maiden name: Smith) was born on March 24, 1824, in New York to Henry Smith and Amanda Waring. Her father was a farmer who owned $2,000 of real estate and $750 of personal property in 1860. She probably grew up in Monroe County, New York, and she married Henry E. Richmond around 1846.
They had at least eight children: Sarah, born around 1847; Addison, born around 1848; Samuel, born around 1853; Emma, born around 1855; Fremont, born around 1857; Billings, born around 1861; Ernest, born around 1863; and Delevan, born around 1868. The family lived in Churchville, New York, and her husband worked as a farmer. By 1870, they owned 23 acres of land worth $7,500 of real estate, as well as $3,500 of personal property. She died in New York on October 12, 1896.