Jarvis Phelps was born on June 5, 1827, in Keene, New Hampshire, to John Phelps. He married Hannah Reynolds around 1849, and they had at least three children: Charles, born around 1850; Franklin, born around 1851; and Amanda, born around 1859. They lived in Brandon, Vermont, and he worked as a laborer. By 1860, he owned $50 of personal property. By 1870, he was working as a carpenter, and he owned $300 of personal property. He died of a cerebral hemorrhage in Brandon on August 21, 1902.