Stephen F. Rathbone was born around 1830 in Washington County, New York, to Kenyon Rathbone and Perlina Freeman. His father was a farmer who owned $17,700 of real estate in 1850. Rathbone married Jane Hill around 1850, and they had at least four children, including: Sarah, born around 1852; Hiram, born around 1858; and Estella, born around 1859. Their son Hiram died on January 12, 1859. Rathbone worked as a laborer and farmer in Easton, New York.
He enlisted in the Union army on November 6, 1861, and mustered in as a sergeant in Company E of the 96th New York Infantry on November 19. According to his service records, he was 5 feet, 7 inches tall, with brown hair and brown eyes. The regiment took part in the Battle of Seven Pines and the Seven Days Battles. He was reduced to the ranks on July 11, 1862, but he became a sergeant again on September 1, 1863. Army officials transferred him to a hospital in Albany, New York, on May 28, 1864, and he mustered out there on July 1, 1865. Rathbone returned to Easton after the war, and he applied for a federal pension in December 1872. He eventually moved to Aspen, Colorado, and he died there on July 22, 1898.