James D. Stearns was born on April 29, 1822, in Florida, New York, to Amory and Maria Stearns. His father was a baker who owned $450 of real estate in 1850. He married Emeline Frank around 1849, and they had at least four children: Frank, born around 1850; Ella, born around 1853; Warren, born around 1860; and Elizabeth, born around 1869. They settled in Jamestown, New York, and Stearns worked as a lumberman. By 1860, they owned $480 of real estate and $300 of personal property.
He enlisted in the Union army on August 23, 1862, and mustered in as a private in Company 7 of the 1st New York Sharpshooters on September 12. According to her service records, he was 5 feet, 7¼ inches tall, with gray hair and blue eyes. The regiment took part in the siege of Suffolk, the Overland Campaign, the siege of Petersburg, and the Appomattox campaign. He mustered out near Washington, D.C., on June 3, 1865.
Stearns settled in Ellicott, New York, after the war and earned a living as a night watchman. By 1870, he owned $3,600 of real estate and $200 of personal property. His wife died in January 1893. He suffered a “stroke of paralysis” on January 12, 1899, and he died the following day.