Isaac Stafford was born around 1836 in New York. He married Cathaline Crounse on October 21, 1858, and they had at least two children: Mary, born around 1860; and Annette, born around 1863. They lived in Duanesburg, New York, and Stafford worked as a farm laborer. By 1860, he owned $100 of personal property.
He enlisted in the Union army on August 30, 1862, and he mustered in as a private in Company H of the 43rd New York Infantry on September 14. According to his service records, he was 5 feet, 6 inches tall, with light hair and brown eyes. The regiment took part in the Battle of Antietam and the Battle of Fredericksburg. In November 1862, he reported hearing rumors that "england is triing to make peice," and he confessed that, "if that is the case I would like it very much." Nonetheless, he remained devoted to the Union, writing that his comrades had "taken thare oath to lay down thare lives for the union if it was recuired of them." He was severely wounded at Fredericksburg on December 13, 1862, and surgeons amputated his right leg. He died in Washington, D.C., on December 30, 1862.