James Hilton was born around 1843 in England to John and Amelia Hilton. His father was a weaver who owned $100 of personal property in 1860. The family immigrated to America sometime in the 1840s and settled in Philadelphia, Pennsylvania. Hilton attended school there, and by 1860, he was working as a wheelwright’s apprentice.
He enlisted in the Union army on August 15, 1862, and mustered in as a private in Company I of the 121st Pennsylvania Infantry later that day. His father mustered into the same regiment soon afterward. The regiment took part in the Battle of Fredericksburg, the Battle of Chancellorsville, and the Battle of Gettysburg. Confederate forces eventually captured him and sent him to a prisoner of war camp in Florence, South Carolina. He died there on October 1, 1864.