James Lawrence Bryant was born on May 31, 1841, in Jefferson, New Hampshire, to Benjamin Bryant and Frances Tenney. His father was a laborer who owned $200 of personal property in 1860. The family moved to Framingham, Massachusetts, sometime in the 1850s. By the early 1860s, Bryant was working in a factory. He enlisted in the Union army on August 6, 1862, and mustered in as a private in Company C of the 33rd Massachusetts Infantry later that day. The regiment took part in the Battle of Chancellorsville and the Battle of Gettysburg. The army transferred the regiment to the Western Theater later in 1863, and the men fought in the Battle of Missionary Ridge, the Atlanta Campaign, the March to the Sea, and the Carolinas Campaign. He was promoted to sergeant on May 1, 1865, and he mustered out on June 11, 1865.
Bryant returned to Framingham after the war, and he married Anna Eliza Brown around 1867. They had at least five children: Edna, born around 1869; Anna, born around 1871; Mary, born around 1878; Walter, born around 1880; and Bessie, born around 1892. The family lived in Newburyport, Massachusetts, and Bryant worked in a cotton mill. By 1900, he was working as a factory overseer. He died in Newburyport on March 12, 1928.