Joseph H. Rounds was born on June 3, 1841, in Danville, Maine, to Nathaniel Rounds and Susan Libby. His father was a farmer who owned $1,200 of real estate by 1850. Rounds grew up and attended school in Danville, and by 1860, he was working as a teacher. He attended the Maine State Seminary in the early 1860s. On March 22, 1865, he received a commission as a 1st lieutenant in Company H of the 14th Maine Infantry. He was promoted to captain soon afterward, and he mustered out on August 28, 1865. He returned to Maine after the war, and he died on January 20, 1867.