Joseph Henry Polley was born on December 28, 1795, in Whitehall, New York, to Jonathan Polley. He served in the New York militia during the War of 1812. He moved to St. Louis, Missouri, around 1818 and then settled in Texas in 1821. He married Mary Bailey in 1823, and they had at least nine children: Mary, born around 1829; Susan, born around 1835; Sarah, born around 1837; Catharine, born around 1839; Joseph, born on October 27, 1840; Harriet, born around 1843; Abner, born around 1845; Jonathan, born around 1849; and Walter, born around 1852.
They lived in Brazoria County, Texas. According to one historian, Polley “accrued land and slaves through homesteading, purchase, and his wife’s inheritance. He tried to grow cotton, but uncertain prices drove him to focus his efforts on cattle.” He moved to Guadalupe County, Texas, in 1847. He worked as a farmer and stock raiser, and by 1850, he owned $20,000 of real estate. A decade later, he owned $70,000 of real estate and $65,351 of personal property. He owned at least twenty enslaved laborers. During the Civil War, one scholar writes, he “grew cotton, which he sold for specie in Mexico, along with cattle. He abandoned the effort when emancipation came.” He died on March 26, 1869.