Luther Sizer was born around 1798 in Connecticut to Eli Sizer and Jedida Prior. He married Perlina Barnum around 1820, and they had at least nine children: Warren, born around 1822; Olive, born around 1825; Emma, born around 1830; Luther, born around 1832; Eber, born around 1834; Emeline, born around 1836; Jane, born around 1838; Franklin, born around 1840; and Julia Ann, born around 1842. He moved to Western, New York, sometime before 1840, and he worked as a farmer. He owned $8,000 of real estate by 1850, and a decade later, he owned $9,000 of real estate and $120 of personal property. By 1870, his wealth had grown to $13,000 of real estate and $3,000 of personal property. He died sometime before 1880.