Joel B. Norris was born on April 2, 1821, in Canandaigua, New York, to John Norris and Betsey Gage. His father was a carpenter who served in the War of 1812. He grew up and attended school in Canandaigua. He married Margaret Brown on December 16, 1846, and they had at least two children: Harriet, born around 1848; and Charles, born on March 2, 1852. They lived in Canandaigua, and Norris worked as a farmer. By 1850, he owned $300 of real estate. They moved to Cambria, Michigan, around 1853, and by 1860, he owned $500 of personal property. His name was selected in the draft in March 1865, but he apparently never served in the Union army.
He supported the Democratic Party, and he attended a local party convention in 1866. He ran for a seat in the state legislature in 1868 and 1876, but he was apparently defeated in both elections. He served as superintendent of the poor for several years. He died of pneumonia in Hillsdale, Michigan, on March 11, 1895.