Edward Clifton Wharton was born on November 18, 1827, at Fort Gibson in the Oklahoma Territory to Franklin and Mary Jane Wharton. His father was a prominent lawyer. The family settled in New Orleans, Louisiana, when he was a child, and by 1850, he was working as a newspaper editor there. He married Louisiana Goodman, and they had at least five children: Mary, born around 1854; Amelia, born around 1856; Theodore, born around 1858; Edward, born around 1862; and George, born around 1864.
He moved to Galveston, Texas, in the late 1850s. He supported the Confederacy during the Civil War, and he served on General John B. Magruder’s staff. He returned to New Orleans after the war. His wife died in 1868, and he remarried sometime before 1880. He died in New Orleans on June 13, 1891.