Agnes D. Fraser was born on May 9, 1837, in Scotland to Alexander Fraser and Margaret Prugh. Her father was a plasterer. The family immigrated to America around 1844 and settled in Allegheny, Pennsylvania, and her father died there sometime in the 1850s. By 1860, the family owned $1,300 of real estate and $200 of personal property. She began working as a schoolteacher in the early 1860s. She and her sister Ann lived together until at least 1880, and they employed at least one white domestic servant. She died of uterine cancer in Pittsburgh, Pennsylvania, on July 19, 1908.