Joseph A. Carroll was born on November 28, 1832 in Pike County, Missouri. He was the youngest of eleven children. He received a common school and academic education, clerked in the commercial house of Edwin Draper Brothers in Louisiana. He left for Texas in 1853, and arrived in Denton, Texas where he was appointed commissioner to sell lots and practiced law till 1861 and he married Miss Celia J. Burrows on March 28, 1858. They later had three children together. After being discharged from the Confederate army as a major at Hempstead, Texas on June, 1865. Joseph A. Carroll returned to practice at the bar and he prospered. In 1869, his wife died. Later, he married his second wife, Miss Martha Inmon. In February, 1876, Major Carroll was elected judge of the district until Januray 1881. He then embarked on the banking business in Denton, Texas.