This historic congregation was formed by Reverend Edward Lanier and Reverend Jesse Sratton on 27 March 1829. The Presbyterians of  Greenville were granted a lot in town for the building of a house of worship but sold it and built this one-room church a few miles from town in 1836, preferring a rural setting. Though it never boasted a large congregation, Greenville Presbyterian was quite active in the community. Dwindling membership and a newer church in Greenville, Stacy Presbyterian, led to the closure of the church in 1963 but it reopened in 1972. A small but determined congregation still holds services here. Greenville Presbyterian is significant as one of just a few antebellum Presbyterian churches in Georgia.

The cemetery is one of the most historic in the area. Perhaps the most fascinating interment is John Gaston, who was famously known as “The Giant”, for his 7’6″, 340-pound stature at a time when the average height was about 5’7″. Gaston was born in Chester County, South Carolina in 1821 and died in Woodbury in 1866. His slab has been damaged over the years, and a smaller adjacent slab corrects previous statistics, which stated his height as 7′ and his weight at 430 pounds.