I’ve been unable to locate a history of the congregation, but I believe the building dates to the first two decades of the 20th century, when this style was quite common with Baptists.
Vanishing Georgia: Photographs by Brian Brown
Since 2008. A visual survey of Georgia's historic architecture and culture. 8,000 locations. 27,000 images.
Vanishing Georgia: Photographs by Brian Brown 
it goes back to the early 1800’s
Could it have been a synagogue, maybe? They often have that sort of dome especially the ca. 1920’s ones.
I believe this has always been a Baptist Church.
Impressive!