Founded in 1887 by the Reverend Joseph Williams, Ebenezer Presbyterian is an important African-American congregation. Reverend Williams’s headstone, which faces the church from across US 17, reads: In Memory of Rev. Joseph Williams – Founder of Presbyterianism among the colored people of Georgia – Born in Providence Island West Indies A. D. 1805 – Died at Riceboro Ga U.S.A. Nov. 22, 1899.

A Chronicle of Black History in Liberty County, Georgia, by Lillie Walthour Gillard, gives insight into the work of the Reverend Mr. Williams, as he was widely known. She writes: The Reverend Joseph Williams, a native of the West Indies, came to Liberty County from Macon, Georgia, in the year 1867. On April 12, 1868, he organized a Presbyterian church in the building of “Old Midway” church with 300 members and worshipped there for eighteen years. This congregation became part of Knox Presbytery and of the Presbyterian Church U.S.A., and was known as the Midway Presby­terian Church. Following a period of controversy over the rights of occupancy of Old Midway-Congre­gational or Presbyterian-the Reverend Mr. Williams organized a group of forty-six persons formerly members of a church pastored by Dr. C. C. Jones. In 1880 the church moved to Riceboro where a new building was constructed with the assistance of the Presbyterian Church U.S.A.