Like its neighbor across the street (Old Mount Zion), Shiloh Baptist is one of the Mother Churches of the Albany Movement. The congregation was organized in 1888. The present structure was built by contractor A. S. Cobb in 1953, during the pastorate of W. H. Calhoun. Jim Bishop notes in his 1971 book, The Days of Martin Luther King, Jr., that one of his most famous phrases was first delivered in a sermon here: “We shall overcome. Don’t stop now. Keep moving. Don’t get weary children. We will wear them down by our capacity to suffer.”

A 2014 historical marker notes: The Albany Movement began here, at Shiloh Baptist Church, in November 1961. A coalition of black improvement associations and student activists from SNCC and Albany State College, the protest group set an unprecedented goal: the desegregation of an entire community, from bus stations to lunch counters. Demonstrations over two years resulted in the detention of 1,500 protesters. The participation and repeated arrests of Dr. Martin Luther King, Jr. brought national attention to the Albany Movement. Music sustained the campaign in Southwest Georgia and gave birth to the SNCC Freedom Singers. Legal action and the increase in black voter registration led to school desegregation, the end of public employee discrimination, and the election of black political officials in the region. Lessons learned in Albany influenced events in Birmingham in 1963.