In the fall of 1892, Dr. George Romoser of Concordia College in Conover was invited by a small group of Lutheran families to hold preaching services at the old Setzer Schoolhouse, which stood about 400 yards behind the present site of the old church and preschool building.  These first services were held in a grove of trees, with slab benches serving as seating.  None of the hardy souls attending those first services could have foreseen what the next century would hold.

The Mt. Olive Congregation was formally organized in 1895, with a membership of 50 souls.  The congregation went to build a sanctuary on land donated by Mrs. George Moose and Mr. J. Noah Setzer.  The men of the congregation cut the timber, had the logs sawn, and erected the building themselves, dedicating it in the fall of 1895.  Rev. George Luecke was called to serve the congregation, doing so for one year before being called to faculty position at Concordia College.  Rev. George E. Long was called in 1896, serving until 1909.  In that year, Rev. Sidney Keisler became the congregation's third minister, serving Mt. Olive until 1912.

In 1912, Rev. George Long returned to Mt. Olive, remaining here until his death in 1936.  During his pastorate, the congregation's growth dictated the need for a larger sanctuary.  Despite the fact that the Great Depression held the nation in its grip, the Mt. Olive family was determined to move ahead.  In 1932, the original wooden structure was torn down and a larger brick church building was erected, with part of the work being done by members of the congregation.