During the 19th century, the village of Cold Spring and the daily lives of its inhabitants revolved around the West Point Foundry, established in 1818.
The president of the Foundry, Gouverneur Kemble, concerned for the religious life of the workers, responded favorably when the Catholic workers expressed a
desire for their own place of worship. In 1830, Mr. Kemble donated a site on the banks of the Hudson, a considerable sum of money towards the project, and he commissioned the architect, Thomas
Kellah Wharton to design the chapel. The 16 year old Wharton drew plans for a building of light ochre brick with a four columned portico in the Greek Revival order. In
1834, the building was consecrated by Bishop Dubois as St. Mary's. This was an outstanding event because St. Mary's was the first Catholic church in the
Hudson Valley and the first beyond Manhattan Island in the New York Archdiocese.
In 1867, the chapel was enlarged and two Victorian steeples were added to the original classical structure - an unfortunate architectural marriage.
The renovated chapel was rededicated in 1870 and served the faithful until they outgrew this building and erected a new church in 1906 at a more convenient location.
Over the years, the abandoned chapel fell into disrepair and in the 1930's was all but destroyed by a fire. However, in 1966, a restoration effort
began and in 1977, the chapel, restored to its original classical style, was rededicated as Our Lady Chapel.
Presently, Our Lady Chapel is non-denominational and reserved for ecumenical services, weddings and other special services.