Strictly speaking, the most important difference is the way the building is used. While a church is a building for public worship (most commonly of the Christian faith), to which people will travel and worship. ... Even though a church refers expressly to a specific building, that isn't always the case with a monastery.