Main breaker does NOT turn power to house off
From the small amount of information, I can give you some answers that may be more like starting points.

Quick Answer?  Hire an Electrician.

Long Answer:  If you took out the breaker to replace it, the power should have gone out/off.  If that didn't happen, there are three scenarios I can think of:

1 - Your breakers have been bypassed to send power directly to the house.  Really unsafe as no one can turn out the power to the house.

If the panel was bypassed, you're going to need someone with the proper training and experience to handle it.

If there are other breakers, that may be the reason why the breakers didn't work.

2 - There is, at least, one more panel box.

I have seen older houses with one "breaker" panel inside a closet inside the house.  The interesting part was those were the main circuit breaker.

If the power is from an overhead line, look at it and see where it's coming into the house.  It's harder for underground. 

Where it comes into the house, that is normally where the breaker panel is.

If there are other breakers, that may be the reason why the breakers didn't work. 

3- There is another panel/circuit box and the breakers you have have been rerouted around this one to the other one.

If this is the case, the outside breaker can be reinstalled, but your going to need to contact your power company and have the power for the entire house switched off while repairs are made.  This will probably be a contractor/electrician's job and may require permits and inspections by the municipality where it's located AND the power company.

Please use caution.  The breaker boxes, at least in California, have a lot of juice and one misstep could cost you your life.  Not knowing how the system is set up could also make wires and circuits have a charge when you don't expect it.

Use a detector to determine if circuits are charged and NEVER assume or guess about these things.

Absent a photo or video, it's difficult to make an exact diagnosis.  Your location will also dictate what the normal practices and codes are for your area.

When in doubt, call a professional to look at it.  Most in the US will give you a free estimate of any repair work they propose.  Main lines are NOT for the average handyman and even many electricians don't want to get into them. 

I am a serial DIYer and rennovator and have put in a LOT of circuits, outlets, exterior and interior lines in my houses over the last 30 years.  My Father-in-Law was a contractor and he taught me a lot about household electrical.