How do I replace a triple dimmer switch to a much smaller conventional switch?
Well, this is a really old question but it sounds like your dimmer is a decora style switch.  You can switch this decora dimmer to a regular style switch and buy a new cover plate.  Alternatively, you could install a standard decora switch in place of the dimmer switch.

However, if by 'triple' switch, you mean three way switch, you should be warned that diagnosing an incorrectly wired 3-way switch can be confusing to the electrical neophyte.  Additionally, because a traveler can be electrified when you expect the wire to be off behind a switch, it can be much more dangerous to work on for someone who is a novice with 3-way switches.  I would strongly recommend using an electrician to solve this problem.  If not, I'd strongly recommend turning off all the power while working on this problem, labeling everything as/before you disconnect the switches, and using a continuity tester (while the power is off) to determine which wire is the traveler to which switch.

Of course, another possibility is that you have a 3-way configuration but the dimmer is a single pole dimmer.  Switching to a 3-way dimmer switch could solve this problem but you'll still to know or figure out the correct traveler configuration for your situation, which will require a decent understanding of 3-way switches.

If in doubt, be safe and use an electrician.