Can I realistically replace my drill with a rotary hammer?
Not really. I did this and regretted it. 

The problem is that the rotary hammers that are worth it are very heavy and overbuilt (the larger Bosch SDS series etc.). They make fantastic rotary hammers but are too heavy for use as a drill for extended use unless you have really beefy forearms. In this scenario, you should get a 18v cordless drill for when  you want to drill.

Then there are very lightweight rotary hammers (such as from Harbor Freight) where the rotary hammer action is extremely weak. These do a good job as drills but not as rotary hammers.