What are some cost-effective tips for reducing external sound from entering a room?
Step 1: Prevent air transmission between the rooms. Sweeps for the door or a heavier door; look at the heating registers and windows; things like that. You're saying the sound is muffled, which implies the high frequencies aren't making it in, so I don't think this is the major problem.

Step 2: Damp the transmitted vibration. Put big heavy bookcases on the wall between the two rooms (which side doesn't matter so much). Blow heavy insulation into the wall between. Hang a heavy quilt on the wall.

Step 3: Break up the sound in the source room. Use natural diffusers (heavy sofas with heavy foam pillows work great, or the aforementioned bookcase with loads of stuff on it), or buy foam diffusers to stick in the corners/edges of the room.

Step 4: Give up and wear sound-cancellation headphones...or install a white-noise generator in your office (either a fan-type one, or a water feature if you like more 'natural' sounds).

See also How Do You Reduce Floor Sound Transmission for more details.