Replacing Tall Baseboards with Smaller Baseboard
I guess it's too late to consider 5-1/4" trim?

Hit the line of old caulk with a long blade utility knife (like an Olfa), a sharp chisel or a wallpaper scraper. Then sand lightly to knock down any other lumps and apply a coat or two of mud. (Premixed if you have time; 20 or 90 minute setting compound if you're in a hurry.) When dry, sand lightly.