What to use when sealing floor-to-wall and window-to-wall?
You can caulk the window to the drywall or moulding. I like siliconized latex caulk, but pure silicone can work well too (lasts longer, but not paintable, and collects dust more if it's on a horizontal joint).

For the baseboards, there are a couple of things you can do. If removing them is an option, you can stick foam backer rod between the customary gap between the drywall and the floor, if it's accessible. If that's not an option, you can make sure the baseboard is well-caulked to the wall at the top and the floor at the bottom. Clear caulk works well for this.