Can a new chain significantly improve the riding experience? If yes - what else can?
The one substitution that will probably give you the most bang for your buck is better tires. High-quality tires roll with much less rolling resistance, have better grip, weigh less, and roll over small imperfections more easily. And if you're riding on tires with knobby treads, but only riding on the road, tires with minimal/no tread will ride much faster and smoother.

Keeping your drivetrain clean and properly lubed will also make a noticeable difference.

Making sure your bike fits you will make a big difference, and you may want to get new handlebar/stem/saddle to achieve a good fit.