Visitor drop off since moving over to Craft CMS
I wouldn't be overly concerned about bounce rates and such, I'd be more concerned about conversion metrics. The reason is that many things affect bounce rate; if the new site is better designed, people may not need to click on additional pages to find out what they want to know, for instance.

There's nothing inherent with Craft that would result in this type of a falloff. For sure making sure all of the 404 errors get buttoned up would be a good start, and the Retour plugin can assist with that. You could also consider installing the SEOmatic plugin to assist you with modern SEO.

Disclaimer: I'm the author of both of those plugins.