Strange artifacts when using Solidify
There is a vertex mishmash. Press M to Merge by Distance with value like 0,001

I did some more manual cleaning (Merge by Last to merge some others vertices that were close to each other) and Slide by Edge GG to straight back some edges of cylinder part.

To keep straight cut in some axis I would suggest to use Mirror modifier and as final touch apply Solidify and Mirror and use Bisect tool to cut.
Topology is not good, so if you will have more issues, you would have to retopology your object anyway.