Solidify Modifier going in all directions
The problem with this mesh is it's mb its topology in order to fix that,  first go into edit mode and select all faces and then press shift+N to calculate normals, then try pressing alt+M and selecting merge by distance for removing double, if that still doesn't solve the problem then apply a remedy modifier above solidify (you even remedy manually or by some other add-on).  Try these one by one or all at the same time.