mesh collision rigid bodies acting very strange. (bouncing, shaking, etc...)
I think that you're compounding workaround with workaround here.  You're trying to do stuff with collision when rigid body constraints would work better.  If that's the origin of your hammer, you're placing it at the center of rotation, rather than the center of mass, probably to get behavior you want, yet placing the origin here is creating other problems.
The important, non-obvious things for rigid body physics are:

All mass lies at the origin of the object.  No mass lies anywhere else.  If you place your origin someplace other than the intuitive center of mass of the object, expect unintuitive results.

Apply scale on all rigid body objects.

Simpler collision models are more accurate collision models.  (Which it sounds like you've already learned.)


Now, you want to hit something with the hammer.  Do you want to hit something with the handle?  If you don't, don't try to make this a rigid body; don't try to use collision to control the rotation of the hammer; use rigid body constraints to control the action of a hammer-only rigid body instead (a rigid body with an appropriately placed origin.)  Note that the hammer head itself is convex and convex-hull collision would work fine for it.  (But for your needs, I'd expect that cube collision would work just as well.)
So try this:

Apply scale on all objects.
Separate the hammer head to a different object.  Give it convex hull collision.  Set origin to center of geometry.
Remove rigid body physics on the handle, and parent the handle to the hammer head instead.
At the center of the axis, create an empty, give it a rigid body constraint physics, joining the axis (object 1) to the hammer head (object 2.)  Make the constraint &quot;hinge&quot; type and rotate it such that its Z axis points in the same direction as the axis.  Don't parent it to anything.

I think you'll find that does what you want.