How can I eliminate the added height when applying a thickness to Solidify modifier?
Adjust normals

Solidify modifier extrudes faces along their normal.

A default UV hemisphere has the bottom faces whose normals are not parallel to the XY plane. So the extrusion vector will have a Z component that will create the effect you see in the image below (which is an outer extrusion, while yours is an inner one, but it follows the same principle):



In blue are evidenced the vertex normals. Note how the last line of vertices has a non-horizontal normal projection.

To keep the bottom of the extrusion co-planar with hemisphere's bottom vertices you should start from a geometry who's last faces normals are not incident with XY plane:



The last picture was made by extruding the last line of vertices a bit on the Z-axis, so the geometry may not exactly represent a standard hemisphere.

To obtain a more precise result you can work with the default UV sphere settings and set an odd number of rings. This way, when you cut the sphere in half the faces will already have the correct normals and the resulting mesh will be a more precise approximation than extruding manually.