Restrict HDRI Lighting to affecting certain places
Found the answer, For Cycles this node setup using the geometry node is to be used for making the object transparent on one side and completely opaque/diffused on the other, flipping the normals will help switch the sides and determine which side has which shader