What does GHASH really do?
The problem is probably in the usage of BN_bin2bn to construct K (H in the picture) and ADD. GCM specifies that a 16-byte buffer should be transformed into a binary polynomial in little-endian format, but in each byte, it considers the bits in reversed order. Yes, it's confusing. So for example:


The byte array 80 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 represents the polynomial $1$ (e.g. with a poly = {0} when using BN_GF2m_arr2poly
The byte array 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 represents the polynomial $z^7$ (poly = {7})
The byte array 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 represents the polynomial $z^{127}$ (poly = {127})


So, before using BN_bin2bn, you need to reverse the bytes, and then reverse the bits in each byte. Maybe there is something already implemented in OpenSSL, but I didn't check it.