For hardware encryptions, is it possible to know the key by analysing the hardware? For example, Square reader uses hardware encryptions when swiping credit cards.

Is it possible to know the key that is used by the hardware encryption cipher by analysing the hardware circuitries?

The key has to be somewhere in the hardware, right?
That really depends on the device. Usually there is a full, layered, key management scheme. That may involve a key in ROM, but that's then just use to load the next keys for the higher layer in persistent memory (flash or EEPROM mostly). In the end session keys are generated in transient memory (RAM).

It's very likely that if you find a key in ROM that that key is only used to load the initial software. The keys to actually encrypt the (credit card) data are likely to be in EEPROM somewhere, possibly encrypted with a static key in the software.

"Hardware encryption" usually means that the encryption algorithms themselves are in the chip. That makes them less vulnerable to side channel and fault injection attacks. It will usually also give a large performance enhancement; 8 and 16 bit CPU's are not too efficient with block ciphers like AES.

Sometimes a key store is however implemented using hardware. The key that protects the data in that key store may be in ROM.