What's the name of this cipher? And how can it be broken?
This would appear to be CFB mode, only using a hash function (with a secret key as part of the input), rather than a block cipher.

As such, assuming that the hash function with the partially secret input acts as a random Oracle, this provides privacy.  However, it doesn't provide integrity (that is, the attacker can modify the ciphertext, and the resulting ciphertext will decrypt, and in fact produce semipredictable changes to the plaintext), and will probably end up being slower than a more conventional encryption method.