Determine hashing algorithm
I just tried this tool to no avail

https://www.onlinehashcrack.com/hash-generator.php#res

My best guess is that the program is actually doing more than a simple hash, it may be a number of hash rounds or some other manipulations

You said that changing the plaintext makes the hash change. So you have a program which performs such unknown hash. Decompile it and work out the process behind. The algorithm is not "secret", it's stored in the executable