Will my time machine back up go crazy if I changed my password for file vault?
No, this doesn’t change things. If Time Machine goes crazy, it’s for other reasons than this.
Now, if you are going to change the encryption keys, the pass phrase to unlock the actual encryption key can be changed trivially but the underlying encryption can be a very lengthy process of decrypting and then encrypting and that can be tedious or worse if you mess up.
You would want to get a thorough dump of your exact hardware, diskutil list and possibly more. There are many flavors of filesystem and “file vault” around.