Should I calculate SHA-256 hash of a raw bytes of a token or it's hex encoding?
The encoding doesn't change the output of the entropy source. They are reversible operations. You can use what encoding suits you, change to base64 for transmission and strong on the database, and change to byte is a good choice since, in general, the cryptographic hash functions are accepting bytes to process, so it might better to convert them bytes before applying SHA-256 so that you can get the benefit of the speed of the SHA-256. Base64 or Hex may need more time to hash the same data due to the increase in the length, that is $~3n/4$ in Base64, and $2n$ in Hex for $n$ bytes.
If the CSPRNG has good entropy then you even don't need the SHA-256, or at least 4096-bit token, hashing the output of the source is a good idea, however, if the source needs 4096-bit to produce good 256-bit randomness, there might be a problem there. This, however, really depends on your CSPRNG.