Need help with padding
Consider ANSI X.923 padding for example, how would the decryptor know that the sequence 00 00 00 04 is not some special data but padding?


Because padding is always used. Thus if the file before padding was 41 42 43 44 00 00 00 04, it would be padded 41 42 43 44 00 00 00 04 00 00 00 00 00 00 00 08 then encrypted.

This rule, or similar ones differing on how the padding octets other than the last are generated, are common and fine when padding octet strings.

For the most general problem of padding arbitrary bit strings, one of the simplest and most general is bit padding, which appends a single 1 bit, then zero to b-1 0 bit(s) as necessary to reach a multiple of the blocksize $b$ (in bits). That can be designated as ISO/IEC&nbsp;9797-1 padding method 1, but this standard does not prescribe un-padding. This should fail is there's not at least one block; removes final 0 bit(s) in the last block, and fail if that exhausts the last block; finally, remove the final 1 in that last block.

When restricted to octets in big-endian convention, bit padding becomes appending a 80 octet, then zero to b/8-1 00 octets as necessary to reach a multiple of the blocksize.

Use a simple padding rule, and always apply it; complexity and exceptions are the enemy of practical security. Whatever the method, beware of padding oracle attacks on decryption, and off-by-one coding errors in implementations.