1. An image decoding apparatus adapted for decoding information obtained by implementing inverse quantization and inverse orthogonal transform to an image compressed information in which an input image signal is blocked to implement orthogonal transform on a block basis to quantize the input image signal, the image decoding apparatus comprising: a decoding unit configured to decode quantized and encoded transform coefficients; an inverse-quantizing unit configured to indicate, as a flag, existence of each transform coefficient in a processing block of the inverse-quantization; and an inverse transform unit configured to change, based on the flag, inverse transform processing to be implemented to inverse quantization transform coefficients within the processing block, wherein changing the inverse transform based on the flag comprises changing the inverse transform when a flag pattern indicates the existence of inverse transform coefficient coefficients only at a plurality of pixels of an uppermost row within the processing block.