1. An alpha-map encoding apparatus comprising: a first down-sampling circuit configured to down sample an alpha-map signal, which represents a shape of an object and a position in a frame of the object, at a down-sampling ratio based on size conversion ratio information; an up-sampling circuit configured to up sample the alpha-map signal at an up-sampling ratio based on size conversion ratio information given to restore the alpha-map signal down-sampled by said first down-sampling circuit to an original size, and outputting a local decoded alpha-map signal; a motion compensation circuit configured to generate a motion compensation signal on the basis of a previous decoded video signal and a motion vector signal; a second down-sampling circuit configured to down sample the motion compensation signal at the down-sampling ratio; a binary image encoder configured to encode the alpha-map signal down-sampled by said first down-sampling circuit to a binary image in accordance with the motion compensation signal down-sampled by said second down-sampling circuit, and outputting an encoded binary image signal; and a multiplexer configured to multiplex and output the encoded binary image signal and the size conversion ratio information.