A novel variable-size block multi-resolution motion estimation (MRME) scheme is presented. The motion estimation scheme can be used to estimate motion vectors in subband coding, wavelet coding and other pyramid coding systems for video compression. In the MRME scheme, the motion vectors in the highest layer of the pyramid are first estimated, then these motion vectors are used as the initial estimate for the next layer and gradually refined. A variable block size is used to adapt to its level in the pyramid. This scheme not only considerably reduces the searching and matching time but also provides a meaningful characterization of the intrinsic motion structure. In addition, the variable-MRME approach avoids the drawback of the constant-size MRME in describing small object motion activities. The proposed variable-block size MRME scheme can be used in estimating motion vectors for different video source formats and resolutions including video telephone, NTSC/PAL/SECAM, and HDTV applications.