A highly efficient video coding scheme is disclosed which codes, for transmission, the pel values of a scanned high quality HDTV video signal in such a manner that a low-quality version of the signal as well as a high-quality version are available to different video subscribers. The coder includes a basic layer coder (110) and a contribution layer encoder (150). A subscriber to a low-quality video service receives only the basic layer signal while a high-quality video subscriber receives both the basic layer signal and the contribution layer signal, which when combined together enable the high quality video signal to be reconstructed. The basic layer coder codes the baseband of a frequency decomposed video signal using a hybrid discrete cosine transform/differential pulse code modulation coding structure, such as the CCITT recommended H.261 coder. The contribution layer coder decomposes (123) a differential high quality video signal and separately quantizes (124) and entropy codes (125) the decomposed differential subbands other than the baseband, which are then multiplexed (126) together to form the contribution layer signal. When the baseband is coded by the basic layer coder at a low bit rate, the quantization noise it introduces will deleteriously affect the quality of a high quality signal "built" upon it. A residual signal component is therefore included within the contribution layer signal. This residual layer component, which is derived in part from the basic layer coder, enables the quantization noise introduced into the basic layer to be substantially eliminated when the basic layer signal is combined with the contribution layer signal to reconstruct the high quality video signal at a receiver/decoder.