A method and apparatus for lossless compression and decompression of video image data. Video data is comprised of the RGB, YUV or gray scale (color) information for each pixel of each frame of the video data. Storage of such video data can require large amounts of data. The present invention provides for lossless compression and decompression of the individual frames of video image data. Each individual frame of video image data is processed in n.times.n pixel blocks. The combination of colors (called the color list) and the number of different colors in the pixel block is then determined. If the number of colors is above a predetermined threshold, no compression occurs. If the number of colors is below a predetermined threshold, searches for prior occurrences of the combination of colors is than performed. In some instances, a color list for contiguous pixel blocks will be the same. In this instance, a pixel map representing the positioning of the colors in the pixel block is output. Further, each encountered combination of colors is stored as an entry in a color cache. The color cache is subsequently used as a color combination dictionary. In such instances of a pixel block where its color list is in the color cache, an index to the entry into the color cache is used to represent the colors found in the pixel block. The color cache need not be stored with the compressed data stream since it is reconstructed during decompression of the compressed data stream.