A modular building system is adapted for the construction of various temporary and semi-permanent relatively light structures, such as yard storage sheds, greenhouses, gazebos, sun rooms and the like. The system includes various modular framing elements, with base and header plate components each containing a integral roller having an extendible and retractable flexible wall or roof panel respectively attached thereto. Thus, the frame members may be assembled to form the desired structure, with the wall and roof panels being selectively extended from the base and header plate members as desired to close various respective wall or roof portions of the structure as desired. The extendible and retractable wall and roof panels may be formed of transparent, translucent, or opaque thin and flexible sheet materials as desired, either of woven or non-woven materials, metal or plastic screen, etc. The frame components are preferably formed of a plastic such as polyvinyl chloride; other materials may be used. The wall and roof panels may be sealed respectively to the stud and rafter members, to preclude entry of dust, rain, insects, and/or other undesirable elements and to form a weather tight structure.