The present invention subdivides a growing data collection into individual compartments while preserving the overall logical data context. The data is stored as information units, and each information unit comprises a set of data elements and a set of index elements. The information units are classified information units into a plurality of information unit types. The index elements are classified into a plurality of index classes. The information unit types associated with a query are determined and a search scope is determined based on the information unit types. The query is processed by generating subqueries referencing the index classes corresponding to the information unit types.