A hearing aid system utilizes digital signal processing to correct for the hearing deficit of a particular user and to maximize the intelligibility of the desired audio signal relative to noise. An analog signal from a microphone is converted to digital data which is operated on by a digital signal processor, with the output of the digital signal processor being converted back to an analog signal which is amplified and provided to the user. The digital signal processor includes a time varying spectral filter having filter coefficients which can be varied on a quasi-real time basis to spectrally shape the signal to match the hearing deficit of the user and to accommodate ambient signal and noise levels. The coefficients of the spectral filter are determined by estimating the energy in several frequency bands within the frequency range of the input signal, and using those energy estimates to calculate desired gains for the frequency bands and corresponding spectral filter coefficients. The spectral energy analysis may be carried out using pairs of high pass and low pass digital filters in cascade relation, with the output of each low pass filter being provided to the next pair of high pass and low pass filters. The rate at which output data is provided from the filters in each pair may be reduced from the sample rate of input data by one half for succeeding pairs of filters in the cascade to thereby reduce the computation time required.