This paper describes about the stable inverse filtering of the crosstalk canceller. The key issue is inverse filtering or magnitude and phase equalization of Head Related Transfer Function (HRTF). Inverse filtering, however, is formidable and can be instable, since the HRTF can often be the all-pass component. The locations of the HRTF-zeros can be closely to the unit circle in the complex frequency domain and those are highly sensitive to the location of source receiving positions even in a local area. If the inverse filter is utilized for sound reproduction at the target point, then receiving signals could be instable at other points even closely located to the target point. This paper investigates stable inverse filtering in a local area for the space-variant HRTF based on cepstrum decomposition of poles and zeros. From the results of computer simulation, we can understand that this smoothing inverse filtering is useful to get the original sound source in spite of the listener’s head movement.