The use of a novel motion compensated spatio-temporal nonseparable filter (MCSTF) is proposed for scalable video coding. This filter is an alternative to the present motion-compensated temporal filtering (MCTF), a fundamental component for scalable video coding in the next generation. MCTF improve the coding efficiency and provide the temporal scalability. We classify MCSTF as a non-separable sub-sampling version of MCTF, and it provides interlaced pictures as intermediate video sequence by using a spatio-temporal split process. Furthermore, its 1/3-transform structure, which excludes the lifting-update-step, significantly suppresses the PSNR fluctuation that occurs when the existing MCTF technique is used. Our proposed system suppresses PSNR fluctuation with a higher average PSNR than the existing MCTF. Two types of sub-sampling lattices were investigated: the vertical-temporal (VT) quincunx and the face-centered orthorhombic (FCO) ones. Experimental results with entropy coded scalar quantization show the signifaicance of our proposed technique.