Secure inner product, namely the computation of inner product whose terms are all in encrypted form, is the central technique for various privacy-preserving applications. In this paper, we propose a generic yet efficient method to compute secure inner products of vectors (or matrices) using matrix trace properties. Indeed, our method not only applies to both LWE-based and ring-LWE-based homomorphic encryption schemes, but also is more efficient compared to previously known methods.