This paper presents a new type of algorithm for solving independent component analysis (ICA) problems. Instead of being based on additive updating, which is used in conventional algorithms, this new algorithm is based on an effective updating scheme in which learning updating acts as a series of orthonormal matrix transformations (i.e., power iteration). The criterion for the independence between outputs is based on diagonality of a non-linearized covariance matrix, which is defined by ICA outputs and their non-linear mappings, and the Bussgang property. One attractive feature of the algorithm is that it does not include any predetermined parameters, such as a learning step size, as do gradient-based algorithms, which is especially expected for ICA applications with unknown types of sources (but with the condition that at most one source is Gaussian distributed). Another feature is that the convergence rate is faster, even for very short observations. If the same ICA criteria are applied to the proposed and gradient-based algorithms; the relationship between these algorithms is quite similar to the relationship between the least-mean-square (LMS) algorithm and the recursive least-square (RLS) algorithm in the batch mode for supervised adaptive filtering. In this paper, we also analyze the algorithm mathematically to determine why and how the algorithm works. We show that learning updating in the form of matrix transformation permits finite-scale learning, and at the same time, maintains the orthonormal property of the separation matrix. This is essentially different from the case with gradient-based algorithms, which permits only a small-scale learning that is controlled by the learning step size. We also analyze the relationship between the new algorithm with other well-known algorithms, such as the Bussgang algorithm, the non-linear principal component analysis (PCA), and the FastICA.