In this paper we propose a new local learning based regression method which utilizes ensemble-learning as a form of regularization to reduce the variance of local estimators. This makes it possible to use local learning methods even with very high-dimensional datasets. The efficacy of the proposed method is illustrated on two publicly available high-dimensional sets in comparison with several global learning methods, and it is shown that the proposed ensemble-based local learning method significantly outperforms the global ones.