The effectiveness of defect prediction depends on modeling techniques as well as their parameter optimization, data preprocessing and ensemble development. This paper focuses on auto-sklearn, which is a recently-developed software library for automated machine learning, that can automatically select appropriate prediction models, hyperparameters and data preprocessing techniques for a given data set and develop their ensemble with optimized weights. In this paper we empirically evaluate the effectiveness of auto-sklearn in predicting the number of defects in software modules. In the experiment, we used software metrics of 20 OSS projects for cross-release defect prediction and compared auto-sklearn with random forest, decision tree and linear discriminant analysis by using Norm(Popt) as a performance measure. As a result, auto-sklearn showed similar prediction performance as random forest, which is one of the best prediction models for defect prediction in past studies. This indicates that auto-sklearn can obtain good prediction performance for defect prediction without any knowledge of machine learning techniques and models.