© 2019, ICIC International. All rights reserved. Surface inspection relying on computer vision has been widely used in many fields. However, the existing computer vision-based industrial parts surface defect detection methods mostly adopt an image registration algorithm. This method is limited by environmental factors, and the image preprocessing process is complex. On the other hand, with the rapid development of deep learning, the appearance of Convolutional Neural Network (CNN) has led to the rapid development of computer vision research based on deep learning. CNNs have excellent performance for image processing and do not require a manual image extraction feature. In this study, the two type CNNs are trained through a large number of pictures, and then they are integrated to an ensemble CNN for the surface defect detection, and encouragement results are obtained.