© 2018 IEEE. In this paper, we propose a new approach to classify ovarian follicles into two classes. A smoothing filter which is designed to consider speckle patterns under the resolution of the ultrasound devices is applied for filtering ovarian follicle images. Then, convolutional neural networks are used for extracting features from the filtered ovarian follicle images. Finally, both features extracted by CNNs from the filtered ovarian follicle images and numerical features defined by our previous works are used for classification. From experimental results, we show the effectiveness of our proposed method.