An endoscope is a medical instrument that acquires images inside the human body. This paper proposes a new approach for the automatic detection of polyp regions in an endoscope image by generating a likelihood map with both of edge and color information to obtain high accuracy so that probability becomes high at around polyp candidate region. Next, Histograms of Oriented Gradients (HOG) features are extracted from the detected region and random forests are applied for the classification to judge whether the detected region is polyp region or not. It is shown that the proposed approach has high accuracy for the polyp detection and the usefulness is confirmed through the computer experiments with endoscope images.