For flexible interactions between a robot and humans, we address the issue of automatic recognition of human emotions during the interaction such as embarrassment, pleasure, and affinity. To construct classifiers of emotions, we used the dialogue data between a humanoid robot, Robovie, and children, which was collected with the WOZ (Wizard of Oz) method. Besides prosodic features extracted from a single utterance, characteristics specific to dialogues such as utterance intervals and differences with previous utterances were also used. We used the SVM (Support Vector Machine) as a classifier to recognize two temporary emotions such as embarrassment or pleasure, and the decision tree learning algorithm, C5.0, as a classifier to recognize persistent emotion, i.e. affinity. The accuracy of classification was 79% for embarrassment, 74% for pleasure, and 87% for affinity. The humanoid Robovie in which this emotion classification module was implemented demonstrated adaptive behaviors based on the emotions it recognized.