Although speaking tests have become more important in Japan over the past decades with the recent emphasis on communicative language teaching, they remain infrequent at most schools. This is because assessing speaking skills is quite difficult in terms of its practicality. However, since MEXT (2014b) announced that students' English proficiency and learning status in terms of the four language skills will be assessed and analyzed even in entrance examination in the near future, speaking test development can be on urgent business. There are two main types of test formats toassess speaking skills so far: a semi-direct, or person-to-machine, test, and a direct, or face-to-face, test. This study investigated whether the speaking ability measures would make a difference depending on two testing formats. The results showed there was no statistically significant difference between two test formats. However, according to questionnaire results, test takers preferred direct testing to semi-direct testing.