Recently, due to large amount of available machine translation services, selecting proper translation service is getting important. However, the problem is that service selection based on evaluation of the translation accuracy is too expensive to evaluate them for all translation services. In this work, we propose the method to obtain comparison of translation accuracy by crowdsourcing and the ranking of from comparison data. The correlation between the rankings by proposed method and professional accuracy showed the feasibility of our method.