In recent years, automatic video caption generation has attracted
considerable attention. This paper focuses on the generation of Japanese
captions for describing human actions. While most currently available video
caption datasets have been constructed for English, there is no equivalent
Japanese dataset. To address this, we constructed a large-scale Japanese video
caption dataset consisting of 79,822 videos and 399,233 captions. Each caption
in our dataset describes a video in the form of "who does what and where." To
describe human actions, it is important to identify the details of a person,
place, and action. Indeed, when we describe human actions, we usually mention
the scene, person, and action. In our experiments, we evaluated two caption
generation methods to obtain benchmark results. Further, we investigated
whether those generation methods could specify "who does what and where."