論文

査読有り
2017年

A Generator of Hadoop MapReduce Programs that Manipulate One-dimensional Arrays

J. Inf. Process.
  • Reina Miyazaki
  • ,
  • Kiminori Matsuzaki
  • ,
  • Shigeyuki Sato

25
開始ページ
841
終了ページ
851
記述言語
英語
掲載種別
研究論文(学術雑誌)
DOI
10.2197/ipsjjip.25.841
出版者・発行元
Information Processing Society of Japan

MapReduce is a framework for large-scale data processing proposed by Google, and its open-source implementation, Hadoop MapReduce, is now widely used. Several language systems have been proposed to make developing MapReduce programs easier, for instance, Sawzall, FlumeJava, Pig, Hive, and Crunch. These language systems mainly target applications that can be naturally solved by using a MapReduce-like programming model. In this study, we propose a new MapReduce-program generator that accepts programs manipulating one-dimensional arrays. By using the proposed generator, users only need to write sequential programs to generate Hadoop MapReduce programs automatically. We applied some program optimization techniques to the generation of Hadoop MapReduce programs. In this paper, we also report our experiment results that compare programs generated by the proposed generator with hand-written MapReduce programs.

リンク情報
DOI
https://doi.org/10.2197/ipsjjip.25.841
DBLP
https://dblp.uni-trier.de/rec/journals/jip/MiyazakiMS17
URL
http://dblp.uni-trier.de/db/journals/jip/jip25.html#journals/jip/MiyazakiMS17
ID情報
  • DOI : 10.2197/ipsjjip.25.841
  • ISSN : 1882-6652
  • ISSN : 0387-5806
  • DBLP ID : journals/jip/MiyazakiMS17
  • ORCIDのPut Code : 92041477
  • SCOPUS ID : 85040936096

エクスポート
BibTeX RIS