Multi-parent advanced generation inter-cross (MAGIC) lines have broader genetic variation than bi-parental recombinant inbred lines. Genome-wide association study (GWAS) using high number of DNA polymorphisms such as single-nucleotide polymorphisms (SNPs) is a popular tool for allele mining in MAGIC populations, in which the associations of phenotypes with SNPs are investigated; however, the effects of haplotypes from multiple founders on phenotypes are not considered. Here, we describe an improved method of allele mining using the newly developed Japan-MAGIC (JAM) population, which is derived from eight high-yielding rice cultivars in Japan. To obtain information on the haplotypes in the JAM lines, we predicted the haplotype blocks in the whole chromosomes using 16,345 SNPs identified via genotyping-by-sequencing analysis. Using haplotype-based GWAS, we clearly detected the loci controlling the glutinous endosperm and culm length traits. Information on the alleles of the eight founders, which was based on the effects of mutations revealed by the analysis of next-generation sequencing data, was used to narrow down the candidate genes and reveal the associations between alleles and phenotypes. The haplotype-based allele mining (HAM) proposed in this study is a promising approach to the detection of allelic variation in genes controlling agronomic traits in MAGIC populations.