Japanese chestnut (Castanea crenata Sieb. et Zucc.) has a long juvenile phase, so breeders have to wait many years to evaluate nut traits. Molecular markers associated with genes of interest would accelerate selection in chestnut breeding programs. We evaluated five nut traits (nut harvest date, nut weight, pericarp splitting, insect infestation, and specific gravity) in 99 Japanese chestnut cultivars and selections. A wide range of phenotypic variation was observed for each of the traits, suggesting that the collection harbored sufficient genetic diversity for breeding. A Bayesian genome-wide association study was conducted using 162 simple sequence repeat markers and 741 single nucleotide polymorphism markers. To evaluate the potential of marker-assisted selection, we examined 12 molecular markers found to be associated with nut traits: 4 for nut harvest date, 4 for nut weight, 1 for pericarp splitting, and 3 for insect infestation. The percentages of phenotypic variance explained ranged from 4.8 to 37.1%. Although insect infestation showed only medium heritability (0.672), we obtained two quantitative trait loci (QTLs) with extremely high posterior probabilities (0.93 and 1.00). Out of the 12 molecular markers, 3 of the 4 markers for nut harvest time could be validated in a breeding population. Accuracies of genomic selection were extremely high for nut harvest date (0.841) and moderate for insect infestation (0.604), suggesting that genomic selection based on Bayesian regression would reduce the cost of phenotypic evaluation of these traits and possibly others.