Japanese cedar (Cryptomeria japonica) is the most important timber species in Japan; however, its pollen is the primary cause of pollinosis in Japan. The total number of pollen grains produced by a single tree is determined by the number of male strobili (male flowers) and the number of pollen grains per male strobilus. While the number of male strobili is a visible and well-investigated trait, little is known about the number of pollen grains per male strobilus. We hypothesized that genetic and environmental factors affect the pollen number per male strobilus and explored the factors that affect pollen production and genetic variation among clones. We counted pollen numbers of 523 male strobili from 26 clones using a cell counter method that we recently developed. Piecewise Structural Equation Modeling (pSEM) revealed that the pollen number is mostly affected by genetic variation, male strobilus weight, and pollen size. Although we collected samples from locations with different environmental conditions, statistical modeling succeeded in predicting pollen numbers for different clones sampled from branches facing different directions. Comparison of predicted pollen numbers revealed that they varied >3-fold among the 26 clones. The determination of the factors affecting pollen number and a precise evaluation of genetic variation will contribute to breeding strategies to counter pollinosis. Furthermore, the combination of our efficient counting method and statistical modeling will provide a powerful tool not only for Japanese cedar but also for other plant species.