The rumen microbiota comprises a vast range of bacterial taxa, which may affect the production of high-quality meat in Japanese Black cattle. The aim of this study was to identify core rumen microbiota in rumen fluid samples collected from 74 Japanese Black cattle raised under different dietary conditions using 16S rRNA gene amplicon sequencing. In the rumen of fattening Japanese Black cattle, 10 bacterial taxa, showing >1% average relative abundance and >95% prevalence, irrespective of the dietary conditions and the fattening periods, were identified as the core rumen bacterial taxa, which accounted for approximately 80% of the rumen microbiota in Japanese Black cattle. Additionally, population dynamics of the core rumen bacterial taxa revealed two distinct patterns: Prevotella spp. and unclassified Bacteroidales decreased in the mid-fattening period, whereas unclassified Clostridiales, unclassified Ruminococcaceae, Ruminococcus spp., and unclassified Christensenellaceae increased during the same period. Therefore, the present study reports the wide distribution of the core rumen bacterial taxa in Japanese Black cattle, and the complementary nature of the population dynamics of these core taxa, which may ensure stable rumen fermentation during the fattening period.