A simulation analysis and real phenotype analysis were performed to evaluate the impact of three different relationship matrices on heritability estimation and prediction accuracy in a closed-line breeding of Duroc pigs. The numerator relationship matrix (NRM), single nucleotide polymorphism (SNP)-based genomic relationship matrix (GRM) (G(S)), and haplotype-based GRM (GH) were applied in this study. We used PorcineSNP60 genotype array data (38 114 SNPs) of 831 Duroc pigs with four selection traits. In both heritability estimation and prediction accuracy, the accuracy depended on the number of animals with records. For heritability estimation, a large difference in the results among three relationship matrices was not shown, but the trend of the estimated heritabilities between GRMs, that is G(S) < G(H), was shown in this population. For the accuracy of prediction values in test animals, the accuracies of prediction values obtained by two GRMs were higher than that by the NRM in this population. The accuracies obtained by GRMs using animals with no records were lower than that by the NRM using animals with their performance records, but were close to that by the NRM using animals with full-sib testing records.