Nonlinear damping with respect to vibration amplitude is particularly important in mechanical dynamics. The addition of short fibers to damping materials is considered to result in strong nonlinear damping due to interfacial peeling at the edges of the fibers. However, little has been reported on the occurrence of nonlinear damping in short-fiber reinforced rubber due to compounding difficulties. In this study, we investigated the relationship between the damping characteristics and deformation behavior of microdeformed short-fiber reinforced rubber by X-ray computed tomography (CT). We prepared a damping material with a natural rubber (NR) matrix and micrometer-sized polyethylene terephthalate (PET) fiber filler. The loss factor was identified by dynamic mechanical analysis, and three-dimensional strain maps were obtained using marker tracking in the CT data. The addition of 5 wt% PET fibers to NR resulted in an increase in the loss factor. Experimentally, we found that the nonlinear damping of the composite rubber is affected by the peeling of the filler/matrix interface and the strain inside the material.