Understanding the geological modification processes on asteroids is fundamental for elucidating their surface evolution. Images of small asteroids from spacecrafts show a depletion in smaller craters. Seismic shaking was considered to be responsible for erasing such small craters and the main driver modifying the geology of asteroids. However, a recent artificial impact experiment on asteroid Ryugu conducted by the Japanese Hayabusa2 spacecraft revealed minimal seismic efficiency on the asteroid. To investigate whether a standard seismic shaking model can reproduce the observed crater record, we analyzed the crater distributions on four asteroids, i.e., Eros, Itokawa, Ryugu, and Bennu, using crater production functions under cohesionless conditions. Crater retention ages were estimated as a function of crater diameter for the four asteroids using the crater size-frequency distribution and crater production function estimated for each asteroid. We obtained the relation between a crater retention age t and crater diameter D in the form of power-law function (t ∝ Da).We found that the power-law indices a are inconsistent with diffusion processes (e.g., seismic shaking, where a = 2). This result suggests that seismic shaking models based on a linear diffusion equation cannot explain the crater distribution on the small asteroids. Alternative crater obliteration processes include surface flows suggested by geomorphological and spectral features of Ryugu. Using the crater statistics, we also show that the vertical mixing of the Ryugu material at the depths shallower than 1 m occurs in 103–105 yr by cratering. This rapid resurfacing and replacement of the surface layer in the short timescale would account for the decrease in space weathering rate suggested by previous studies. Furthermore, the timescale required for vertically transporting Ryugu materials to the depths of 2–4 m (104–106 yr), where cosmic rays would not reach, can be compared with the cosmic-ray exposure ages of returned samples. This comparison can be used to constrain the distribution of impactors that collide with Ryugu.