The gravitational baryogensis may not generate a sufficient baryon asymmetry in the standard thermal history of the Universe when we take into account the gravitino problem. Hence, it has been suggested that anisotropy of the Universe can enhance the generation of the baryon asymmetry through the increase of the time change of the Ricci scalar curvature. We study the gravitational baryogenesis in the presence of anisotropy, which is produced at the end of an anisotropic inflation. Although we confirm that the generated baryon asymmetry is enhanced compared with the original isotropic cosmological model, taking into account the constraint on the anisotropy by the recent CMB observations, we find that it is still difficult to obtain the observed baryon asymmetry only through the gravitational baryogenesis without suffering from the gravitino problem.