We measured the pressure dependence of polarized transmission spectra on AgGaS2 at 2 K. These spectra show an oscillatory behavior around the zero-birefringence energy E-0 where the ordinal and extraordinal refractive indices coincide. Using this oscillation we determined E-0 under hydrostatic pressures. From the difference of pressure dependences of E-0 and the energy gap for the upper valence band, it is expected that the pressure dependence of the energy gap for the next valence bands is smaller than that fur the upper one. Thus the tetragonal crystal field splitting decreases with increasing pressure. We discuss it in terms of p-d hybridization in the valence bands.