We investigate correlations of the Polyakov loop fluctuations with eigenmodes of the lattice Dirac operator. Their analytic relations are derived on the temporally odd-number size lattice with the normal nontwisted periodic boundary condition for the link variables. We find that the low-lying Dirac modes yield negligible contributions to the Polyakov loop fluctuations. This property is confirmed to be valid in confined and deconfined phases by numerical simulations in SU(3) quenched QCD. These results indicate that there is no direct, one-to-one correspondence between confinement and chiral symmetry breaking in QCD in the context of different properties of the Polyakov loop fluctuation ratios.