We revisit the constraints on the small scale density perturbations
($10^4\,\mathrm{Mpc}^{-1}\lesssim k \lesssim10^5\,\mathrm{Mpc}^{-1}$) from the
modification of the freeze-out value of the neutron-proton ratio at big-bang
nucleosynthesis era. Around the freeze-out temperature $T\sim
0.5\,\mathrm{MeV}$, the universe can be divided into several local patches
which have different temperatures since any perturbation which enters the
horizon after the neutrino decoupling has not diffused yet. Taking account of
this situation, we calculate the freeze-out value in detail. We find that the
small scale perturbations decrease the n-p ratio in contrast to previous works.
With use of the latest observed $^4$He abundance, we obtain the constraint on
the power spectrum of the curvature perturbations as
$\Delta^2_\mathcal{R}\lesssim 0.018$ on $10^4\,\mathrm{Mpc}^{-1}\lesssim k
\lesssim 10^5\,\mathrm{Mpc}^{-1}$.