Frequency delta-sigma modulator (FDSM) employing a variable frequency oscillator is a novel replacement of the classical delta-sigma modulators. This is advantageous for application to sensors, because an ADC can be intrinsically integrated with the sensors. We have already proposed to use this technique to various sensors. However, the signal-to-noise ratio was significantly degraded by noise floor, in the previous papers. In this paper, we have investigated the origin of the noise floor in the FDSM microphone sensors as a promising example. It was demonstrated that improving the phase noise of the oscillator can drastically reduce the noise floor. For this reduction we improved the Q-factor of the cavity resonator, and the design of the oscillator circuit. With these improvements, the phase noise, and, hence, the noise floor, were improved by approximately 40 dB. In addition, we obtained an SNR of 57 dB for 114 dBSPL sound input with 96 kHz bandwidth, which corresponds to the dynamic range of 87 dB for maximum 140 dBSPL. A much larger dynamic range of around 120 dB is expected by increasing the sampling rate and decreasing the Al diaphragm thickness. These results also indicate the promise of the FDSM to varieties of physical sensors.