We present a novel approach to enhance the spontaneous emission rate of single quantum emitters in an optical nanofiber-based cavity by introducing a narrow air-filled groove into the cavity. Our results show that the Purcell factor for single quantum emitters inside the groove of the nanofiber-based cavity can be at least six times greater than for such an emitter on the fiber surface when using an optimized cavity mode and groove width. Moreover, the coupling efficiency of single quantum emitters into the guided mode of this nanofiber-based cavity can reach up to ∼80% with only 35 cavity-grating periods. This new system has the potential to act as an all-fiber platform to realize efficient coupling of photons from single emitters into an optical fiber for quantum information applications.