A novel system configuration for repeaterless transmission in conjunction with optical booster amplifiers is proposed. A nonlinear propagation section is introduced in the transmission fiber for the suppression of stimulated Brillouin scattering (SBS) and Raman amplification. The basic characteristics of the proposed nonlinear propagation section are clarified experimentally. It is confirmed that the modulation index of the continuous-phase frequency shift keying (CPFSK) scheme can be selected without SBS restriction. A transmission experiment is conducted using a 364.3-km nondispersion shifted fiber and an erbium-doped fiber (EDF) amplifier with an output of about +20 dBm to test the proposed system configuration. Further, the CPFSK modulation conditions are clarified to design repeaterless long span systems from the viewpoints of the maximum input power as restricted by SBS, LD linewidth, and chromatic dispersion in transmission fiber.