A study was conducted to propose a new cross-synthesis framework based on an interference-free representation of a power spectrum combined with normalization and modulation transfer function design for spectral envelope preprocessing of speech sounds. The proposed cross-synthesis enabled control of the linguistic information and the timbre identity. The spectral envelope of speech was extracted in the proposed method using a F0-adaptive procedure called TANDEM-STRAIGHT. It was demonstrated that the procedure effectively removed interference caused by periodic excitation from the spectrogram of the speech and yielded a smooth representation. A two-staged procedure was also introduced to remove the timbre-modifying components from the speech spectral envelope. The primary procedure involved the approximation of the global spectral shape and the secondary one was the filtering of temporal modulations.