This study investigated the differences in first-language-based (L1-based) phonetic processing for second language (L2) phonemes among different age groups of adults. A speech-in-speech masking paradigm was utilized to examine the contribution of the L1-based processing. A phoneme identification task in one language was conducted in the presence or absence of an interferer of a masker of the same or a different language. The degree of interference (i.e., the decrease in identification performance) was postulated to increase as the similarity of underlying processes for the target and masker increases. Experiment 1 was conducted to test the effectiveness of the paradigm. As expected, the interference increased as the similarity of underlying processes for the target and masker increased. Experiment 2 examined the perception of English /r/–/l/ and other phonetic contrasts by Japanese listeners in various adult age groups, to examine whether the degree of interference differs depending on the putative degrees of L1-based processing and on age. The results demonstrated such differences and showed that the L1-based processing can be estimated from the decrease in the identification performance. They also suggested that the perception of /r/–/l/ in the initial singleton and initial cluster positions was high L1-based in older adults.