(1) Background: Atypical auditory perception has been reported in individuals with autism spectrum disorder (ASD). Altered auditory evoked brain responses are also associated with childhood ASD. They are likely to be associated with atypical brain maturation. (2) Methods: This study examined children aged 5–8 years old: 29 with ASD but no intellectual disability and 46 age-matched typically developed (TD) control participants. Using magnetoencephalography (MEG) data obtained while participants listened passively to sinusoidal pure tones, bilateral auditory cortical response (P1m) was examined. (3) Results: Significantly shorter P1m latency in the left hemisphere was found for children with ASD without intellectual disabilities than for children with TD. Significant correlation between P1m latency and language conceptual ability was found in children with ASD, but not in children with TD. (4) Conclusions: These findings demonstrated atypical brain maturation in the auditory processing area in children with ASD without intellectual disability. Findings also suggest that ASD has a common neural basis for pure-tone sound processing and language development. Development of brain networks involved in language concepts in early childhood ASD might differ from that in children with TD.