Children make rapid transitions in their neural and intellectual development. Compared to other brain regions, the auditory cortex slowly matures, and children show immature auditory brain activity. This auditory neural plasticity largely occurs as a response to human-voice stimuli, which are presented more often than other stimuli, and can even be observed in the brainstem. Early psychologists have proposed that sensory processing and intelligence are closely related to each other. In the present study, we identified brain activity related to human-voice processing and investigated a crucial neural correlate of child development and intelligence. We also examined the neurophysiological activity patterns during human-voice processing in young children aged 3 to 8 years. We investigated auditory evoked fields (AEFs) and oscillatory changes using child-customized magnetoencephalography within a short recording time (<6 min). We examined the P1m component of AEFs, which is a predominant component observed in young children. The amplitude of the left P1m was highly correlated with age, and the amplitude of the right P1m was highly correlated with the intelligence quotient. For auditory-related oscillatory changes, we found a positive correlation between the intelligence quotient and percent change of gamma increase relative to baseline in the right auditory cortex. We replicated the finding of age-related changes in auditory brain activity in young children, which is related to the slow maturation of the auditory cortex. In addition, these results suggest a close link between intelligence and auditory sensory processing, especially in the right hemisphere.