In this research, we propose a new index of emotional arousal level using sound pressure change acceleration, called the emotional arousal level voice index (EALVI), and investigate the relationship between this index and depression severity. First, EALVI values were calculated from various speech recordings in the interactive emotional dyadic motion capture database, and the correlation with the emotional arousal level of each voice was examined. The resulting correlation coefficient was 0.52 (n = 10,039, p < 2.2 × 10−16). We collected a total of 178 datasets comprising 10 speech phrases and the Hamilton Rating Scale for Depression (HAM-D) score of outpatients with major depression at the Ginza Taimei Clinic (GTC) and the National Defense Medical College (NDMC) Hospital. The correlation coefficients between the EALVI and HAM-D scores were − 0.33 (n = 88, p = 1.8 × 10−3) and − 0.43 (n = 90, p = 2.2 × 10−5) at the GTC and NDMC, respectively. Next, the dataset was divided into “no depression” (HAM-D < 8) and “depression” groups (HAM-D ≥ 8) according to the HAM-D score. The number of patients in the “no depression” and “depression” groups were 10 and 78 in the GTC data, and 65 and 25 in the NDMC data, respectively. There was a significant difference in the mean EALVI values between the two groups in both the GTC and NDMC data (p = 8.9 × 10−3, Cliff’s delta = 0.51 and p = 1.6 × 10−3; Cliff’s delta = 0.43, respectively). The area under the curve of the receiver operating characteristic curve when discriminating both groups by EALVI was 0.76 in GTC data and 0.72 in NDMC data. Indirectly, the data suggest that there is some relationship between emotional arousal level and depression severity.