Background: Although physical inactivity is a major public health problem, the causative factors for physical inactivity per se are poorly understood. To address this issue, we investigated the relationship between deep white matter lesions (DWMLs) on magnetic resonance imaging, apathy, and physical activities using structural equation modeling (SEM). Methods: We examined 317 community-dwelling elderly subjects (137 men and 180 women with a mean age of 64.5 years) without dementia or clinically apparent depression. Physical activity was assessed with a questionnaire consisting of 3 components (leisure-time, work, and sport activities). Results: The mean score from the apathy scale (a visual analogue version of Starkstein's apathy scale) of the Grades 2-3 DWML group was 420 (95% confidence interval [CI] 379-461), which was lower (more apathetic) than the Grade 0 DWML group score of 478 (95% CI 463-492) after adjustment for education as a covariate. SEM showed that the direct paths from DWMLs or education to apathy were significant, and the direct path from apathy to leisure-time activity was highly significant (beta =.25, P < .001). The degree of apathetic behavior was negatively associated with sport activity in female subjects and positively associated with TV watching in male subjects. Conclusions: The results of the study show that DWMLs are one of the major factors that cause apathetic behavior and that apathy has significant negative effects on leisure-time physical activity in community-dwelling elderly subjects. Even a minor level of apathy without major depression would have a significant impact on activities of daily living and quality of life.