The possible effects of a neighborhood's built environment on physical activity have not been studied in Asian countries as much as in Western countries. The present study cross-sectionally examined the relationship between geographic information system (GIS) measured residence and worksite neighborhood walkability, and the number of parks/green spaces and sports facilities within a 1 km radius of home and workplace, with self-reported leisure-time habitual (3-4 times per week or more) walking and moderate-to-vigorous intensity habitual exercise among local government workers aged 18 to 64 years living in an urban-suburban area of Aichi, Japan in 2013. A single-level binomial regression model was used to estimate the multivariable odds ratios (ORs) and 95% confidence intervals (95% CIs). Of the 1959 male and 884 female participants, 288 (15%) and 141 (16%) reported habitual walking, respectively, and 18% and 17% reported habitual exercise, respectively. Compared with women who resided in neighborhood with a walkability index of 4-30, those living in an area with that of 35-40 were significantly more likely to engage in leisure-time habitual exercise (multivariable OR: 1.70, 95% CI: 1.08-2.68). Marginally significant positive associations were found between leisure-time habitual exercise and the residential neighborhood's number of parks/green spaces among women, as well as the number of sports facilities among men. In conclusion, a residential neighborhood environment characterized by higher walkability may contribute to the initiation or maintenance of moderate-to-vigorous intensity leisure-time exercise among working women living in an urban-suburban area of Japan.