Clinical implication of a high ankle-brachial index (ABI) is not well known. Based on our previous study, we suspected that body composition may be a determinant of a high ABI and may consequently modulate the clinical significance of a high ABI. Datasets of two studies with independent cohorts, the anti-aging study cohort (n = 1765) and the Nagahama study cohort (n = 8,039), were analyzed in this study, in which appendicular muscle mass was measured by computed tomography and bioelectrical impedance analysis, respectively. Brachial and ankle blood pressures were measured using a cuff-oscillometric method. In the anti-aging study cohort, thigh muscle area (β = 0.387, p < 0.001), but not fat area, showed a strong positive association with the ABI independent of the body mass index (p = 0.662) and other possible covariates, including systolic brachial blood pressure (p = 0.054), carotid hypertrophy (p = 0.559), and arterial stiffness (β = 0.102, p = 0.001). This positive association was replicated in the Nagahama cohort. When the subjects were subdivided by the 75th percentiles of the ABI and appendicular muscle mass, multinomial logistic regression analysis identified insulin resistance as an independent determinant of an elevated ABI in subjects with normal muscle mass (coefficient = 0.134, p = 0.010), whereas insulin resistance was inversely associated with an elevated ABI in subjects with high muscle mass (coefficient = -0.268, p = 0.001). Appendicular muscle mass was a strong determinant of the ABI. The clinical background, particularly insulin resistance, of individuals with an elevated ABI may differ based on the amount of muscle mass.