BACKGROUND: Cognitive dysfunction has been recognized as a diabetes-related complication. Whether hyperglycemia or elevated fasting glucose are associated with cognitive decline remains controversial. We aimed to investigate the relationship between fasting glucose levels and cognitive function in diabetic and non-diabetic individuals. METHODS: Participants were Japanese diabetic (n = 191) and non-diabetic (n = 616) men, aged 46-81 years, from 2010-2014. Blood samples were taken after a 12 h fast. The Cognitive Ability Screening Instrument (CASI), with a maximum score of 100, was used for cognitive assessment. Cognitive domains of CASI were also investigated. Fractional logit regression with covariate adjustment for potential confounders was used to model cross-sectional relationships between fasting blood glucose and CASI score. RESULTS: For diabetic individuals, CASI score was 0.38 (95% confidence interval: 0.66-0.12) lower per 1 mmol/L higher fasting glucose level. Short-term memory domain also exhibited an inverse association. For non-diabetic individuals, a reverse U-shaped relationship was observed between fasting glucose and cognitive function, identifying a threshold for highest cognitive performance of 91.8 CASI score at 3.97-6.20 mmol/L (71.5-111.6 mg/dL) fasting glucose. Language ability domain displayed a similar relationship with fasting glucose. CONCLUSIONS: Elevated fasting glucose levels in diabetic men were associated with lower cognitive function, in which short-term memory was the main associated domain. Interestingly, in non-diabetic men, we identified a threshold for the inverse relationship of elevated fasting glucose with cognitive function. Contrastingly to diabetic men, language ability was the main associated cognitive domain among non-diabetic men.