<b><i>Introduction:</i></b> The prevalence of hearing and visual impairment (HI and VI) and dual sensory impairment (DSI), which is a combination of both, is increasing as the population ages. These sensory impairments are expected to increase the cognitive load of information processing from hearing and vision and impair appropriate cognitive processing. Although an association between DSI and cognitive decline has been reported, a more detailed study of the effects on each cognitive domain is required. This study aimed to investigate the prevalence of self-reported sensory impairment in community-dwelling older adults and to examine the impact of DSI on the severity of mild cognitive impairment (MCI) and on each cognitive domain (memory, attention, executive function, and processing speed). <b><i>Methods:</i></b> The participants were recruited from a sub-cohort of the National Center for Geriatric Gerontology-Study on Geriatric Syndromes (NCGG-SGS) conducted by the National Center for Geriatrics and Gerontology. We included 4,471 community-dwelling older adults (age: 75.9 ± 4.3 years; females: 52.3%) who fulfilled the inclusion criteria. The HI and VI were identified using a self-report questionnaire. Cognitive and other parameters were also assessed by trained staff. Logistic regression analysis was used to evaluate the relationship between the presence of HI and VI and the severity of MCI, and functional decline in each cognitive domain. <b><i>Results:</i></b> DSI was identified in 11.4% of community-dwelling older adults. Regarding sensory impairment and MCI severity, the odds ratio (OR) for single-domain MCI was significantly higher in VI (OR: 1.31; 95% CI: 1.06–1.61), and the OR for multiple-domain MCI was significantly higher in DSI (OR: 1.58; 95% CI: 1.10–2.29). In relation to the four cognitive domains, ORs for impaired executive function were higher for VI and DSI (VI, OR: 1.37; 95% CI: 1.09–1.72. DSI, OR: 1.39; 95% CI: 1.06–1.81). DSI also exhibited a higher odds ratio for reduced processing speed (OR: 2.03; 95% CI: 1.42–2.91). <b><i>Discussion/Conclusion:</i></b> DSI is predicted to increase as the population ages and is associated with various health problems. Further, DSI has been reported to decrease quality of life, which needed to establish appropriate treatment and prevention measures.