Neurodegenerative changes in the preclinical stage of Alzheimer’s disease (AD) have recently been the focus of attention because they may present a range of treatment opportunities. A total of 134 elderly volunteers who lived in a local community were investigated and grouped into preclinical and mild cognitive impairment stages according to the Clinical Dementia Rating test; we also estimated amyloid deposition in the brain using positron emission tomography (PET). A significant interaction between clinical stage and amyloid PET positivity on cerebral atrophy was observed in the bilateral parietal lobe, parahippocampal gyri, hippocampus, fusiform gyrus, and right superior and middle temporal gyri, as previously reported. Early AD-specific voxel of interest (VOI) analysis was also applied and averaged Z-scores in the right, left, bilateral, and right minus left medial temporal early AD specific area were computed. We defined these averaged Z-scores in the right, left, bilateral, and right minus left early AD specific VOI in medial temporal area as R-MedT-Atrophy-score, L-MedT-Atrophy-score, Bil-MedT-Atrophy-score, and R_L-MedT-Atrophy-score, respectively. It revealed that the R_L-MedT-Atrophy-scores were significantly larger in the amyloid-positive than in the amyloid-negative cognitively normal (CN) elderly group, that is, the right medial temporal areas were smaller than left in amyloid positive CN group and these left-right differences were significantly larger in amyloid positive than amyloid negative CN elderly group. The L-MedT-Atrophy-score was slightly larger (p = 0.073), that is, the left medial temporal area was smaller in the amyloid-negative CN group than in the amyloid-positive CN group. Conclusively, the left medial temporal area could be larger in CN participants with amyloid deposition than in those without amyloid deposition. The area under the receiver operating characteristic curve for differentiating amyloid positivity among CN participants using the R_L-MedT-Atrophy-scores was 0.73; the sensitivity and specificity were 0.828 and 0.606, respectively. Although not significant, a negative correlation was observed between the composite cerebral standardized uptake value ratio in amyloid PET images and L-MedT-Atrophy-score in CN group. The left medial temporal volume might become enlarged because of compensatory effects against AD pathology occurring at the beginning of the amyloid deposition.