In this study, we fabricated optically transparent cotton fibers composite by resin impregnation without having to prepare cellulose nanofibers. Although the surface acetylation of the cotton microfibrils aided the resin impregnation process, excess acetylation resulted in the gradual breakdown of the cellulose crystal structure, with the resulting composite showing a high coefficient of thermal expansion (CTE). To prevent this, a swelling pretreatment, namely surface carboxylation by TEMPO-mediated oxidation, was performed to facilitate the surface acetylation of the microfibrils and hence the resin impregnation process. After the swelling pretreatment and the subsequent surface acetylation process, an optically transparent cotton fiber composite with a high regular light transmittance (83.23% at 600 nm) and a low CTE (19.5 ppm/K) was obtained. Finally, when this technique was used with a commercial cotton cloth, a transparent composite with a high total transmittance (88.5% at 600 nm) could be obtained.