Complex permittivity spectra were obtained herein by performing broadband terahertz (THz) spectroscopy on cellulose, paramylon, and paramylon ester. Absorption peaks observed for cellulose and paramylon at approximately 3 THz are attributed to hydrogen bonds. In addition, a broad absorption peak around 2 THz was observed for all the polymers, demonstrating a general feature of polymer glasses derived from weak interatomic van der Waals forces. The boson peak was observed for cellulose and paramylon ester. The boson peak frequency for cellulose nearly equaled that for glassy glucose-a unit structure of the cellulose polymer. Additionally, the insensitivity of cellulose to the polymerization degree was consistent with recent results obtained via molecular dynamics simulations. In contrast, the boson peak frequency of paramylon ester was markedly smaller than that of cellulose. These results demonstrate the importance of hydrogen bonds as determinants of the boson peak frequency.