The cerebellum is derived from the dorsal part of the anterior-most hindbrain. The vertebrate cerebellum contains glutamatergic granule cells (GCs) and gamma-aminobutyric acid (GABA) ergic Purkinje cells (PCs). These cerebellar neurons are generated from neuronal progenitors or neural stem cells by mechanisms that are conserved among vertebrates. However, vertebrate cerebella are widely diverse with respect to their gross morphology and neural circuits. The cerebellum of cyclostomes, the basal vertebrates, has a negligible structure. Cartilaginous fishes have a cerebellum containing GCs, PCs, and deep cerebellar nuclei (DCNs), which include projection neurons. Ray-finned fish lack DCNs but have projection neurons termed eurydendroid cells (ECs) in the vicinity of the PCs. Among ray-finned fishes, the cerebellum of teleost zebrafish has a simple lobular structure, whereas that of weakly electric mormyrid fish is large and foliated. Amniotes, which include mammals, independently evolved a large, foliated cerebellum, which contains massive numbers of GCs and has functional connections with the dorsal telencephalon (neocortex). Recent studies of cyclostomes and cartilaginous fish suggest that the genetic program for cerebellum development was already encoded in the genome of ancestral vertebrates. In this review, we discuss how alterations of the genetic and cellular programs generated diversity of the cerebellum during evolution.