Video holography has attracted attention after its invention in 1947; however, the enormous amount of data involved in recording and transmitting three-dimensional (3D) images remains a serious issue in electro-holography. Majority of the studies that have investigated holography transmission target the system that transmits the 3D images by compressing the holograms created on the distributor side using various compression techniques such as the conventional video compression techniques. However, the importance of the information in frequency space and characteristics, such as the correlation between adjacent pixels and frames, is different in natural images and holograms; therefore, these approaches are not always effective. In this study, we propose an effective electro-holography compression scheme based on the vector quantization of point light sources (PLSs). Instead of directly compressing a hologram, our method compresses and transmits PLSs from the distributor side and generates a hologram on the receiver side. To reduce the computational load that is required for creating a computer-generated hologram (CGH) on the receiver side, a fast CGH calculation technique has been developed for the vector-quantized PLS data based on the lookup tables (LUTs). This reduces the data rate by 76% when compared to that observed in case of uncompressed CGH transmission with 2K resolution and results in a calculation speed that is 1.34 times faster than that obtained using the conventional LUT method.