Parallel system with distributed memory is a promising platform to achieve a high performance computing with less construction cost. Applications with few communications, such as a kind of parameter sweep applications (PSA), can be efficiently carried out on such a parallel system, but some applications are not suitable for the parallel system due to a large communication cost. We focus on PNN (Pairwise Nearest Neighbor) codebook generation algorithm for VQ (Vector Quantization) compression algorithm and proposed a parallel version of the PNN algorithm suitable for the parallel system with distributed memory, called "multi-step parallel PNN". However, the computational complexity of the second half of this algorithm increases as the number of worker computers grows, so we apply a tree-distribution implementation to the second half in order to cope with this problem. We call this approach "reduced parallel PNN". We confirm the effectiveness of the reduced parallel PNN by the evaluation of the computational complexity of the algorithm and the experiment executed on a PC cluster system and a PC Grid system. Copyright 2009 ACM.