We present the GPU calculation with the common unified device architecture (CUDA) for the Swendsen-Wang multi-cluster algorithm of two-dimensional classical spin systems. We adjust the two connected component labeling algorithms recently proposed with CUDA for the assignment of the cluster in the Swendsen-Wang algorithm. Starting with the q-state Potts model, we extend our implementation to the system of vector spins, the q-state clock model, with the idea of embedded cluster. We test the performance, and the calculation time on GTX580 is obtained as 2.51 nsec per a spin flip for the q = 2 Potts model (Ising model) and 2.42 nsec per a spin flip for the q = 6 clock model with the linear size L = 4096 at the critical temperature, respectively. The computational speed for the q = 2 Potts model on GTX580 is 12.4 times as fast as the calculation speed on a current CPU core. That for the q = 6 clock model on GTX580 is 35.6 times as fast as the calculation speed on a current CPU core. (C) 2012 Elsevier B.V. All rights reserved.