In this study, the robot system which follows the movement of the human hand is suggested. In the system, the movement of the human hand is obtained using the Kinect sensor, and the three-dimensional position of the human hand is send to the robot. The robot generates the reference trajectory based on the position which has been sent from the Kinect sensor. The position of the human hand is converted into the smooth motion using KANSEI transfer function as the reference position, and the robot follows the smooth reference position. As a result, the system which followed the movement of the hand smoothly is realized.