In recent years, various rescue robots have been developed to investigate in disaster area, for example, Fukushima No.1 Nuclear Power Station. Particularly, it has been for investigation in a narrow pipe and under debris that snake robots, active scope camera and others have been developed. We have paid our attention to multibody robots resembling earthworms' muscle. This robot is good at moving in narrow space and on rough ground. However, there is a problem that it is not enough to examine what kind of movements are appropriate depending on the environment. This study is aimed at simulating forward movement in a pipe by using Q-learning, which is one of the reinforcement learning. Moreover, we compare its result with the one of the real robot.