This paper presents a framework of the hierarchical transfer learning (HTL) for a heterogeneous multi-robot transfer learning method utilizing of cloud-computing resources. A multi-agent robot system (MARS) that utilizes reinforcement learning and transfer learning methods has recently been deployed in real-world situations. In MARS, autonomous agents obtain behaviors autonomously through multi-agent reinforcement learning and the transfer learning method enables the reuse of the knowledge of other robots' behavior, such as for cooperative behavior. These methods, however, have not been fully and systematically discussed. In prior research, we developed an HTL method and investigated its effectiveness in a dynamic multi-agent environment. The HTL method hierarchically abstracts obtained knowledge by ontological methods. Here, we evaluate the effectiveness of HTL in heterogeneous multi-agent situation with action ontology by conducting a computer simulation.