This paper presents a framework, called the knowledge co-creation framework (KCF), for the heterogeneous multi-robot transfer learning method with utilization of cloud-computing resources. A multi-agent robot system (MARS) that utilizes reinforcement learning and transfer learning methods has recently been deployed in real-world situations. In MARS, autonomous agents obtain behavior autonomously through multi-agent reinforcement learning and the transfer learning method enables the reuse of the knowledge of other robots' behavior, such as for cooperative behavior. These methods, however, have not been fully and systematically discussed. To address this, KCF leverages the transfer learning method and cloud-computing resources. In prior research, we developed a hierarchical transfer learning (HTL) method as the core technology of knowledge co-creation and investigated its effectiveness in a dynamic multi-agent environment. The HTL method hierarchically abstracts obtained knowledge by ontological methods. Here, we evaluate the effectiveness of HTL with two types of ontology: action and state.