Many natural hazards like earthquake and seismic sea waves happen every year all over the world. In addition, many manmade disasters happen in many countries. To deal with these disasters, rescue robots are being developed in many countries. These robots support the rescue workers to search for the victims in the disaster environments. In this paper, we propose a flexible and generic interface system for operating multiple rescue robots. The system can operate multiple heterogeneous robots from a common operation interface. The system also supports changing the operation mode of any robot between autonomous and tele-operation. As a demonstration, we used this system on actual robots in RoboCup Rescue league[1] and evaluated the proposed system.