The purpose of this study is to illuminate current issues facing Computer Security Incident Response Teams (CSIRTs) in Japan by discussing two CSIRT processes: constructing organizational resilience and falling into a "resilience trap." Through interview-based case studies, we find that CSIRTs face an organizational dilemma. Ensuring present resilience by coping with security incidents reduces future resilience by reducing opportunities to train team members to deal with future incidents. It is vital for Japanese CSIRTs to find solutions to this dilemma, which we call the resilience trap.