In this paper, we propose a target-tracking system for a mobile robot equipped with a stereo camera. Mobile service robots with the ability to track a specific person in dynamic environments have been required. In such environments, varying illumination and the presence of multiple people are challenges to carrying out target tracking. Color and location information is used for the targets features, that are useful for distinguishing a target from the other people. However, color information is not resilient to illumination changes. On the other hand, location information might be infeasible when non-target people present in the environments. Therefore, it is necessary to combine each information according to the situations, in which either information is infeasible to use as the feature. In order to make the system robust to varying illumination and presence of non-target people, a parameter of illumination changes is introduced in this paper. The parameter is defined using automatically adjusted white balance. The color and location features are weighted based on white-balance changes to determine a target. The effectiveness of the proposed system is verified through target-tracking experiments in outdoor environments. It is demonstrated that the proposed method can successfully recognize a target in the environments where the lighting condition changes extremely and non-target people present.