This paper proposes a user-centric access point (AP) selection scheme to minimize the number of unnecessary reconfigurations of cognitive terminal, where the number of terminal reconfiguration can be estimated from user's mobility and coverage area (e.g. cell's size) of the radio access network. In this scheme, each cognitive terminal selects one AP from a candidate list that generated by itself. There are 4 different algorithms with different rules of generating the list of candidate APs. The target of this paper is to show the important of considering the number of reconfiguration of cognitive terminal in AP selection, and to show the best combination of parameters such as terminal's dwell-time in heterogeneous networks and AP's load information in order to minimize the unnecessary number of reconfiguration, and to achieve load balancing at the same time.