We investigate the mechanism of weak measurement by using an interferometric framework. In order to appropriately elucidate the interference effect that occurs in weak measurement, we introduce an interferometer for particles with internal degrees of freedom. It serves as a framework common to quantum eraser and weak measurement. We demonstrate that the geometric phase, particularly the Pancharatnam phase, results from the post-selection of the internal state, and thereby the interference pattern is changed. It is revealed that the extraordinary displacement of the probe wavepackets in weak measurement is achieved owing to the Pancharatnam phase associated with post-selection.