We find theoretically that the vortex core in the Fulde-Ferrell-Larkin-Ovchinnikov (FFLO) state is quite different from the ordinary core for a simple topological reason. The intersection point of a vortex and nodal plane of the FFLO state empties the excess spins. This leads to observable consequences in the spatial structure of the spontaneous magnetization. We analyze this topological structure based on the low lying excitation spectrum by solving a microscopic Bogoliubov-de Gennes equation to clarify its physical origin.