Soft x-ray magnetic circular dichroism (XMCD) have been measured on the single crystal of the charge-ordered type multiferroelectric material LuFe2O4. By comparing the XMCD results with the bulk magnetizations, the valence-specific magnetizations (VSMs) of Fe2+ and Fe3+ ions are obtained. Both Fe2+ and Fe3+ ions show positive and negative remanent VSMs below the spin ordering temperature T-SO similar to 250 K, showing the ferrimagnetic long-range ordering directly. Both of the remanent VSMs disappear above the TSO and the field dependencies of the VSMs behave in paramagnetic fashions with convex and concave curvatures. However, the signs of the VSMs of Fe2+ and Fe3+ remain opposite, nevertheless the system is in a paramagnetic phase. This is direct evidence of the robust antiferromagnetic interaction between Fe2+ and Fe3+ ions even above the T-SO. As temperature increases, furthermore, the opposite VSM of Fe3+ is reversed around the three-dimensional charge ordering temperature T-CO similar to 330 K showing that both net magnetic moments of Fe2+ and Fe3+ become parallel to the applied magnetic field. Possible correlation between these unusual magnetic behaviors and the ferroelectric properties of LuFe2O4 is discussed.