© 2016 IEEE. Highly sensitive measurement of the very weak magnetic characteristics of magnetic particles in solution or of a solution itself is required for applications, such as biomedical examinations. In this paper, higher sensitivity and additional performance with relaxation measurement and harmonic signal detection were achieved by an improvement of the driving mechanism and the high-resolution electric magnet, and optimization of the superconducting quantum interference device (SQUID) detection unit. A servomotor was used for sample vibration and rotation with precise control and high speed. For detection of the M-H characteristics, the sample was vibrated under the magnetic field in the electromagnet. For measuring the relaxation of the magnetization, the sample was rotated. A first-order differential pickup coil with a normal conducting wire connected to the input coil of a higherature superconductor-SQUID was used to detect the magnetic signal from the sample. High-resolution measurement of the magnetic moment on the order of 10-11 A m2 was achieved with this system configuration. An ac magnetic field with a dc bias could then be applied to the sample for detection of the harmonic signal. The magnetic relaxation signal from the sample was measured by another of the same type of differential detection coil equipped outside of the electric magnet by rotating, instead of vibrating, the sample. The magnetization curve for a low concentration of iron nanoparticles with a superparamagnetic character in the solution was successfully measured using the developed magnetometer, and the relaxation phenomenon was also characterized.