This paper describes a systematic method to model and compensate geometric errors of machine tools. In order to separate geometric errors from other errors, measured errors are analyzed in the frequency domain by using the Fourier series. Then, the frequency components corresponding to geometric errors are selected based on the repeatability of their wavelength. Finally, the components are reconstructed and forwarded for the compensation by a fine motion drive. A CNC machine tool with a fine motion mechanism oil the Z-axis was developed to compensate the error components in the Z direction oil the XY plane. A flat surface machining with non-rotational cutting tools was tested to validate our approach. On the plane of 45 mm x 70 mm, the fluctuation of the relative displacement was reduced from 1.3 to 0.5 mu m P-V. Machining experiments with a single-crystal diamond tool were also carried out and the straightness of the profile curve was reduced from 1.0 to 0.4 mu m. The result of the experiments showed that the geometric errors were compensated separately from the vibration due to the bending mode of the machine column. (c) 2008 Elsevier Ltd. All rights reserved.