A phase difference image using phase-shifting digital holography, which is applied for non-contact shape measurement, contains speckle noises and filtering to reduce them is inevitable for its application. We have developed a new filter which causes less deformation than commonly used filters.