Phase difference image using phase-shifting digital holography is one of the prospective methods for non-contact shape measurement. For the application, filtering of the obtained image is needed to reduce the speckle noise. Conventional filter, e.g. moving average filter, causes loss of the shape information especially around the region with high steepness. To solve this problem novel filter has been developed that kept steepness of the shape in noise reduction. The filtered data were compared with conventionally filtering ones and the former was closer to the data observed by a laser measuring microscope.