Polarity memory often commonly stabilizes a polarity order in eukaryotic cells and self-propelled disks. This seemingly does not hold for dense cases: the polarity memory of dense cells stabilizes the order whereas that of dense self-propelled disks destabilizes their order. As the origin of this difference in the stabilization of the order from the aspect of eukaryotic cells, we consider the inhibition of polarity fluctuation, owing to polarity memory in dense cells. We examine this inhibition in model cells and evaluate the polarity fluctuation defined as the variance from the polarity order. We show that the polarity fluctuation is inhibited by the polarity memory averaging out configuration fluctuation. Therefore, the order is stabilized only against cell configuration fluctuation, but probably not against the internal polarity fluctuation considered in self-propelled disks. This result implies that the difference in the polarity order reflects the difference in fluctuations between these systems.