Recently, tomato production in large-sized greenhouse facilities has been popular in Japan. In large-scale greenhouse, however, harvesting tomatoes by hand is tough work. Therefore, robotic agricultural system has been addressed. In order to promote robotic agricultural system, it is important to develop the machine vision system. Although we can acquire the ideal images with proper lighting at night and at daytime, it is difficult to exclude the halation, the shadow, and so on. The objective of this study is to reduce the halation with polarizing (PL) filter, which is the prevalent negative factor for vision system. In this experiment, single PL filter was precisely rotated by a stepping motor. Since the sun moves regularly, the relationship between the sun position and the rotational angles of the PL filter was calculated to reduce the halation. We carried out two experiments. One was done outdoors, and the other was in greenhouse. We could find the equation to determine the rotational angles of PL filter that created least halation. The angle was compared with measured angle obtained from the acquired images. Although the calculated value was useful for control PL filter on the condition that we could assume the sun was a single light source, it was difficult to use the calculated value in greenhouse, because not only sun light but also other reflected light influenced on fruit surfaces.