The vastness of vineyards in Hokkaido makes it hard for workers to eradicate weeds. To solve this problem, the aim of the current research has been to develop a weeding robot with a multi-link manipulator that uses a stereo camera system as a visual sensing tool for recognizing grape stems. However, the accuracy of this technique is lacking due to unavoidable measurement errors from the stereo camera system. In consideration of this problem, a method that defines the grape stem area has been devised. Additionally, to compensate for errors in the visual sensing technique, a tactile sensing technique has been employed that can haptically recognize a grape stem by means of a multi-link manipulator. In this paper, a robust sensing system is reported that analyzes the displacement of a grape stem to detect it base position such that weeding can be performed.