In this paper, a grip-force control of an elastic object is proposed based on a visual slip-margin feedback. When an elastic object is pressed and slid slightly on a rigid plate, a partial slip, called "incipient slip," occurs on the contact surface. The slip margin between an elastic object and a rigid plate is estimated based on the analytic solution of a Hertzian contact model. A one-degree-of-freedom gripper consisting of a camera and a force sensor is developed. The slip margin can be estimated from the tangential force measured by a force sensor, the deformation of the elastic object and the radius on the contact area both measured by a camera. In the proposed method, the friction coefficient is not explicitly needed. The "eccentricity" is used to estimate the displacement of the elastic object at the contact area with high accuracy. The grip force is controlled by a direct feedback of the estimated slip margin. The proof of the contact stability by the proposed control is analytically given. As a result, the slip margin is maintained at a desired value, without occurring the gross slip against a disturbance traction force to the object. The validity of the proposed method is confirmed by experiments.