We discuss a method for 3d shape reconstruction from a stereoscopic image sequence. Once the depth map and camera parameters are given, any image can be warped to the reference image. Therefore, the depth map and the camera parameters can be estimated by minimizing the sum of the variances among these warped images. The depth map is described by a set of bicubic patches, i.e. each patch is determined by cubic spline interpolation using neighboring 16 points. A regularization term is introduced to the cost function to stabilize the estimation. The method is verified using several real image sets.