The goal of this study is to develop a prediction method to recognize the wake field behind a ship using a convolutional neural network (CNN) model. First, a new representation method for a 3D curved surface is proposed suitable for the CNN, called an image-based hull form representation (IHR). The advantages of the proposed method are the high fidelity of its hull form representation using more than 20,000 input data points and its fast prediction speed, which requires less than 0.01 s for a task that traditionally took more than an hour to estimate by physics-based simulation. The IHR regards that a two-dimensional grid formed on the 3D curved hull surface, which is used for structured-grid-based CFD, as a data set with the same data structure as the image data. Because CNNs recognize image data at accuracy rates higher than humans, a CNN is also be expected to recognize 3D surface characteristics with higher accuracy than humans. The image data are represented by three primary colors (cyan, magenta, yellow) in vertical and horizontal (i × j) pixels. The hull-form-structured grid can also be expressed as an i × j structure data with (x, y, z) coordinates that have the same data structure as the three primary colors in the image data. A CFD calculation data set of 2730 ship types with different stern shapes was constructed to verify the proposed method. The validation results proves that the root mean squared error of the proposed model is 0.005 to predict axial wake velocity on a propeller plane, and the coefficient of determination R2 achieves 0.986. In addition, the estimation speed for each hull-form prediction is 100,000 times faster than are physics-based simulations. The results lead to the conclusion that the representation method of a curved surface and the proposed prediction method of the stern wake field is a promising tool in the initial hull form design.