Decision making in geotechnical engineering is always related to a project carried out at a specific site. It is natural for data-driven site characterization (DDSC) to attract the most attention in data-centric geotechnics. This paper proposed eight benchmark examples and a benchmarking procedure to support unbiased and competitive evaluation of emerging ML methods. The primary goal of DDSC is to bring the value of a “data first” agenda to practice, specifically to produce a 3D stratigraphic map of the subsurface volume below a full-scale project site and to estimate relevant engineering properties at each spatial point based on site investigation data and other relevant Big Indirect Data (BID). A reasonable full-scale ground 20 m long × 20 m wide × 10 m deep is adopted. Virtual grounds containing horizontal, inclined, or discontinuous soil layers and spatially varying synthetic cone penetration test data are created to test the performance of DDSC methods over a range of ground conditions. A benchmark example is defined by a combination of a virtual ground (“reality”) and a training dataset (measured “reality”). An additional benchmark example based on actual CPT data is included to check whether performance under virtual ground conditions holds under real ground conditions.