For perovskite oxides, ABO(3), slight octahedral distortions have close links to functional properties. While perovskite oxide heterostructures offer a good platform for controlling functionalities, atomistic understanding of octahedral distortion at the interface has been a challenge as it requires precise measurements of the oxygen atomic positions. Here we demonstrate an approach to clarify distortions at an atomic level using annular bright-field imaging in aberration-corrected scanning transmission electron microscopy, which provides precise mappings of cation and oxygen atomic positions from distortion-minimized images. This technique revealed significant distortions of RuO6 and ScO6 octahedra at the heterointerface between a SrRuO3 film and a GdScO3 substrate. We also found that structural mismatch was relieved within only four unit cells near the interface by shifting the oxygen atomic positions to accommodate octahedral tilt angle mismatch. The present results underscore the critical role of the oxygen atom in the octahedral connectivity at the perovskite oxide heterointerface.