Understanding the structure-property relationship of glass material is still challenging due to a lack of periodicity in disordered materials. Here, we report the properties and atomic structure of vanadium phosphate glasses characterized by reverse Monte Carlo modelling based on neutron/synchrotron X-ray diffraction and EXAFS data, supplemented by Raman and NMR spectroscopy. In vanadium-rich glass, the water durability, thermal stability and hardness improve as the amount of P2O5 increases, and the network former of the glass changes from VOx polyhedra to the interplay between VOx polyhedra and PO4 tetrahedra. We find for the first time that the coordination number of oxygen atoms around a V4+ is four, which is an unusually small coordination number, and plays an important role for water durability, thermal stability and hardness. Furthermore, we show that the similarity between glass and crystal beyond the nearest neighbour distance is important for glass properties. These results demonstrate that controlling the oxygen coordination and valence of the network-forming cation is necessary for designing the properties of glass.