A method is presented that extends the real world into all buildings. This building-scale virtual reality (VR) method differs from augmented reality (AR) in that it uses automatically generated 3D point cloud maps of building interiors. It treats an entire indoor area a pose tracking area by using data collected using an RGB-D camera mounted on a VR headset and using deep learning to build a model from the data. It modifies the VR space in accordance with its intended usage by using segmentation and replacement of the 3D point clouds. This is difficult to do with AR but is essential if VR is to be used for actual real-world applications, such as disaster simulation including simulation of fires and flooding in buildings. 3D pose tracking in the building-scale VR is more accurate than conventional RGB-D simultaneous localization and mapping.