This study examined the housing effect on quality of life among Japanese people. In the current cross-sectional study, we analyzed the 1-year of data (November 2015-March 2016) with 2533 participants. We used the Short Form-8 questionnaire, an 8-item instrument that measures general aspects of health-related QOL. Comprehensive Assessment System for Built Environment Efficiency housing checklist which was developed by Ministry of Land, Infrastructure, Transport and Tourism was used to assess the housing aspects. This checklist has six health elements including thermal comfort, acoustic environment, lighting environment, hygiene, safety, and security for 8 distinctive rooms/places of home. Multilevel analysis was done to identify the relationship between the perceived level of housing problem and PCS and MCS by clustering by sex. Compared to those who always felt unsafe at home due to interior design problem, participants who never felt unsafe showed an average of 10.51 (95% CI = 7.69-13.34, p < 0.0001) and 5.78 (95% CI = 2.90-8.65, p < 0.0001) higher physical and mental component score (better quality of life), respectively. Those who never had thermal, acoustic, lighting, hygiene, and security problems of housing also exhibited significantly better quality of life compared to participants who felt these problems.