We are developing an ultra-wideband spectroscopic instrument, DESHIMA (DEep Spectroscopic HIgh-redshift MApper), based on the technologies of an on-chip filter bank and microwave kinetic inductance detector (MKID) to investigate dusty starburst galaxies in the distant universe at millimeter and submillimeter wavelengths. An on-site experiment of DESHIMA was performed using the ASTE 10-m telescope. We established a responsivity model that converts frequency responses of the MKIDs to line-of-sight brightness temperature. We estimated two parameters of the responsivity model using a set of skydip data taken under various precipitable water vapor (PWV 0.4-3.0 mm) conditions for each MKID. The line-of-sight brightness temperature of sky is estimated using an atmospheric transmission model and the PWVs. As a result, we obtain an average temperature calibration uncertainty of 1 sigma=4, which is smaller than other photometric biases. In addition, the average forward efficiency of 0.88 in our responsivity model is consistent with the value expected from the geometrical support structure of the telescope. We also estimate line-of-sight PWVs of each skydip observation using the frequency response of MKIDs and confirm the consistency with PWVs reported by the Atacama Large Millimeter/submillimeter Array.