Many studies have reported that natural sounds (e.g., birdsong) are more restorative than urban noise. These studies have used physiological and psychological indicators, such as the skin conductance level (SCL) and the Perceived Restorativeness Scale (PRS), to evaluate the restorative effect of natural sounds. However, the effect of faint background noise mixed with birdsong on the restorativeness of birdsong has not been described yet. In the current experiment, we examined whether traffic noise affects the perceived restorativeness and the physiological restorativeness of birdsong in a low-stress condition using the SCL and the PRS. The scores of the PRS showed that birdsong significantly increased the perceived restorativeness of the place regardless of the car noise, but no significant difference was found between these two birdsongs. In contrast, physiologically, the birdsong without car noise decreased the participants' SCL significantly more than the birdsong with car noise did. These results indicate that the SCL would be useful to detect the effect of background noise on natural sound when the noise is too low to affect the perceived restorativeness. This study highlights the importance of measuring the SCL besides assessing perceived restorativeness to describe the characteristics of restorative natural sound in future research.