In this study, we analyzed the spread and decay durations of the COVID-19 pandemic in several cities of China, England, Germany, and Japan, where the first wave has undergone decay. Differences in medical and health insurance systems, as well as in regional policies incommoded the comparison of the spread and decay in different cities and countries. The spread and decay durations in the cities of the four studied countries were reordered and calculated based on an asymmetric bell-shaped model. We acquired the values of the ambient temperature, absolute humidity, and population density to perform multivariable analysis. We found a significant correlation (p < 0.05) of the spread and decay durations with population density in the four analyzed countries. Specifically, spread duration showed a high correlation with population density and absolute humidity (p < 0.05), whereas decay duration demonstrated the highest correlation with population density, absolute humidity, and maximum temperature (p < 0.05). The effect of population density was almost nonexistent in China because of the implemented strict lockdown. Our findings will be useful in policy setting and governmental actions in the next pandemic, as well as in the next waves of COVID-19.