Chemical mechanical polishing (CMP) using a suede polishing pad is an essential fabrication process for glass substrates that require ultra-high planarization. However, the effect of surface asperities of the suede pad on its polishing characteristics is not completely understood because the structure of the suede pad in the thickness direction is not constant, and its surface asperities can easily change during the pad conditioning or marathon polishing processes. In addition, many previous studies have discussed the polishing mechanism using a suede pad; however, these studies used suede pads with a pore size of approximately 100 μm. This paper discusses the polishing characteristics of a suede pad with fine micrometer-sized pores by clarifying the relationships between the removal rate, friction coefficient, pore parameters, and roughness as the pad surface asperities. In this study, a series of marathon polishing tests were performed with and without conditioning. It was discovered that the removal rate was affected not only by the pore parameters but also by the surface roughness of the suede pad with fine pores. The relationship between the removal rate and the friction coefficient changed owing to the influence of pad conditioning, and this change is significant when the break-in conditioning time is short.