Regulatory CD4+ T cell (Treg), a subset of tumor-infiltrating lymphocytes (TILs), are known to suppress anticancer immunity but its clinical relevance in human breast cancer remains unclear. In this study, we estimated the relative abundance of Tregs in breast cancer of multiple patient cohorts by using the xCell algorithm on bulk tumor gene expression data. In total, 5177 breast cancer patients from five independent cohorts (TCGA-BRCA, GSE96058, GSE25066, GSE20194, and GSE110590) were analyzed. Treg abundance was not associated with cancer aggressiveness, patient survival, or immune activity markers, but it was lower in metastatic tumors when compared to matched primary tumors. Treg was associated with a high mutation rate of TP53 genes and copy number mutations as well as with increased tumor infiltration of M2 macrophages and decreased infiltration of T helper type 1 (Th1) cells. Pathological complete response (pCR) after neoadjuvant chemotherapy (NAC) was significantly associated with low Treg abundance in triple negative breast cancer (TNBC) but not in ER-positive/Her2-negative subtype. High Treg abundance was significantly associated with high tumor expression of multiple immune checkpoint inhibitor genes. In conclusion, Treg abundance may have potential as a predictive biomarker of pCR after NAC in TNBC.