Existing approaches for grammatical error correction (GEC) largely rely on
supervised learning with manually created GEC datasets. However, there has been
little focus on verifying and ensuring the quality of the datasets, and on how
lower-quality data might affect GEC performance. We indeed found that there is
a non-negligible amount of "noise" where errors were inappropriately edited or
left uncorrected. To address this, we designed a self-refinement method where
the key idea is to denoise these datasets by leveraging the prediction
consistency of existing models, and outperformed strong denoising baseline
methods. We further applied task-specific techniques and achieved
state-of-the-art performance on the CoNLL-2014, JFLEG, and BEA-2019 benchmarks.
We then analyzed the effect of the proposed denoising method, and found that
our approach leads to improved coverage of corrections and facilitated fluency
edits which are reflected in higher recall and overall performance.