Skip to main content
placeholder image

Metamorphic Robustness Testing of Google Translate

Conference Paper


Abstract


  • Current research on the testing of machine translation software mainly focuses on functional correctness for valid, well-formed inputs. By contrast, robustness testing, which involves the ability of the software to handle erroneous or unanticipated inputs, is often overlooked. In this paper, we propose to address this important shortcoming. Using the metamorphic robustness testing approach, we compare the translations of original inputs with those of follow-up inputs having different categories of minor typos. Our empirical results reveal a lack of robustness in Google Translate, thereby opening a new research direction for the quality assurance of neural machine translators.

Publication Date


  • 2020

Publisher


Citation


  • Lee, D. T. S., Zhou, Z. Q., & Tse, T. H. (2020). Metamorphic Robustness Testing of Google Translate. In Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020 (pp. 388-395). doi:10.1145/3387940.3391484

Scopus Eid


  • 2-s2.0-85093073831

Web Of Science Accession Number


Start Page


  • 388

End Page


  • 395

Abstract


  • Current research on the testing of machine translation software mainly focuses on functional correctness for valid, well-formed inputs. By contrast, robustness testing, which involves the ability of the software to handle erroneous or unanticipated inputs, is often overlooked. In this paper, we propose to address this important shortcoming. Using the metamorphic robustness testing approach, we compare the translations of original inputs with those of follow-up inputs having different categories of minor typos. Our empirical results reveal a lack of robustness in Google Translate, thereby opening a new research direction for the quality assurance of neural machine translators.

Publication Date


  • 2020

Publisher


Citation


  • Lee, D. T. S., Zhou, Z. Q., & Tse, T. H. (2020). Metamorphic Robustness Testing of Google Translate. In Proceedings - 2020 IEEE/ACM 42nd International Conference on Software Engineering Workshops, ICSEW 2020 (pp. 388-395). doi:10.1145/3387940.3391484

Scopus Eid


  • 2-s2.0-85093073831

Web Of Science Accession Number


Start Page


  • 388

End Page


  • 395