Skip to main content
placeholder image

Metamorphic testing for software quality assessment: a study of search engines

Journal Article


Abstract


  • Metamorphic testing is a testing technique that can be used to verify the functional correctness of software in the absence of an ideal oracle. This paper extends metamorphic testing into a user-oriented approach to software verification, validation, and quality assessment, and conducts large scale empirical studies with four major web search engines: Google, Bing, Chinese Bing, and Baidu. These search engines are very difficult to test and assess using conventional approaches owing to the lack of an objective and generally recognized oracle. The results are useful for both search engine developers and users, and demonstrate that our approach can effectively alleviate the oracle problem and challenges surrounding a lack of specifications when verifying, validating, and evaluating large and complex software systems.

Authors


Publication Date


  • 2016

Citation


  • Zhou, Z., Xiang, S. & Chen, T. (2016). Metamorphic testing for software quality assessment: a study of search engines. IEEE Transactions on Software Engineering, 42 (3), 260-280.

Scopus Eid


  • 2-s2.0-84963808208

Ro Metadata Url


  • http://ro.uow.edu.au/eispapers/5699

Has Global Citation Frequency


Number Of Pages


  • 20

Start Page


  • 260

End Page


  • 280

Volume


  • 42

Issue


  • 3

Place Of Publication


  • United States

Abstract


  • Metamorphic testing is a testing technique that can be used to verify the functional correctness of software in the absence of an ideal oracle. This paper extends metamorphic testing into a user-oriented approach to software verification, validation, and quality assessment, and conducts large scale empirical studies with four major web search engines: Google, Bing, Chinese Bing, and Baidu. These search engines are very difficult to test and assess using conventional approaches owing to the lack of an objective and generally recognized oracle. The results are useful for both search engine developers and users, and demonstrate that our approach can effectively alleviate the oracle problem and challenges surrounding a lack of specifications when verifying, validating, and evaluating large and complex software systems.

Authors


Publication Date


  • 2016

Citation


  • Zhou, Z., Xiang, S. & Chen, T. (2016). Metamorphic testing for software quality assessment: a study of search engines. IEEE Transactions on Software Engineering, 42 (3), 260-280.

Scopus Eid


  • 2-s2.0-84963808208

Ro Metadata Url


  • http://ro.uow.edu.au/eispapers/5699

Has Global Citation Frequency


Number Of Pages


  • 20

Start Page


  • 260

End Page


  • 280

Volume


  • 42

Issue


  • 3

Place Of Publication


  • United States