Skip to main content

Automated functional testing of online search services

Journal Article


Abstract


  • Search services are the main interface through which people discover information on the Internet. A

    fundamental challenge in testing search services is the lack of oracles. The sheer volume of data on the

    Internet prohibits testers from verifying the results. Furthermore, it is difficult to objectively assess the

    ranking quality because different assessors can have very different opinions on the relevance of a Web

    page to a query. This paper presents a novel method for automatically testing search services without

    the need of a human oracle. The experimental findings reveal that some commonly used search engines,

    including Google, Yahoo!, and Live Search, are not as reliable as what most users would expect. For

    example, they may fail to find pages that exist in their own repositories, or rank pages in a way that is

    logically inconsistent. Suggestions are made for search service providers to improve their service quality.

Authors


Publication Date


  • 2012

Citation


  • Zhou, Z. Quan., Zhang, S., Hagenbuchner, M., Tse, T. H., Kuo, F. & Chen, T. Y. (2012). Automated functional testing of online search services. Software Testing, Verification and Reliability, 22 (4), 221-243.

Scopus Eid


  • 2-s2.0-84862066626

Ro Metadata Url


  • http://ro.uow.edu.au/infopapers/3319

Has Global Citation Frequency


Number Of Pages


  • 22

Start Page


  • 221

End Page


  • 243

Volume


  • 22

Issue


  • 4

Place Of Publication


  • United Kingdom

Abstract


  • Search services are the main interface through which people discover information on the Internet. A

    fundamental challenge in testing search services is the lack of oracles. The sheer volume of data on the

    Internet prohibits testers from verifying the results. Furthermore, it is difficult to objectively assess the

    ranking quality because different assessors can have very different opinions on the relevance of a Web

    page to a query. This paper presents a novel method for automatically testing search services without

    the need of a human oracle. The experimental findings reveal that some commonly used search engines,

    including Google, Yahoo!, and Live Search, are not as reliable as what most users would expect. For

    example, they may fail to find pages that exist in their own repositories, or rank pages in a way that is

    logically inconsistent. Suggestions are made for search service providers to improve their service quality.

Authors


Publication Date


  • 2012

Citation


  • Zhou, Z. Quan., Zhang, S., Hagenbuchner, M., Tse, T. H., Kuo, F. & Chen, T. Y. (2012). Automated functional testing of online search services. Software Testing, Verification and Reliability, 22 (4), 221-243.

Scopus Eid


  • 2-s2.0-84862066626

Ro Metadata Url


  • http://ro.uow.edu.au/infopapers/3319

Has Global Citation Frequency


Number Of Pages


  • 22

Start Page


  • 221

End Page


  • 243

Volume


  • 22

Issue


  • 4

Place Of Publication


  • United Kingdom