Skip to main content

Zhi Quan (George) Zhou Associate Professor

Associate Professor

  • Faculty of Engineering and Information Sciences
  • School of Computing & Information Technology

Overview


Dr Zhi Quan (George) Zhou received the BSc degree in Computer Science from Peking University, China, and the PhD degree in Software Engineering from The University of Hong Kong. His current research interests include software testing and debugging; cybersecurity and security testing; the interplay among software testing, machine learning, and big data; and self-driving vehicles. His research has been reported in Nature (https://go.nature.com/2TdWoM6). He was selected for a Virtual Earth Award by Microsoft Research, Redmond, USA, and a 2018 Researcher of the Year Gold Disruptor Award by the Australian Computer Society.

Zhou was the main contributor to some of the earliest research papers on metamorphic testing, and was one of the few pioneers who opened up and established the metamorphic testing research field. In 2016, he co-founded and chaired the IEEE/ACM 1st International Workshop on Metamorphic Testing (MET '16), in conjunction with the 38th International Conference on Software Engineering (ICSE '16) in Austin, USA. As of July 2019, the annual MET workshop was recognized as one of the "popular conferences, workshops and journals related to software engineering" (p. 8, https://arxiv.org/abs/1907.09427).

Zhou was an invited keynote speaker at ICSE MET 2017 held in Buenos Aires, Argentina, and at the IEEE International Conference on Artificial Intelligence Testing in San Francisco, 2019 (slides), where he was also invited to organize the Special Track on Metamorphic Testing and Artificial Intelligence. He was an ACM SIGSOFT Webinar speaker, an ICSE '18 Technical Briefings speaker, and an ICSE '16 and ICSE '19 journal-first speaker, introducing metamorphic testing through all these venues.

Zhou has been invited to serve as a reviewer/adviser for various organizations including the Australian Research Council, the Research Grants Council of Hong Kong, and the Air Force Office of Scientific Research of the United States Air Force, and for various international journals including IEEE Transactions on Software Engineering, on Computers, on Reliability, on Services Computing, and so on. He has served as a guest editor for the Journal of Systems and Software.
 

Top Publications


Research Overview


  • Software engineering, software testing and debugging, verification and validation, user experience, citation analysis.

Available as Research Supervisor

Selected Publications


Impact Story


  • An independent study conducted by American researchers identified Zhou as one of the top five researchers internationally in the important research field of test oracles [1]. Another independent study conducted by European researchers identified Zhou as one of the top four researchers internationally in metamorphic testing [2].<br /><br />References:<br /><br />[1] R. A. P. Oliveira, U. Kanewala, and P. A. Nardi. "Automated test oracles: State of the art, taxonomies, and trends," In Advances in Computers, volume 95, pp. 113-199. Elsevier, 2014. Available online: <a href="http://iii.library.uow.edu.au/record=b2303118" target="_blank" rel="noopener">http://iii.library.uow.edu.au/record=b2303118</a><br /><br />[2] S. Segura, G. Fraser, A. Sanchez, and A. Ruiz-Cortes, "A survey on metamorphic testing", IEEE Transactions on Software Engineering, 42(9), pp. <strong> </strong> 805-824, 2016. Available online: <a href="http://dx.doi.org/10.1109/TSE.2016.2532875" target="_blank" rel="noopener">http://dx.doi.org/10.1109/TSE.2016.2532875</a>
  • <p>Zhou initiated a "metamorphic robustness testing" method and led his research team to use this method to verify the major citation database systems <a href="https://www.scopus.com" target="_blank" rel="noopener"><em>Scopus</em></a> and <a href="https://clarivate.com/products/web-of-science/" target="_blank" rel="noopener"><em>Web of Science</em></a>, revealing robustness problems in citation counts and journal impact factors computed by these systems. A paper titled "Metamorphic Robustness Testing: Exposing Hidden Defects in Citation Statistics and Journal Impact Factors" by Zhi Quan Zhou, T.H. Tse, and Matt Witheridge has been published in <em>IEEE Transactions on Software Engineering</em>, the top journal in the field (URL for download: <a href="https://doi.org/10.1109/TSE.2019.2915065" target="_blank" rel="noopener">https://doi.org/10.1109/TSE.2019.2915065</a>).</p>The high impact of the results lies in the finding that the presence of simple hyphens in a paper title adversely affects the citation statistics. Thus, metrics based on citation counts, which are commonly used for staff evaluations in universities, are not as robust as widely believed. This will be of interest not only to professors seeking tenure or promotion, but also to the senior management such as presidents, deans, and heads. The surprising results are applicable to all faculties in any university.<br /><br />Zhou et al.'s findings have been reported by international news media, including <a href="https://documents.uow.edu.au/~zhiquan/TheHyphenMan.pdf" target="_blank" rel="noopener"><em>The Australian</em></a>, <a href="https://www.psychologytoday.com/intl/blog/the-athletes-way/201906/one-small-hyphen-in-title-could-diminish-academic-success" target="_blank" rel="noopener"><em>Psychology Today</em></a><a href="https://www.sciencealert.com/hyphens-break-our-entire-system-of-scientific-ranking-new-analysis-reveals" target="_blank" rel="noopener"><em></em></a>, <a href="https://cacm.acm.org/news/237400-a-single-punctuation-mark-has-been-skewing-our-entire-system-of-scientific-ranking/fulltext" target="_blank" rel="noopener"><em>Communications of the ACM</em></a>, and <em><a href="https://www.eurekalert.org/" target="_blank" rel="noopener">EurekAlert!</a></em>, among others. <br /><br />On the Internet, the implications of Zhou et al.'s findings have been discussed from diverse perspectives, some of which extend beyond the conventional computing and citation domains. For example, see "<a href="https://www.slashgear.com/brave-new-world-predictions-are-coming-true-with-hyphens-31578773/" target="_blank" rel="noopener">Brave New World predictions are coming true with hyphens</a>" written by <a href="https://www.slashgear.com/author/chrisburns/" target="_blank" rel="noopener">Chris Burns</a> on <a href="https://www.slashgear.com/brave-new-world-predictions-are-coming-true-with-hyphens-31578773/" target="_blank" rel="noopener"><em>SlashGear</em></a>. Inquiries have been made from various communities including, among others, the community of patients with <a href="https://www.3cr.org.au/chronicallychilled/episode-201901021800/asher-wolf-ehlers-danlos-syndrome" target="_blank" rel="noopener">Ehlers-Danlos Syndrome</a> (a <a href="https://en.wikipedia.org/wiki/List_of_syndromes" target="_blank" rel="noopener">condition</a> that has a hyphenated name and is known to be under-reported and under-researched).<br /><br />On 5 June 2019, the <a href="http://www.webofsciencegroup.com/" target="_blank" rel="noopener">Web of Science Group</a> posted a <a href="https://clarivate.com/webofsciencegroup/download/36464/" target="_blank" rel="noopener">response</a> to Zhou et al.'s paper on the <a href="https://clarivate.com/blog/science-research-connect/web-of-science-group-response-to-metamorphic-robustness-testing-exposing-hidden-defects-in-citation-statistics-and-journal-impact-factors/" target="_blank" rel="noopener">Clarivate Analytics website</a>. Feedback from the authors was reported by <a href="https://www.sciencealert.com/hyphens-break-our-entire-system-of-scientific-ranking-new-analysis-reveals" target="_blank" rel="noopener"><em>ScienceAlert</em></a>.  On 7 August 2019, Zhou et al.'s work was reported by <em><a href="https://go.nature.com/2TdWoM6" target="_blank" rel="noopener"><strong>Nature</strong></a> </em>(https://go.<strong>nature.com</strong>/2TdWoM6).
  • On March 10, 2018, Zhou and his MPhil student Liqun Sun detected a critical fault in the real-world self-driving system <a href="http://apollo.auto" target="_blank" rel="noopener">Baidu Apollo</a>, and they reported this finding to the Apollo team on the same day (this was <strong>8 days before</strong> <a href="https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg" target="_blank" rel="noopener">a fatal accident of a Uber's self-driving vehicle in Tempe, Arizona</a>). The fault caused the self-driving car's LiDAR perception system to ignore pedestrians or other obstacles on the roadway. The Apollo team confirmed the fault on March 19, 2018 (24 hours after <a href="https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg" target="_blank" rel="noopener">the Uber accident</a>, where the vehicle involved used exactly the same type of LiDAR sensor as Apollo). This study was reported in an article titled "Metamorphic Testing of Driverless Cars" by Zhou and Sun, published in <a href="https://cacm.acm.org/" target="_blank" rel="noopener"><em>Communications of the ACM</em></a>, ACM's flagship magazine (<a href="https://doi.org/10.1145/3241979" target="_blank" rel="noopener">original paper</a>; <a href="https://doi.org/10.1145/3332409" target="_blank" rel="noopener">follow-up letter</a>). Zhou covered this story in his <a href="http://doi.org/10.5281/zenodo.2636979" target="_blank" rel="noopener">keynote speech</a> at the <a href="http://www.ieeeaitests.com/html/speakers.html" target="_blank" rel="noopener">IEEE International Conference on Artificial Intelligence Testing</a> (AITest 2019) held in San Francisco East Bay, California.

Available as Research Supervisor

Advisees


  • Graduate Advising Relationship

    Degree Research Title Advisee
    Doctor of Philosophy Un Similarity-Based Software Text case Prioritisation Liu, Chen
    Doctor of Philosophy Metamorphic Testing applied to citation systems and other border applications Witheridge, Matt
    Doctor of Philosophy (Integrated) A New Approach to the Detection of Software Security Vulnerabilities Zhu, Junting
    Doctor of Philosophy Testing Navigation Systems Brown, Joshua
    Doctor of Philosophy A Machine Learning Based Approach for SoftwareTesting. Wu, Sarah
    Doctor of Philosophy Software testing of web and cloud based applications Han, Jiacheng
    Doctor of Philosophy Testing methods for intelligent systems Iqbal, Muhammad
    Master of Philosophy - EIS A Case Study of Web Security on Social Media Advertising Liu, Yelin
    Doctor of Philosophy (Integrated) On Antomated Testing of Web Applications Selay, Elmin

Top Publications


Research Overview


  • Software engineering, software testing and debugging, verification and validation, user experience, citation analysis.

Selected Publications


Impact Story


  • An independent study conducted by American researchers identified Zhou as one of the top five researchers internationally in the important research field of test oracles [1]. Another independent study conducted by European researchers identified Zhou as one of the top four researchers internationally in metamorphic testing [2].<br /><br />References:<br /><br />[1] R. A. P. Oliveira, U. Kanewala, and P. A. Nardi. "Automated test oracles: State of the art, taxonomies, and trends," In Advances in Computers, volume 95, pp. 113-199. Elsevier, 2014. Available online: <a href="http://iii.library.uow.edu.au/record=b2303118" target="_blank" rel="noopener">http://iii.library.uow.edu.au/record=b2303118</a><br /><br />[2] S. Segura, G. Fraser, A. Sanchez, and A. Ruiz-Cortes, "A survey on metamorphic testing", IEEE Transactions on Software Engineering, 42(9), pp. <strong> </strong> 805-824, 2016. Available online: <a href="http://dx.doi.org/10.1109/TSE.2016.2532875" target="_blank" rel="noopener">http://dx.doi.org/10.1109/TSE.2016.2532875</a>
  • <p>Zhou initiated a "metamorphic robustness testing" method and led his research team to use this method to verify the major citation database systems <a href="https://www.scopus.com" target="_blank" rel="noopener"><em>Scopus</em></a> and <a href="https://clarivate.com/products/web-of-science/" target="_blank" rel="noopener"><em>Web of Science</em></a>, revealing robustness problems in citation counts and journal impact factors computed by these systems. A paper titled "Metamorphic Robustness Testing: Exposing Hidden Defects in Citation Statistics and Journal Impact Factors" by Zhi Quan Zhou, T.H. Tse, and Matt Witheridge has been published in <em>IEEE Transactions on Software Engineering</em>, the top journal in the field (URL for download: <a href="https://doi.org/10.1109/TSE.2019.2915065" target="_blank" rel="noopener">https://doi.org/10.1109/TSE.2019.2915065</a>).</p>The high impact of the results lies in the finding that the presence of simple hyphens in a paper title adversely affects the citation statistics. Thus, metrics based on citation counts, which are commonly used for staff evaluations in universities, are not as robust as widely believed. This will be of interest not only to professors seeking tenure or promotion, but also to the senior management such as presidents, deans, and heads. The surprising results are applicable to all faculties in any university.<br /><br />Zhou et al.'s findings have been reported by international news media, including <a href="https://documents.uow.edu.au/~zhiquan/TheHyphenMan.pdf" target="_blank" rel="noopener"><em>The Australian</em></a>, <a href="https://www.psychologytoday.com/intl/blog/the-athletes-way/201906/one-small-hyphen-in-title-could-diminish-academic-success" target="_blank" rel="noopener"><em>Psychology Today</em></a><a href="https://www.sciencealert.com/hyphens-break-our-entire-system-of-scientific-ranking-new-analysis-reveals" target="_blank" rel="noopener"><em></em></a>, <a href="https://cacm.acm.org/news/237400-a-single-punctuation-mark-has-been-skewing-our-entire-system-of-scientific-ranking/fulltext" target="_blank" rel="noopener"><em>Communications of the ACM</em></a>, and <em><a href="https://www.eurekalert.org/" target="_blank" rel="noopener">EurekAlert!</a></em>, among others. <br /><br />On the Internet, the implications of Zhou et al.'s findings have been discussed from diverse perspectives, some of which extend beyond the conventional computing and citation domains. For example, see "<a href="https://www.slashgear.com/brave-new-world-predictions-are-coming-true-with-hyphens-31578773/" target="_blank" rel="noopener">Brave New World predictions are coming true with hyphens</a>" written by <a href="https://www.slashgear.com/author/chrisburns/" target="_blank" rel="noopener">Chris Burns</a> on <a href="https://www.slashgear.com/brave-new-world-predictions-are-coming-true-with-hyphens-31578773/" target="_blank" rel="noopener"><em>SlashGear</em></a>. Inquiries have been made from various communities including, among others, the community of patients with <a href="https://www.3cr.org.au/chronicallychilled/episode-201901021800/asher-wolf-ehlers-danlos-syndrome" target="_blank" rel="noopener">Ehlers-Danlos Syndrome</a> (a <a href="https://en.wikipedia.org/wiki/List_of_syndromes" target="_blank" rel="noopener">condition</a> that has a hyphenated name and is known to be under-reported and under-researched).<br /><br />On 5 June 2019, the <a href="http://www.webofsciencegroup.com/" target="_blank" rel="noopener">Web of Science Group</a> posted a <a href="https://clarivate.com/webofsciencegroup/download/36464/" target="_blank" rel="noopener">response</a> to Zhou et al.'s paper on the <a href="https://clarivate.com/blog/science-research-connect/web-of-science-group-response-to-metamorphic-robustness-testing-exposing-hidden-defects-in-citation-statistics-and-journal-impact-factors/" target="_blank" rel="noopener">Clarivate Analytics website</a>. Feedback from the authors was reported by <a href="https://www.sciencealert.com/hyphens-break-our-entire-system-of-scientific-ranking-new-analysis-reveals" target="_blank" rel="noopener"><em>ScienceAlert</em></a>.  On 7 August 2019, Zhou et al.'s work was reported by <em><a href="https://go.nature.com/2TdWoM6" target="_blank" rel="noopener"><strong>Nature</strong></a> </em>(https://go.<strong>nature.com</strong>/2TdWoM6).
  • On March 10, 2018, Zhou and his MPhil student Liqun Sun detected a critical fault in the real-world self-driving system <a href="http://apollo.auto" target="_blank" rel="noopener">Baidu Apollo</a>, and they reported this finding to the Apollo team on the same day (this was <strong>8 days before</strong> <a href="https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg" target="_blank" rel="noopener">a fatal accident of a Uber's self-driving vehicle in Tempe, Arizona</a>). The fault caused the self-driving car's LiDAR perception system to ignore pedestrians or other obstacles on the roadway. The Apollo team confirmed the fault on March 19, 2018 (24 hours after <a href="https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg" target="_blank" rel="noopener">the Uber accident</a>, where the vehicle involved used exactly the same type of LiDAR sensor as Apollo). This study was reported in an article titled "Metamorphic Testing of Driverless Cars" by Zhou and Sun, published in <a href="https://cacm.acm.org/" target="_blank" rel="noopener"><em>Communications of the ACM</em></a>, ACM's flagship magazine (<a href="https://doi.org/10.1145/3241979" target="_blank" rel="noopener">original paper</a>; <a href="https://doi.org/10.1145/3332409" target="_blank" rel="noopener">follow-up letter</a>). Zhou covered this story in his <a href="http://doi.org/10.5281/zenodo.2636979" target="_blank" rel="noopener">keynote speech</a> at the <a href="http://www.ieeeaitests.com/html/speakers.html" target="_blank" rel="noopener">IEEE International Conference on Artificial Intelligence Testing</a> (AITest 2019) held in San Francisco East Bay, California.

Advisees


  • Graduate Advising Relationship

    Degree Research Title Advisee
    Doctor of Philosophy Un Similarity-Based Software Text case Prioritisation Liu, Chen
    Doctor of Philosophy Metamorphic Testing applied to citation systems and other border applications Witheridge, Matt
    Doctor of Philosophy (Integrated) A New Approach to the Detection of Software Security Vulnerabilities Zhu, Junting
    Doctor of Philosophy Testing Navigation Systems Brown, Joshua
    Doctor of Philosophy A Machine Learning Based Approach for SoftwareTesting. Wu, Sarah
    Doctor of Philosophy Software testing of web and cloud based applications Han, Jiacheng
    Doctor of Philosophy Testing methods for intelligent systems Iqbal, Muhammad
    Master of Philosophy - EIS A Case Study of Web Security on Social Media Advertising Liu, Yelin
    Doctor of Philosophy (Integrated) On Antomated Testing of Web Applications Selay, Elmin
uri icon