Zhou, Zhi Q.

An independent study conducted by American researchers identified Zhou as one of the top five researchers internationally in the important research field of test oracles [1]. Another independent study conducted by European researchers identified Zhou as one of the top four researchers internationally in metamorphic testing [2].<br /><br />References:<br /><br />[1] R. A. P. Oliveira, U. Kanewala, and P. A. Nardi. "Automated test oracles: State of the art, taxonomies, and trends," In Advances in Computers, volume 95, pp. 113-199. Elsevier, 2014. Available online: <a href="http://iii.library.uow.edu.au/record=b2303118" target="_blank" rel="noopener">http://iii.library.uow.edu.au/record=b2303118</a><br /><br />[2] S. Segura, G. Fraser, A. Sanchez, and A. Ruiz-Cortes, "A survey on metamorphic testing", IEEE Transactions on Software Engineering, 42(9), pp. <strong> </strong> 805-824, 2016. Available online: <a href="http://dx.doi.org/10.1109/TSE.2016.2532875" target="_blank" rel="noopener">http://dx.doi.org/10.1109/TSE.2016.2532875</a>

Zhou has been working with the IT industry on a series of software testing projects. Those industry partners include Adobe Inc., Microsoft Corporation, Partech Systems, and the Baidu Apollo autonomous driving team, among others. An independent review of Zhou's joint work with Adobe Inc. can be found <a href="https://semiwiki.com/eda/284509-innovation-in-verification-april-2020/" title="Innovation in Verification April 2020" target="_blank" rel="noopener">here</a>, where Paul Cunningham (GM of the Verification Group at Cadence) commented that "I like this. I see it as a way to do statistical anomaly-based QA. You compare a lot of runs, looking at distributions to spot bugs. I see a lot of applications: anything performance-related, heuristic-based, machine-learning-based will be naturally statistical. Distribution analyses can then reveal more complex issues than pass/fail analyses. MT gives us tools to find those kinds of problem." And Jim Hogan commented that "This is the first of the papers we've looked at in this series which to me is more than just a feature. This paper would definitely be worth putting money behind, trying to get to production. It looks like a product, perhaps a new class of verification tool. It might even work as a startup." Interested readers may refer to the full discussion <a href="https://semiwiki.com/eda/284509-innovation-in-verification-april-2020/" target="_blank" rel="noopener">Innovation in Verification April 2020</a>.

<p>Zhou initiated a "metamorphic robustness testing" method and led his research team to use this method to verify the major citation database systems <a href="https://www.scopus.com" target="_blank" rel="noopener"><em>Scopus</em></a> and <a href="https://clarivate.com/products/web-of-science/" target="_blank" rel="noopener"><em>Web of Science</em></a>, revealing robustness problems in citation counts and journal impact factors computed by these systems. A paper titled "Metamorphic Robustness Testing: Exposing Hidden Defects in Citation Statistics and Journal Impact Factors" by Zhi Quan Zhou, T.H. Tse, and Matt Witheridge has been published in <em>IEEE Transactions on Software Engineering</em>, the top journal in the field (URL for download: <a href="https://doi.org/10.1109/TSE.2019.2915065" target="_blank" rel="noopener">https://doi.org/10.1109/TSE.2019.2915065</a>).</p>The high impact of the results lies in the finding that the presence of simple hyphens in a paper title adversely affects the citation statistics. Thus, metrics based on citation counts, which are commonly used for staff evaluations in universities, are not as robust as widely believed. This will be of interest not only to professors seeking tenure or promotion, but also to the senior management such as presidents, deans, and heads. The surprising results are applicable to all faculties in any university.<br /><br />Zhou et al.'s findings have been reported by the international media, including <a href="https://documents.uow.edu.au/~zhiquan/TheHyphenMan.pdf" target="_blank" rel="noopener"><em>The Australian</em></a>, <a href="https://www.psychologytoday.com/intl/blog/the-athletes-way/201906/one-small-hyphen-in-title-could-diminish-academic-success" target="_blank" rel="noopener"><em>Psychology Today</em></a><a href="https://www.sciencealert.com/hyphens-break-our-entire-system-of-scientific-ranking-new-analysis-reveals" target="_blank" rel="noopener"><em></em></a>, <a href="https://cacm.acm.org/news/237400-a-single-punctuation-mark-has-been-skewing-our-entire-system-of-scientific-ranking/fulltext" target="_blank" rel="noopener"><em>Communications of the ACM</em></a>, and <em><a href="https://www.eurekalert.org/" target="_blank" rel="noopener">EurekAlert!</a></em>, among others. <br /><br />On the Internet, the implications of Zhou et al.'s findings have been discussed from diverse perspectives, some of which extend beyond the conventional computing and citation domains. For example, see "<a href="https://www.slashgear.com/brave-new-world-predictions-are-coming-true-with-hyphens-31578773/" target="_blank" rel="noopener">Brave New World predictions are coming true with hyphens</a>" written by <a href="https://www.slashgear.com/author/chrisburns/" target="_blank" rel="noopener">Chris Burns</a> on <a href="https://www.slashgear.com/brave-new-world-predictions-are-coming-true-with-hyphens-31578773/" target="_blank" rel="noopener"><em>SlashGear</em></a>. Inquiries have been made from various communities including, among others, the community of patients with <a href="https://www.3cr.org.au/chronicallychilled/episode-201901021800/asher-wolf-ehlers-danlos-syndrome" target="_blank" rel="noopener">Ehlers-Danlos Syndrome</a> (a <a href="https://en.wikipedia.org/wiki/List_of_syndromes" target="_blank" rel="noopener">condition</a> that has a hyphenated name and is known to be under-reported and under-researched).<br /><br />On 5 June 2019, the <a href="http://www.webofsciencegroup.com/" target="_blank" rel="noopener">Web of Science Group</a> posted a <a href="https://clarivate.com/webofsciencegroup/download/36464/" target="_blank" rel="noopener">response</a> to Zhou et al.'s paper on the <a href="https://clarivate.com/blog/science-research-connect/web-of-science-group-response-to-metamorphic-robustness-testing-exposing-hidden-defects-in-citation-statistics-and-journal-impact-factors/" target="_blank" rel="noopener">Clarivate Analytics website</a>. Feedback from the authors was reported by <a href="https://www.sciencealert.com/hyphens-break-our-entire-system-of-scientific-ranking-new-analysis-reveals" target="_blank" rel="noopener"><em>ScienceAlert</em></a>.  On 7 August 2019, Zhou et al.'s work was reported by <em><a href="https://go.nature.com/2TdWoM6" target="_blank" rel="noopener"><strong>Nature</strong></a> </em>(https://go.<strong>nature.com</strong>/2TdWoM6).

On March 10, 2018, Zhou and his MPhil student Liqun Sun detected a critical fault in the real-world self-driving system <a href="http://apollo.auto" target="_blank" rel="noopener">Baidu Apollo</a>, and they reported this finding to the Apollo team on the same day (this was <strong>8 days before</strong> <a href="https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg" target="_blank" rel="noopener">a fatal accident of a Uber's self-driving vehicle in Tempe, Arizona</a>). The fault caused the self-driving car's LiDAR perception system to ignore pedestrians or other obstacles on the roadway. The Apollo team confirmed the fault on March 19, 2018 (24 hours after <a href="https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg" target="_blank" rel="noopener">the Uber accident</a>, where the vehicle involved used exactly the same type of LiDAR sensor as Apollo). This study was reported in an article titled "Metamorphic Testing of Driverless Cars" by Zhou and Sun, published in <a href="https://cacm.acm.org/" target="_blank" rel="noopener"><em>Communications of the ACM</em></a>, ACM's flagship magazine (<a href="https://doi.org/10.1145/3241979" target="_blank" rel="noopener">original paper</a>; <a href="https://doi.org/10.1145/3332409" target="_blank" rel="noopener">follow-up letter</a>). Zhou covered this story in his <a href="http://doi.org/10.5281/zenodo.2636979" target="_blank" rel="noopener">keynote speech</a> at the <a href="http://www.ieeeaitests.com/html/speakers.html" target="_blank" rel="noopener">IEEE International Conference on Artificial Intelligence Testing</a> (AITest 2019) held in San Francisco East Bay, California.

An independent study conducted by American researchers identified Zhou as one of the top five researchers internationally in the important research field of test oracles [1]. Another independent study conducted by European researchers identified Zhou as one of the top four researchers internationally in metamorphic testing [2].<br /><br />References:<br /><br />[1] R. A. P. Oliveira, U. Kanewala, and P. A. Nardi. "Automated test oracles: State of the art, taxonomies, and trends," In Advances in Computers, volume 95, pp. 113-199. Elsevier, 2014. Available online: <a href="http://iii.library.uow.edu.au/record=b2303118" target="_blank" rel="noopener">http://iii.library.uow.edu.au/record=b2303118</a><br /><br />[2] S. Segura, G. Fraser, A. Sanchez, and A. Ruiz-Cortes, "A survey on metamorphic testing", IEEE Transactions on Software Engineering, 42(9), pp. <strong> </strong> 805-824, 2016. Available online: <a href="http://dx.doi.org/10.1109/TSE.2016.2532875" target="_blank" rel="noopener">http://dx.doi.org/10.1109/TSE.2016.2532875</a>

Zhou has been working with the IT industry on a series of software testing projects. Those industry partners include Adobe Inc., Microsoft Corporation, Partech Systems, and the Baidu Apollo autonomous driving team, among others. An independent review of Zhou's joint work with Adobe Inc. can be found <a href="https://semiwiki.com/eda/284509-innovation-in-verification-april-2020/" title="Innovation in Verification April 2020" target="_blank" rel="noopener">here</a>, where Paul Cunningham (GM of the Verification Group at Cadence) commented that "I like this. I see it as a way to do statistical anomaly-based QA. You compare a lot of runs, looking at distributions to spot bugs. I see a lot of applications: anything performance-related, heuristic-based, machine-learning-based will be naturally statistical. Distribution analyses can then reveal more complex issues than pass/fail analyses. MT gives us tools to find those kinds of problem." And Jim Hogan commented that "This is the first of the papers we've looked at in this series which to me is more than just a feature. This paper would definitely be worth putting money behind, trying to get to production. It looks like a product, perhaps a new class of verification tool. It might even work as a startup." Interested readers may refer to the full discussion <a href="https://semiwiki.com/eda/284509-innovation-in-verification-april-2020/" target="_blank" rel="noopener">Innovation in Verification April 2020</a>.

<p>Zhou initiated a "metamorphic robustness testing" method and led his research team to use this method to verify the major citation database systems <a href="https://www.scopus.com" target="_blank" rel="noopener"><em>Scopus</em></a> and <a href="https://clarivate.com/products/web-of-science/" target="_blank" rel="noopener"><em>Web of Science</em></a>, revealing robustness problems in citation counts and journal impact factors computed by these systems. A paper titled "Metamorphic Robustness Testing: Exposing Hidden Defects in Citation Statistics and Journal Impact Factors" by Zhi Quan Zhou, T.H. Tse, and Matt Witheridge has been published in <em>IEEE Transactions on Software Engineering</em>, the top journal in the field (URL for download: <a href="https://doi.org/10.1109/TSE.2019.2915065" target="_blank" rel="noopener">https://doi.org/10.1109/TSE.2019.2915065</a>).</p>The high impact of the results lies in the finding that the presence of simple hyphens in a paper title adversely affects the citation statistics. Thus, metrics based on citation counts, which are commonly used for staff evaluations in universities, are not as robust as widely believed. This will be of interest not only to professors seeking tenure or promotion, but also to the senior management such as presidents, deans, and heads. The surprising results are applicable to all faculties in any university.<br /><br />Zhou et al.'s findings have been reported by the international media, including <a href="https://documents.uow.edu.au/~zhiquan/TheHyphenMan.pdf" target="_blank" rel="noopener"><em>The Australian</em></a>, <a href="https://www.psychologytoday.com/intl/blog/the-athletes-way/201906/one-small-hyphen-in-title-could-diminish-academic-success" target="_blank" rel="noopener"><em>Psychology Today</em></a><a href="https://www.sciencealert.com/hyphens-break-our-entire-system-of-scientific-ranking-new-analysis-reveals" target="_blank" rel="noopener"><em></em></a>, <a href="https://cacm.acm.org/news/237400-a-single-punctuation-mark-has-been-skewing-our-entire-system-of-scientific-ranking/fulltext" target="_blank" rel="noopener"><em>Communications of the ACM</em></a>, and <em><a href="https://www.eurekalert.org/" target="_blank" rel="noopener">EurekAlert!</a></em>, among others. <br /><br />On the Internet, the implications of Zhou et al.'s findings have been discussed from diverse perspectives, some of which extend beyond the conventional computing and citation domains. For example, see "<a href="https://www.slashgear.com/brave-new-world-predictions-are-coming-true-with-hyphens-31578773/" target="_blank" rel="noopener">Brave New World predictions are coming true with hyphens</a>" written by <a href="https://www.slashgear.com/author/chrisburns/" target="_blank" rel="noopener">Chris Burns</a> on <a href="https://www.slashgear.com/brave-new-world-predictions-are-coming-true-with-hyphens-31578773/" target="_blank" rel="noopener"><em>SlashGear</em></a>. Inquiries have been made from various communities including, among others, the community of patients with <a href="https://www.3cr.org.au/chronicallychilled/episode-201901021800/asher-wolf-ehlers-danlos-syndrome" target="_blank" rel="noopener">Ehlers-Danlos Syndrome</a> (a <a href="https://en.wikipedia.org/wiki/List_of_syndromes" target="_blank" rel="noopener">condition</a> that has a hyphenated name and is known to be under-reported and under-researched).<br /><br />On 5 June 2019, the <a href="http://www.webofsciencegroup.com/" target="_blank" rel="noopener">Web of Science Group</a> posted a <a href="https://clarivate.com/webofsciencegroup/download/36464/" target="_blank" rel="noopener">response</a> to Zhou et al.'s paper on the <a href="https://clarivate.com/blog/science-research-connect/web-of-science-group-response-to-metamorphic-robustness-testing-exposing-hidden-defects-in-citation-statistics-and-journal-impact-factors/" target="_blank" rel="noopener">Clarivate Analytics website</a>. Feedback from the authors was reported by <a href="https://www.sciencealert.com/hyphens-break-our-entire-system-of-scientific-ranking-new-analysis-reveals" target="_blank" rel="noopener"><em>ScienceAlert</em></a>.  On 7 August 2019, Zhou et al.'s work was reported by <em><a href="https://go.nature.com/2TdWoM6" target="_blank" rel="noopener"><strong>Nature</strong></a> </em>(https://go.<strong>nature.com</strong>/2TdWoM6).

On March 10, 2018, Zhou and his MPhil student Liqun Sun detected a critical fault in the real-world self-driving system <a href="http://apollo.auto" target="_blank" rel="noopener">Baidu Apollo</a>, and they reported this finding to the Apollo team on the same day (this was <strong>8 days before</strong> <a href="https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg" target="_blank" rel="noopener">a fatal accident of a Uber's self-driving vehicle in Tempe, Arizona</a>). The fault caused the self-driving car's LiDAR perception system to ignore pedestrians or other obstacles on the roadway. The Apollo team confirmed the fault on March 19, 2018 (24 hours after <a href="https://en.wikipedia.org/wiki/Death_of_Elaine_Herzberg" target="_blank" rel="noopener">the Uber accident</a>, where the vehicle involved used exactly the same type of LiDAR sensor as Apollo). This study was reported in an article titled "Metamorphic Testing of Driverless Cars" by Zhou and Sun, published in <a href="https://cacm.acm.org/" target="_blank" rel="noopener"><em>Communications of the ACM</em></a>, ACM's flagship magazine (<a href="https://doi.org/10.1145/3241979" target="_blank" rel="noopener">original paper</a>; <a href="https://doi.org/10.1145/3332409" target="_blank" rel="noopener">follow-up letter</a>). Zhou covered this story in his <a href="http://doi.org/10.5281/zenodo.2636979" target="_blank" rel="noopener">keynote speech</a> at the <a href="http://www.ieeeaitests.com/html/speakers.html" target="_blank" rel="noopener">IEEE International Conference on Artificial Intelligence Testing</a> (AITest 2019) held in San Francisco East Bay, California.