Network scanning tools play an important role in network information gathering, which is the first step in penetration testing. They are tools that can help both hackers and security auditors to detect the hosts in the network environment. Choice of scanner should take consideration of both the speed and the effectiveness of the scanner. In this paper, we report on an experience of using Metamorphic Testing to test two scanners, Nmap and Masscan. Seven Metamorphic Relations for network scanners are presented, and an investigation into their application is reported. The results show the potential for Metamorphic Testing to evaluate and enable comparisons of scanners, with possible problems identified in both scanners under test.