Rapid developments in artificial intelligence and image processing have presented many new opportunities for defect detection in manufacturing processes. In this work, an intelligent image processing system has been developed to monitor inter-layer deposition quality during a wire arc additive manufacturing (WAAM) process. This system reveals the feasibility and future potential of using computer vision knowledge in WAAM. Information produced from this system is to be used in conjunction with other quality monitoring systems to verify the quality of fabricated components. It is tailored to identify the presence of defects relating to lack of fusion and voids immediately after the deposition of a given layer. The image processing system is built upon the YOLOv3 architecture and through moderate changes on anchor settings and achieves 53% precision on surface anomaly detection and 100% accuracy in identifying the fabricated components’ location, providing a prerequisite for high-precision assessment of welding quality. The work presented in this paper presents an inter-layer vision-based defect monitoring system in WAAM and serves to highlight the feasibility of developing such intelligent computer vision systems for monitoring the WAAM process for defects.