In recent years, opportunities to deal with digital images on the Internet have increased due to the information society, there is a great demand for techniques such as super-resolution processing to make images more beautiful. When super-resolution processing is performed on natural images such as scenery, edges are emphasized to obtain clear images. However, when super-resolution processing is applied to a facial image, the wrinkle and stains of the skin as well as the emphasis of hair and eyes are emphasized, so super-resolution processing on the skin part is not suitable. Therefore, in the previous study, we proposed a method to perform facial correction using non-linear filter on skin part, and tried to solve this problem. This method is composed of super-resolution processing and facial correction processing, and it was possible to realize a super-resolution processing with a sharp sense for facial images. However, we also confirmed that there was a problem that the image quality deteriorated according to the skin color detection accuracy at the time of image synthesis of each processed image. Therefore, in this paper, we study the skin color detection method and try to improve the image quality.