Skip to main content
placeholder image

LoFGAN: Fusing Local Representations for Few-shot Image Generation

Conference Paper


Abstract


  • Given only a few available images for a novel unseen category, few-shot image generation aims to generate more data for this category. Previous works attempt to globally fuse these images by using adjustable weighted coefficients. However, there is a serious semantic misalignment between different images from a global perspective, making these works suffer from poor generation quality and diversity. To tackle this problem, we propose a novel Local-Fusion Generative Adversarial Network (LoFGAN) for few-shot image generation. Instead of using these available images as a whole, we first randomly divide them into a base image and several reference images. Next, LoFGAN matches local representations between the base and reference images based on semantic similarities, and replaces the local features with the closest related local features. In this way, LoFGAN can produce more realistic and diverse images at a more fine-grained level, and simultaneously enjoy the characteristic of semantic alignment. Furthermore, a local reconstruction loss is also proposed, which can provide better training stability and generation quality. We conduct extensive experiments on three datasets, which successfully demonstrates the effectiveness of our proposed method for few-shot image generation and downstream visual applications with limited data. Code is available at https://github.com/edward3862/LoFGAN-pytorch.

Publication Date


  • 2021

Citation


  • Gu, Z., Li, W., Huo, J., Wang, L., & Gao, Y. (2021). LoFGAN: Fusing Local Representations for Few-shot Image Generation. In Proceedings of the IEEE International Conference on Computer Vision (pp. 8443-8451). doi:10.1109/ICCV48922.2021.00835

Scopus Eid


  • 2-s2.0-85124506971

Web Of Science Accession Number


Start Page


  • 8443

End Page


  • 8451

Abstract


  • Given only a few available images for a novel unseen category, few-shot image generation aims to generate more data for this category. Previous works attempt to globally fuse these images by using adjustable weighted coefficients. However, there is a serious semantic misalignment between different images from a global perspective, making these works suffer from poor generation quality and diversity. To tackle this problem, we propose a novel Local-Fusion Generative Adversarial Network (LoFGAN) for few-shot image generation. Instead of using these available images as a whole, we first randomly divide them into a base image and several reference images. Next, LoFGAN matches local representations between the base and reference images based on semantic similarities, and replaces the local features with the closest related local features. In this way, LoFGAN can produce more realistic and diverse images at a more fine-grained level, and simultaneously enjoy the characteristic of semantic alignment. Furthermore, a local reconstruction loss is also proposed, which can provide better training stability and generation quality. We conduct extensive experiments on three datasets, which successfully demonstrates the effectiveness of our proposed method for few-shot image generation and downstream visual applications with limited data. Code is available at https://github.com/edward3862/LoFGAN-pytorch.

Publication Date


  • 2021

Citation


  • Gu, Z., Li, W., Huo, J., Wang, L., & Gao, Y. (2021). LoFGAN: Fusing Local Representations for Few-shot Image Generation. In Proceedings of the IEEE International Conference on Computer Vision (pp. 8443-8451). doi:10.1109/ICCV48922.2021.00835

Scopus Eid


  • 2-s2.0-85124506971

Web Of Science Accession Number


Start Page


  • 8443

End Page


  • 8451