Skip to main content
placeholder image

RDA: Reciprocal Distribution Alignment for��Robust Semi-supervised Learning

Chapter


Abstract


  • In this work, we propose Reciprocal Distribution Alignment (RDA) to address semi-supervised learning (SSL), which is a hyperparameter-free framework that is independent of confidence threshold and works with both the matched (conventionally) and the mismatched class distributions. Distribution mismatch is an often overlooked but more general SSL scenario where the labeled and the unlabeled data do not fall into the identical class distribution. This may lead to the model not exploiting the labeled data reliably and drastically degrade the performance of SSL methods, which could not be rescued by the traditional distribution alignment. In RDA, we enforce a reciprocal alignment on the distributions of the predictions from two classifiers predicting pseudo-labels and complementary labels on the unlabeled data. These two distributions, carrying complementary information, could be utilized to regularize each other without any prior of class distribution. Moreover, we theoretically show that RDA maximizes the input-output mutual information. Our approach achieves promising performance in SSL under a variety of scenarios of mismatched distributions, as well as the conventional matched SSL setting. Our code is available at: https://github.com/NJUyued/RDA4RobustSSL.

UOW Authors


  •   Wang, Lei
  •   Zhou, Luping (external author)

Publication Date


  • 2022

Edition


Citation


  • Duan, Y., Qi, L., Wang, L., Zhou, L., & Shi, Y. (2022). RDA: Reciprocal Distribution Alignment for��Robust Semi-supervised Learning. In Unknown Book (Vol. 13690 LNCS, pp. 533-549). doi:10.1007/978-3-031-20056-4_31

International Standard Book Number (isbn) 13


  • 9783031200557

Scopus Eid


  • 2-s2.0-85144568538

Web Of Science Accession Number


Book Title


  • Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Start Page


  • 533

End Page


  • 549

Place Of Publication


Abstract


  • In this work, we propose Reciprocal Distribution Alignment (RDA) to address semi-supervised learning (SSL), which is a hyperparameter-free framework that is independent of confidence threshold and works with both the matched (conventionally) and the mismatched class distributions. Distribution mismatch is an often overlooked but more general SSL scenario where the labeled and the unlabeled data do not fall into the identical class distribution. This may lead to the model not exploiting the labeled data reliably and drastically degrade the performance of SSL methods, which could not be rescued by the traditional distribution alignment. In RDA, we enforce a reciprocal alignment on the distributions of the predictions from two classifiers predicting pseudo-labels and complementary labels on the unlabeled data. These two distributions, carrying complementary information, could be utilized to regularize each other without any prior of class distribution. Moreover, we theoretically show that RDA maximizes the input-output mutual information. Our approach achieves promising performance in SSL under a variety of scenarios of mismatched distributions, as well as the conventional matched SSL setting. Our code is available at: https://github.com/NJUyued/RDA4RobustSSL.

UOW Authors


  •   Wang, Lei
  •   Zhou, Luping (external author)

Publication Date


  • 2022

Edition


Citation


  • Duan, Y., Qi, L., Wang, L., Zhou, L., & Shi, Y. (2022). RDA: Reciprocal Distribution Alignment for��Robust Semi-supervised Learning. In Unknown Book (Vol. 13690 LNCS, pp. 533-549). doi:10.1007/978-3-031-20056-4_31

International Standard Book Number (isbn) 13


  • 9783031200557

Scopus Eid


  • 2-s2.0-85144568538

Web Of Science Accession Number


Book Title


  • Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

Start Page


  • 533

End Page


  • 549

Place Of Publication