Variational Information Bottleneck for Semi-Supervised Classification

Article Article scientifique Article Variational Information Bottleneck for Semi-Supervised Classification eng 2020 author Voloshynovskyy Svyatoslav 514089 author Taran Olga 799200 author Kondah Mouad 902088 author Holotyak Taras 638661 author Rezende Danilo 943 10.3390/e22090943 1099-4300 https://www.mdpi.com/1099-4300/22/9/943 In this paper, we consider an information bottleneck (IB) framework for semi-supervised classification with several families of priors on latent space representation. We apply a variational decomposition of mutual information terms of IB. Using this decomposition we perform an analysis of several regularizers and practically demonstrate an impact of different components of variational model on the classification accuracy. We propose a new formulation of semi-supervised IB with hand crafted and learnable priors and link it to the previous methods such as semi-supervised versions of VAE (M1 + M2), AAE, CatGAN, etc. We show that the resulting model allows better understand the role of various previously proposed regularizers in semi-supervised classification task in the light of IB framework. The proposed IB semi-supervised model with hand-crafted and learnable priors is experimentally validated on MNIST under different amount of labeled data. Information bottleneck principle Deep networks Semi-supervised classification Latent space representation Hand crafted priors Learnable priors Regularization Dewey 025.063 13 Centre universitaire d'informatique 0D3 32 Département d'informatique 18 false 895 Computer Vision and Multimedia Laboratory 933 Stochastic Information Processing Group Entropy 22 9 a363063a-45d2-4ead-be92-ee626c6ce4fd Article (Published version) Public 2020Voloshynovskiy_variational_info_bottleneck.pdf 2164303 application/pdf OA OTHER openAccess Centres et instituts / Centre universitaire d'informatique Faculté des sciences / Département d'informatique

Title	Published in	Access level	OA Policy	Year	Views	Downloads

Variational Information Bottleneck for Semi-Supervised Classification	Entropy			2020	291	211

Archive ouverte UNIGE

Kondah, Mouad