Variational Information Bottleneck for Semi-Supervised Classification

Voloshynovskyy, Svyatoslav; Taran, Olga; Kondah, Mouad; Holotyak, Taras; Rezende, Danilo

doi:10.3390/e22090943

Scientific article

English

Variational Information Bottleneck for Semi-Supervised Classification

Contributors

Voloshynovskyy, Svyatoslav; Taran, Olga; Kondah, Mouad; Holotyak, Taras; Rezende, Danilo

Published in

Entropy, vol. 22, no. 9, 943

Publication date

2020

Abstract

In this paper, we consider an information bottleneck (IB) framework for semi-supervised classification with several families of priors on latent space representation. We apply a variational decomposition of mutual information terms of IB. Using this decomposition we perform an analysis of several regularizers and practically demonstrate an impact of different components of variational model on the classification accuracy. We propose a new formulation of semi-supervised IB with hand crafted and learnable priors and link it to the previous methods such as semi-supervised versions of VAE (M1 + M2), AAE, CatGAN, etc. We show that the resulting model allows better understand the role of various previously proposed regularizers in semi-supervised classification task in the light of IB framework. The proposed IB semi-supervised model with hand-crafted and learnable priors is experimentally validated on MNIST under different amount of labeled data.

Keywords

Information bottleneck principle
Deep networks
Semi-supervised classification
Latent space representation
Hand crafted priors
Learnable priors
Regularization

UNIGE affiliation entities

UNIGE research groups

Citation (ISO format)

VOLOSHYNOVSKYY, Svyatoslav et al. Variational Information Bottleneck for Semi-Supervised Classification. In: Entropy, 2020, vol. 22, n° 9, p. 943. doi: 10.3390/e22090943

Article (Published version)

OA OTHER

Identifiers

PID : unige:144404
DOI : 10.3390/e22090943

Additional URL for this publication

https://www.mdpi.com/1099-4300/22/9/943

Journal ISSN

1099-4300

297views

222downloads

Creation

20/10/2020 13:32:00

First validation

20/10/2020 13:32:00

Update

15/03/2023 23:18:15

Status update

15/03/2023 23:18:15

Last indexation

31/10/2024 20:18:36

Variational Information Bottleneck for Semi-Supervised Classification

Contributors

Published in

Publication date

Abstract

Keywords

UNIGE affiliation entities

UNIGE research groups

Citation (ISO format)

Main files (1)

Identifiers

Additional URL for this publication

Journal ISSN

Technical informations

Creation

First validation

Update

Status update

Last indexation