Variational Leakage: The Role of Information Complexity in Privacy Leakage. (arXiv:2106.02818v2 [cs.LG] UPDATED)

We study the role of information complexity in privacy leakage about an
attribute of an adversary’s interest, which is not known a priori to the system
designer. Considering the supervised representation learning setup and using
neural networks to parameterize the variational bounds of information
quantities, we study the impact of the following factors on the amount of
information leakage: information complexity regularizer weight, latent space
dimension, the cardinalities of the known utility and unknown sensitive
attribute sets, the correlation between utility and sensitive attributes, and a
potential bias in a sensitive attribute of adversary’s interest. We conduct
extensive experiments on Colored-MNIST and CelebA datasets to evaluate the
effect of information complexity on the amount of intrinsic leakage.