{"title": "Twin Auxilary Classifiers GAN", "book": "Advances in Neural Information Processing Systems", "page_first": 1330, "page_last": 1339, "abstract": "Conditional generative models enjoy significant progress over the past few years. One of the popular conditional models is Auxiliary Classifier GAN (AC-GAN) that generates highly discriminative images by extending the loss function of GAN with an auxiliary classifier. However, the diversity of the generated samples by AC-GAN tends to decrease as the number of classes increases. In this paper, we identify the source of low diversity issue theoretically and propose a practical solution to the problem. We show that the auxiliary classifier in AC-GAN imposes perfect separability, which is disadvantageous when the supports of the class distributions have significant overlap. To address the issue, we propose Twin Auxiliary Classifiers Generative Adversarial Net (TAC-GAN) that adds a new player that interacts with other players (the generator and the discriminator) in GAN. Theoretically, we demonstrate that our TAC-GAN can effectively minimize the divergence between generated and real data distributions. Extensive experimental results show that our TAC-GAN can successfully replicate the true data distributions on simulated data, and significantly improves the diversity of class-conditional image generation on real datasets.", "full_text": "Twin Auxiliary Classi\ufb01ers GAN\n\nMingming Gong *1,3, Yanwu Xu *1, Chunyuan Li2, Kun Zhang3, and Kayhan Batmanghelich1\n\n1Department of Biomedical Informatics, University of Pittsburgh, {mig73,yanwuxu,kayhan}@pitt.edu\n\n2Microsoft Research, Redmond, cl319@duke.edu\n\n3Department of Philosophy, Carnegie Mellon University, kunz1@cmu.edu\n\nAbstract\n\nConditional generative models enjoy remarkable progress over the past few years.\nOne of the popular conditional models is Auxiliary Classi\ufb01er GAN (AC-GAN),\nwhich generates highly discriminative images by extending the loss function of\nGAN with an auxiliary classi\ufb01er. However, the diversity of the generated samples\nby AC-GAN tends to decrease as the number of classes increases, hence limiting its\npower on large-scale data. In this paper, we identify the source of the low diversity\nissue theoretically and propose a practical solution to solve the problem. We\nshow that the auxiliary classi\ufb01er in AC-GAN imposes perfect separability, which\nis disadvantageous when the supports of the class distributions have signi\ufb01cant\noverlap. To address the issue, we propose Twin Auxiliary Classi\ufb01ers Generative\nAdversarial Net (TAC-GAN) that further bene\ufb01ts from a new player that interacts\nwith other players (the generator and the discriminator) in GAN. Theoretically, we\ndemonstrate that TAC-GAN can effectively minimize the divergence between the\ngenerated and real-data distributions. Extensive experimental results show that our\nTAC-GAN can successfully replicate the true data distributions on simulated data,\nand signi\ufb01cantly improves the diversity of class-conditional image generation on\nreal datasets.\n\n1\n\nIntroduction\n\nGenerative Adversarial Networks (GANs) [1] are a framework to learn the data generating distribution\nimplicitly. GANs mimic sampling from a target distribution by training a generator that maps samples\ndrawn from a canonical distribution to the data space. A distinctive feature of GANs is that the\ndiscriminator that evaluates the separability of the real and generated data distributions [1\u20134]. If the\ndiscriminator can hardly distinguish between real and generated data, the generator is likely to provide\na good approximation to the true data distribution. To generate high-\ufb01delity images, much recent\nresearch has focused on designing more advanced network architectures [5, 6], developing more\nstable objective functions [7\u20139, 3], enforcing appropriate constraints on the discriminator [10\u201312], or\nimproving training techniques [7, 13].\nConditional GANs (cGANs) [14] are a variant of GANs that take advantage of extra information\n(condition) and have been widely used for generation of class-conditioned images [15\u201318] and text [19,\n20]. A major difference between cGANs and GANs is that the cGANs feed the condition to both the\ngenerator and the discriminator to lean the joint distributions of images and the condition random\nvariables. Most methods feed the conditional information by concatenating it (or its embedding)\nwith the input or the feature vector at speci\ufb01c layers [14, 21, 15, 22, 23, 20]. Recently, Projection-\ncGAN [24] improves the quality of the generated images using a speci\ufb01c discriminator that takes the\ninner product between the embedding of the conditioning variable and the feature vector of the input\n\n(cid:63)Equal Contribution\nThe code is available at https://github.com/batmanlab/twin_ac.\n\n33rd Conference on Neural Information Processing Systems (NeurIPS 2019), Vancouver, Canada.\n\n\fimage, obtaining state-of-the-art class-conditional image generation on large-scale datasets such as\nImageNet1000 [25].\nAmong cGANs, the Auxiliary Classi\ufb01er GAN (AC-GAN) has received much attention due to its\nsimplicity and extensibility to various applications [17]. AC-GAN incorporates the conditional\ninformation (label) by training the GAN discriminator with an additional classi\ufb01cation loss. AC-GAN\nis able to generate high-quality images and has been extended to various learning problems, such as\ntext-to-image generation [26]. However, it is reported in the literature [17, 24] that as the number\nof labels increases, AC-GAN tends to generate near-identical images for most classes. Miyato et\nal. [24] observed this phenomenon on ImageNet1000 and conjectured that the auxiliary classi\ufb01er\nmight encourage the generator to produce less diverse images so that they can be easily discernable.\nDespite these insightful \ufb01ndings, the exact source of the low-diversity problem is unclear, let alone\nits remedy. In this paper, we aim to provide an understanding of this phenomenon and accordingly\ndevelop a new method that is able to generate diverse and realistic images. First, we show that due to\na missing term in the objective of AC-GAN, it does not faithfully minimize the divergence between\nreal and generated conditional distribution. We show that missing that term can result in a degenerate\nsolution, which explains the lack of diversity in the generated data. Based on our understanding, we\nintroduce a new player in the min-max game of AC-GAN that enables us to estimate the missing\nterm in the objective. The resulting method properly estimates the divergence between real and\ngenerated conditional distributions and signi\ufb01cantly increases sample diversity within each class.\nWe call our method Twin Auxiliary Classi\ufb01ers GAN (TAC-GAN) since the new player is also a\nclassi\ufb01er. Compared to AC-GAN, our TAC-GAN successfully replicates the real data distributions\non simulated data and signi\ufb01cantly improves the quality and diversity of the class-conditional image\ngeneration on CIFAR100 [27], VGGFace2 [28], and ImageNet1000 [25] datasets. In particular, to\nour best knowledge, our TAC-GAN is the \ufb01rst cGAN method that can generate good quality images\non the VGGFace dataset, demonstrating the advantage of TAC-GAN on \ufb01ne-grained datasets.\n\n2 Method\n\nIn this section, we review the Generative Adversarial Network (GAN) [1] and its conditional variant\n(cGAN) [14]. We review one of the most popular variants of cGAN, which is called Auxiliary\nClassi\ufb01er GAN (AC-GAN) [17]. We \ufb01rst provide an understanding of the observation of low-\ndiversity samples generated by AC-GAN from a distribution matching perspective. Second, based\non our new understanding of the problem, we propose a new method that enables learning of real\ndistributions and increasing sample diversity.\n\n2.1 Background\nGiven a training set {xi}n\ni=1 \u2286 X drawn from an unknown distribution PX, GAN estimates PX by\nspecifying a distribution QX implicitly. Instead of an explicit parametrization, it trains a generator\nfunction G(Z) that maps samples from a canonical distribution, i.e., Z \u223c PZ, to the training data.\nThe generator is obtained by \ufb01nding an equilibrium of the following mini-max game that effectively\nminimizes the Jensen-Shannon Divergence (JSD) between QX and PX:\n\nmin\n\nG\n\nmax\n\nD\n\nE\n\nX\u223cPX\n\n[log D(X)] + E\nZ\u223cPZ\n\n[log(1 \u2212 D(G(Z)))],\n\n(1)\n\nwhere D is a discriminator. Notice that the QX is not directly modeled.\ni=1 \u2286 X \u00d7 Y drawn from the joint\nGiven a pair of observation (x) and a condition (y), {xi, y}n\ndistribution (xi, y) \u223c PXY , the goal of cGAN is to estimate a conditional distribution PX|Y . Let\nQX|Y denote the conditional distribution speci\ufb01ed by a generator G(Y, Z) and QXY := QX|Y PY .\nA generic cGAN trains G to implicitly minimize the JSD divergence between the joint distributions\nQXY and PXY :\n\nmin\n\nG\n\nmax\n\nD\n\nE\n\n(X,Y )\u223cPXY\n\n[log D(X, Y )] +\n\nE\n\nZ\u223cPZ ,Y \u223cPY\n\n[log(1 \u2212 D(G(Z, Y ), Y ))].\n\n(2)\n\nIn general, Y can be a continuous or discrete variable. In this paper, we focus on case that Y is the\n(discrete) class label, i.e., Y = {1, . . . , K}.\n\n2\n\n\f(cid:124)\n(cid:124)\n\n(cid:123)(cid:122)\n\nE\na(cid:13)\n\u2212\u03bbc\n\n(cid:125)\n\n(cid:125)\n\n2.2\n\nInsight on Auxiliary Classi\ufb01er GAN (AC-GAN)\n\nAC-GAN introduces a new player C which is a classi\ufb01er that interacts with the D and G players. We\nuse Qc\nY |X to denote the conditional distribution induced by C. The AC-GAN optimization combines\nthe original GAN loss with cross-entropy classi\ufb01cation loss:\nmin\nG,C\n\nLAC(G, D,C) = E\nX\u223cPX\n\n[log(1 \u2212 D(G(Z, Y )))]\n\nZ\u223cPZ ,Y \u223cPY\n\n[log D(X)] +\n\nmax\n\nD\n\n\u2212 \u03bbc\n\nE\n\n(X,Y )\u223cPXY\n\n(cid:123)(cid:122)\n\n[log C(X, Y )]\nb(cid:13)\n\n(cid:125)\n\nE\n\nZ\u223cPZ ,Y \u223cPY\n\n(cid:124)\n\n(cid:123)(cid:122)\n\n[log(C(G(Z, Y ), Y ))]\n\n, (3)\n\n(cid:21)\n\nE\n\nE\n\n(cid:20)\n\nlog\n\n=\n\n=\n\nE\n\nE\n\nE\n\n(X,Y )\u223cPXY\n\n(X,Y )\u223cPXY\n\n(X,Y )\u223cPXY\n\n(X,Y )\u223cPXY\n\n\u2212HP (Y |X) + b(cid:13) =\n\n[log C(X, Y )]\n[log Qc(Y |X)]\n\n(X,Y )\u223cPXY\n= KL(PY |X||Qc\n\nc(cid:13)\nwhere \u03bbc is a hyperparameter balancing GAN and auxiliary classi\ufb01cation losses.\nHere we decompose the objective of AC-GAN into three terms. Clearly, the \ufb01rst term a(cid:13) cor-\nresponds to the Jensen-Shannon divergence (JSD) between QX and PX. The second term b(cid:13)\nis the cross-entropy loss on real data.\nIt is straightforward to show that the second term min-\nimizes Kullback-Leibler (KL) divergence between the real data distribution PY |X and the dis-\ntribution Qc\nY |X speci\ufb01ed by C. To show that, we can add the negative conditional entropy,\n\u2212HP (Y |X) = E(X,Y )\u223cPXY [log P (Y |X)], to b(cid:13), we have\n[log P (Y |X)] \u2212\n[log P (Y |X)] \u2212\nP (Y |X)\nQc(Y |X)\n\n(4)\nSince the negative conditional entropy \u2212HP (Y |X) is a constant term, minimizing b(cid:13) w.r.t. the\nnetwork parameters in C effectively minimizes the KL divergence between PY |X and Qc\nThe third term c(cid:13) is the cross-entropy loss on the generated data. Similarly, if one adds the negative\nentropy \u2212HQ(Y |X) = E(X,Y )\u223cQXY [log Q(Y |X)] to c(cid:13) and obtain the following result:\n\u2212HQ(Y |X) + c(cid:13) =\nY |X ).\nWhen updating C, \u2212HQ(Y |X) can be considered a constant term, thus minimizing c(cid:13) w.r.t. C\neffectively minimizes the KL divergence between QY |X and Qc\nY |X. However, when updating G,\n\u2212HQ(Y |X) cannot be considered as a constant term, because QY |X is the conditional distribution\nspeci\ufb01ed by the generator G. AC-GAN ignores \u2212HQ(Y |X) and only minimizes c(cid:13) when updating\nG in the optimization procedure, which fails to minimize the KL divergence between QY |X and\nY |X. We hypothesize that the likely reason behind low diversity samples generated by AC-GAN is\nQc\nthat it fails to account for the missing term while updating G. In fact, the following theorem shows\nthat AC-GAN can converge to a degenerate distribution:\nTheorem 1. Suppose PX = QX. Given an auxiliary classi\ufb01er C which speci\ufb01es a conditional\nY |X, the optimal G\u2217 that minimizes c(cid:13) induces the following degenerate conditional\ndistribution Qc\ndistribution Q\u2217\nY |X,\n\u2217\nQ\n\nif k=arg maxi Qc(Y = i|X = x),\n\n[log Qc(Y |X)] = KL(QY |X||Qc\n\n(Y = k|X = x) =\n\n[log Q(Y |X)] \u2212\n\n(cid:26) 1,\n\n(X,Y )\u223cQXY\n\n(X,Y )\u223cQXY\n\nY |X ).\n\nY |X.\n\n(5)\n\n0,\n\notherwise.\n\nE\n\nE\n\nProof is given in Section S1 of the Supplementary Material (SM). Theorem 1 shows that, even when\nthe marginal distributions are perfectly matched by GAN loss a(cid:13), AC-GAN is not able to model the\nprobability when class distributions have support overlaps. It tends to generate data in which Y is\ndeterministically related to X. This means that the generated images for each class are con\ufb01ned by\nthe regions induced by the decision boundaries of the auxiliary classi\ufb01er C, which fails to replicate\nconditional distribution Qc\nY |X, implied by C, and reduces the distributional support of each class. The\ntheoretical result is consistent with the empirical results in [24] that AC-GAN generates discriminable\nimages with low intra-class diversity. It is thus essential to incorporate the missing term, \u2212HQ(Y |X),\nin the objective to penalize this behavior and minimize the KL divergence between QY |X and Qc\nY |X.\n\n3\n\n\fFigure 1: Illustration of the proposed TAC-GAN. The generator G synthesizes fake samples X\nconditioned input label Y . The discriminator D distinguishes between real/fake samples. The\nauxiliary classi\ufb01er C is trained to classify labels on both real and fake pairs, while the proposed twin\nauxiliary classi\ufb01er C mi is trained on fake pairs only.\n\n2.3 Twin Auxiliary Classi\ufb01ers GAN (TAC-GAN)\nOur analysis in the previous section motivates adding the missing term, \u2212HQ(Y |X), back to the\nobjective function. While minimizing c(cid:13) forces G to concentrate the conditional density mass on\nthe training data, \u2212HQ(Y |X) works in the opposite direction by increasing the entropy. However,\nestimating \u2212HQ(Y |X) is a challenging task since we do not have access to QY |X. Various methods\nhave been proposed to estimate the (conditional) entropy, such as [29\u201332]; however, these estimators\ncannot be easily used as an objective function to learn G via backpropagation. Below we propose to\nestimate the conditional entropy by adding a new player in the mini-max game.\nThe general idea is to introduce an additional auxiliary classi\ufb01er, C mi, that aims to identify the labels\nof the samples drawn from QX|Y ; the low-diversity case makes this task easy for C mi. Similar to\nGAN, the generator tries to compete with the C mi. The overall idea of TAC-GAN is illustrated in\nFigure 1. In the following, we demonstrate its connection with minimizing \u2212HQ(Y |X).\nProposition: Let us assume, without loss of generality, that all classes are equally likely 1 (i.e.,\nK ). Minimizing \u2212HQ(Y |X) is equivalent to minimizing (1) the mutual information\nP (Y = k) = 1\nbetween Y and X and (2) the JSD between the conditional distributions {QX|Y =1, . . . , QX|Y =K}.\n\nProof.\n\nIQ(Y, X) = H(Y ) \u2212 HQ(Y |X) = HQ(X) \u2212 HQ(X|Y )\n\nK(cid:88)\nK(cid:88)\n\nk=1\n\nk=1\n\n= \u2212 1\nK\n\n=\n\n1\nK\n\nK(cid:88)\n\nk=1\n\nE\n\nX\u223cQX|Y =k\n\nlog Q(X) +\n\n1\nK\n\nE\n\nX\u223cQX|Y =k\n\nlog Q(X|Y = k)\n\nKL(QX|Y =k||QX ) = JSD(QX|Y =1, . . . , QX|Y =K).\n\n(6)\n\n(1) follows from the fact that entropy of Y is constant with respect to Q, (2) is shown above.\nBased on the connection between \u2212HQ(Y |X) and JSD, we extend the two-player minimax approach\nin GAN [1] to minimize the JSD between multiple distributions. More speci\ufb01cally, we use another\nauxiliary classi\ufb01er C mi whose last layer is a softmax function that predicts the probability of X\nbelong to a class Y = k. We de\ufb01ne the following minimax game:\n\nmin\n\nG\n\nmax\nCmi\n\nV (G, C mi) =\n\nE\n\nZ\u223cPZ ,Y \u223cPY\n\n[log(C mi(G(Z, Y ), Y ))].\n\n(7)\n\nThe following theorem shows that the minimax game can effectively minimize the JSD between\n{QX|Y =1, . . . , QX|Y =K}.\nTheorem 2. Let U (G) = max\nV (G, C mi). The global mininum of the minimax game is achieved if\nCmi\nand only if QX|Y =1 = QX|Y =2 = \u00b7\u00b7\u00b7 = QX|Y =K. At the optimal point, U (G) achieves the value\n\u2212K log K.\n\n1If the dataset is imbalanced, we can apply biased batch sampling to enforce this condition.\n\n4\n\nGeneratorDiscriminatorXAAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==YAAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==GAAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRgx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RTW1jc2t4rbpZ3dvf2D8uFRS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAfj25nffkKleSwfzCRBP6JDyUPOqLFS465frrhVdw6ySrycVCBHvV/+6g1ilkYoDRNU667nJsbPqDKcCZyWeqnGhLIxHWLXUkkj1H42P3RKzqwyIGGsbElD5urviYxGWk+iwHZG1Iz0sjcT//O6qQmv/YzLJDUo2WJRmApiYjL7mgy4QmbExBLKFLe3EjaiijJjsynZELzll1dJ66LquVWvcVmp3eRxFOEETuEcPLiCGtxDHZrAAOEZXuHNeXRenHfnY9FacPKZY/gD5/MHm/WMyw==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRgx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RTW1jc2t4rbpZ3dvf2D8uFRS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAfj25nffkKleSwfzCRBP6JDyUPOqLFS465frrhVdw6ySrycVCBHvV/+6g1ilkYoDRNU667nJsbPqDKcCZyWeqnGhLIxHWLXUkkj1H42P3RKzqwyIGGsbElD5urviYxGWk+iwHZG1Iz0sjcT//O6qQmv/YzLJDUo2WJRmApiYjL7mgy4QmbExBLKFLe3EjaiijJjsynZELzll1dJ66LquVWvcVmp3eRxFOEETuEcPLiCGtxDHZrAAOEZXuHNeXRenHfnY9FacPKZY/gD5/MHm/WMyw==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRgx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RTW1jc2t4rbpZ3dvf2D8uFRS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAfj25nffkKleSwfzCRBP6JDyUPOqLFS465frrhVdw6ySrycVCBHvV/+6g1ilkYoDRNU667nJsbPqDKcCZyWeqnGhLIxHWLXUkkj1H42P3RKzqwyIGGsbElD5urviYxGWk+iwHZG1Iz0sjcT//O6qQmv/YzLJDUo2WJRmApiYjL7mgy4QmbExBLKFLe3EjaiijJjsynZELzll1dJ66LquVWvcVmp3eRxFOEETuEcPLiCGtxDHZrAAOEZXuHNeXRenHfnY9FacPKZY/gD5/MHm/WMyw==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRgx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RTW1jc2t4rbpZ3dvf2D8uFRS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAfj25nffkKleSwfzCRBP6JDyUPOqLFS465frrhVdw6ySrycVCBHvV/+6g1ilkYoDRNU667nJsbPqDKcCZyWeqnGhLIxHWLXUkkj1H42P3RKzqwyIGGsbElD5urviYxGWk+iwHZG1Iz0sjcT//O6qQmv/YzLJDUo2WJRmApiYjL7mgy4QmbExBLKFLe3EjaiijJjsynZELzll1dJ66LquVWvcVmp3eRxFOEETuEcPLiCGtxDHZrAAOEZXuHNeXRenHfnY9FacPKZY/gD5/MHm/WMyw==ZAAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae2oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBuMGM3g==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae2oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBuMGM3g==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae2oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBuMGM3g==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ae2oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBuMGM3g==DAAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GNRDx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RTW1jc2t4rbpZ3dvf2D8uFRS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAfj25nffkKleSwfzCRBP6JDyUPOqLFS465frrhVdw6ySrycVCBHvV/+6g1ilkYoDRNU667nJsbPqDKcCZyWeqnGhLIxHWLXUkkj1H42P3RKzqwyIGGsbElD5urviYxGWk+iwHZG1Iz0sjcT//O6qQmv/YzLJDUo2WJRmApiYjL7mgy4QmbExBLKFLe3EjaiijJjsynZELzll1dJ66LquVWvcVmp3eRxFOEETuEcPLiCGtxDHZrAAOEZXuHNeXRenHfnY9FacPKZY/gD5/MHl2mMyA==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GNRDx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RTW1jc2t4rbpZ3dvf2D8uFRS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAfj25nffkKleSwfzCRBP6JDyUPOqLFS465frrhVdw6ySrycVCBHvV/+6g1ilkYoDRNU667nJsbPqDKcCZyWeqnGhLIxHWLXUkkj1H42P3RKzqwyIGGsbElD5urviYxGWk+iwHZG1Iz0sjcT//O6qQmv/YzLJDUo2WJRmApiYjL7mgy4QmbExBLKFLe3EjaiijJjsynZELzll1dJ66LquVWvcVmp3eRxFOEETuEcPLiCGtxDHZrAAOEZXuHNeXRenHfnY9FacPKZY/gD5/MHl2mMyA==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GNRDx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RTW1jc2t4rbpZ3dvf2D8uFRS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAfj25nffkKleSwfzCRBP6JDyUPOqLFS465frrhVdw6ySrycVCBHvV/+6g1ilkYoDRNU667nJsbPqDKcCZyWeqnGhLIxHWLXUkkj1H42P3RKzqwyIGGsbElD5urviYxGWk+iwHZG1Iz0sjcT//O6qQmv/YzLJDUo2WJRmApiYjL7mgy4QmbExBLKFLe3EjaiijJjsynZELzll1dJ66LquVWvcVmp3eRxFOEETuEcPLiCGtxDHZrAAOEZXuHNeXRenHfnY9FacPKZY/gD5/MHl2mMyA==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GNRDx5bsB/QhrLZTtq1m03Y3Qgl9Bd48aCIV3+SN/+N2zYHbX0w8Hhvhpl5QSK4Nq777RTW1jc2t4rbpZ3dvf2D8uFRS8epYthksYhVJ6AaBZfYNNwI7CQKaRQIbAfj25nffkKleSwfzCRBP6JDyUPOqLFS465frrhVdw6ySrycVCBHvV/+6g1ilkYoDRNU667nJsbPqDKcCZyWeqnGhLIxHWLXUkkj1H42P3RKzqwyIGGsbElD5urviYxGWk+iwHZG1Iz0sjcT//O6qQmv/YzLJDUo2WJRmApiYjL7mgy4QmbExBLKFLe3EjaiijJjsynZELzll1dJ66LquVWvcVmp3eRxFOEETuEcPLiCGtxDHZrAAOEZXuHNeXRenHfnY9FacPKZY/gD5/MHl2mMyA==CmiAAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMdgLh4jmAcka5idzCZj5rHMzAphyT948aCIV//Hm3/jJNmDJhY0FFXddHdFCWfG+v63V1hb39jcKm6Xdnb39g/Kh0cto1JNaJMornQnwoZyJmnTMstpJ9EUi4jTdjSuz/z2E9WGKXlvJwkNBR5KFjOCrZNa9YdMsGm/XPGr/hxolQQ5qUCORr/81RsokgoqLeHYmG7gJzbMsLaMcDot9VJDE0zGeEi7jkosqAmz+bVTdOaUAYqVdiUtmqu/JzIsjJmIyHUKbEdm2ZuJ/3nd1MbXYcZkkloqyWJRnHJkFZq9jgZMU2L5xBFMNHO3IjLCGhPrAiq5EILll1dJ66Ia+NXg7rJSu8njKMIJnMI5BHAFNbiFBjSBwCM8wyu8ecp78d69j0VrwctnjuEPvM8fntSPJQ==AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMdgLh4jmAcka5idzCZj5rHMzAphyT948aCIV//Hm3/jJNmDJhY0FFXddHdFCWfG+v63V1hb39jcKm6Xdnb39g/Kh0cto1JNaJMornQnwoZyJmnTMstpJ9EUi4jTdjSuz/z2E9WGKXlvJwkNBR5KFjOCrZNa9YdMsGm/XPGr/hxolQQ5qUCORr/81RsokgoqLeHYmG7gJzbMsLaMcDot9VJDE0zGeEi7jkosqAmz+bVTdOaUAYqVdiUtmqu/JzIsjJmIyHUKbEdm2ZuJ/3nd1MbXYcZkkloqyWJRnHJkFZq9jgZMU2L5xBFMNHO3IjLCGhPrAiq5EILll1dJ66Ia+NXg7rJSu8njKMIJnMI5BHAFNbiFBjSBwCM8wyu8ecp78d69j0VrwctnjuEPvM8fntSPJQ==AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMdgLh4jmAcka5idzCZj5rHMzAphyT948aCIV//Hm3/jJNmDJhY0FFXddHdFCWfG+v63V1hb39jcKm6Xdnb39g/Kh0cto1JNaJMornQnwoZyJmnTMstpJ9EUi4jTdjSuz/z2E9WGKXlvJwkNBR5KFjOCrZNa9YdMsGm/XPGr/hxolQQ5qUCORr/81RsokgoqLeHYmG7gJzbMsLaMcDot9VJDE0zGeEi7jkosqAmz+bVTdOaUAYqVdiUtmqu/JzIsjJmIyHUKbEdm2ZuJ/3nd1MbXYcZkkloqyWJRnHJkFZq9jgZMU2L5xBFMNHO3IjLCGhPrAiq5EILll1dJ66Ia+NXg7rJSu8njKMIJnMI5BHAFNbiFBjSBwCM8wyu8ecp78d69j0VrwctnjuEPvM8fntSPJQ==AAAB7XicbVDLSgNBEOyNrxhfUY9eBoPgKeyKoMdgLh4jmAcka5idzCZj5rHMzAphyT948aCIV//Hm3/jJNmDJhY0FFXddHdFCWfG+v63V1hb39jcKm6Xdnb39g/Kh0cto1JNaJMornQnwoZyJmnTMstpJ9EUi4jTdjSuz/z2E9WGKXlvJwkNBR5KFjOCrZNa9YdMsGm/XPGr/hxolQQ5qUCORr/81RsokgoqLeHYmG7gJzbMsLaMcDot9VJDE0zGeEi7jkosqAmz+bVTdOaUAYqVdiUtmqu/JzIsjJmIyHUKbEdm2ZuJ/3nd1MbXYcZkkloqyWJRnHJkFZq9jgZMU2L5xBFMNHO3IjLCGhPrAiq5EILll1dJ66Ia+NXg7rJSu8njKMIJnMI5BHAFNbiFBjSBwCM8wyu8ecp78d69j0VrwctnjuEPvM8fntSPJQ==CAAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GOxF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoJJfe53nlBpHssHM03Qj+hI8pAzaqzUrA/KFbfqLkDWiZeTCuRoDMpf/WHM0gilYYJq3fPcxPgZVYYzgbNSP9WYUDahI+xZKmmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGtn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUUWZsNiUbgrf68jppX1U9t+o1ryu1uzyOIpzBOVyCBzdQg3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AleWMxw==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GOxF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoJJfe53nlBpHssHM03Qj+hI8pAzaqzUrA/KFbfqLkDWiZeTCuRoDMpf/WHM0gilYYJq3fPcxPgZVYYzgbNSP9WYUDahI+xZKmmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGtn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUUWZsNiUbgrf68jppX1U9t+o1ryu1uzyOIpzBOVyCBzdQg3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AleWMxw==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GOxF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoJJfe53nlBpHssHM03Qj+hI8pAzaqzUrA/KFbfqLkDWiZeTCuRoDMpf/WHM0gilYYJq3fPcxPgZVYYzgbNSP9WYUDahI+xZKmmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGtn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUUWZsNiUbgrf68jppX1U9t+o1ryu1uzyOIpzBOVyCBzdQg3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AleWMxw==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GOxF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoJJfe53nlBpHssHM03Qj+hI8pAzaqzUrA/KFbfqLkDWiZeTCuRoDMpf/WHM0gilYYJq3fPcxPgZVYYzgbNSP9WYUDahI+xZKmmE2s8Wh87IhVWGJIyVLWnIQv09kdFI62kU2M6ImrFe9ebif14vNeGtn3GZpAYlWy4KU0FMTOZfkyFXyIyYWkKZ4vZWwsZUUWZsNiUbgrf68jppX1U9t+o1ryu1uzyOIpzBOVyCBzdQg3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AleWMxw==NoiseTwinAuxiliaryInput+LabelAuxiliaryReal/Fake+Real+Label+YAAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==Input+LabelXAAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lEqMeiF48t2A9oQ9lsJ+3azSbsboQS+gu8eFDEqz/Jm//GbZuDtj4YeLw3w8y8IBFcG9f9dgobm1vbO8Xd0t7+weFR+fikreNUMWyxWMSqG1CNgktsGW4EdhOFNAoEdoLJ3dzvPKHSPJYPZpqgH9GR5CFn1Fip2R2UK27VXYCsEy8nFcjRGJS/+sOYpRFKwwTVuue5ifEzqgxnAmelfqoxoWxCR9izVNIItZ8tDp2RC6sMSRgrW9KQhfp7IqOR1tMosJ0RNWO96s3F/7xeasIbP+MySQ1KtlwUpoKYmMy/JkOukBkxtYQyxe2thI2poszYbEo2BG/15XXSvqp6btVrXlfqt3kcRTiDc7gED2pQh3toQAsYIDzDK7w5j86L8+58LFsLTj5zCn/gfP4AtbmM3A==Fake+RealYAAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==Input+LabelYAAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==AAAB6HicbVBNS8NAEJ3Ur1q/qh69LBbBU0lE0GPRi8cW7Ie0oWy2k3btZhN2N0IJ/QVePCji1Z/kzX/jts1BWx8MPN6bYWZekAiujet+O4W19Y3NreJ2aWd3b/+gfHjU0nGqGDZZLGLVCahGwSU2DTcCO4lCGgUC28H4dua3n1BpHst7M0nQj+hQ8pAzaqzUeOiXK27VnYOsEi8nFchR75e/eoOYpRFKwwTVuuu5ifEzqgxnAqelXqoxoWxMh9i1VNIItZ/ND52SM6sMSBgrW9KQufp7IqOR1pMosJ0RNSO97M3E/7xuasJrP+MySQ1KtlgUpoKYmMy+JgOukBkxsYQyxe2thI2ooszYbEo2BG/55VXSuqh6btVrXFZqN3kcRTiBUzgHD66gBndQhyYwQHiGV3hzHp0X5935WLQWnHzmGP7A+fwBtz2M3Q==\fA complete proof of Theorem 2 is given in Section S2 of the SM. It is worth noting that the global\noptimum of U (G) cannot be achieved in our model because of other terms in our TAC-GAN objective\nfunction, which is obtained by combing (7) and the original AC-GAN objective (3):\nLTAC(G, D, C, C mi) = LAC(G, D, C) + \u03bbcV (G, C mi).\n\n(8)\n\nmin\nG,C\n\nmax\nD,Cmi\n\nThe following theorem provides approximation guarantees for the joint distribution PXY , justifying\nthe validity of our proposed approach.\nTheorem 3. Let PY X and QY X denote the data distribution and the distribution speci\ufb01ed by the\nY |X denote the conditional distribution of Y given X speci\ufb01ed by\ngenerator G, respectively. Let Qc\nthe auxiliary classi\ufb01er C. We have\nJSD(PXY , QXY ) \u2264 2c1\n\n2JSD(PX , QX ) + c2\n\n2KL(QY |X||Qc\n\n(cid:112)\n\n(cid:113)\n\n(cid:113)\n\nwhere c1 and c2 are upper bounds of 1\n2\nmeasure), respectively. A proof of Theorem 3 is provided in Section S3 of the SM.\n\n2\n\n2KL(PY |X||Qc\n\n(cid:82) |PY |X (y|x)|\u00b5(x, y) and 1\n\nY |X ) + c2\n\n(cid:82) |QX (x)|\u00b5(x) (\u00b5 is a \u03c3-\ufb01nite\n\nY |X ),\n\n3 Related Works\n\nTAC-GAN learns an unbiased distribution. Shu et al. [33] \ufb01rst show that AC-GAN tends to\ndown-sample the data points near the decision boundary, causing a biased estimation of the true\ndistribution. From a Lagrangian perspective, they consider AC-GAN as minimizing JSD(PX , QX )\nwith constraints enforced by classi\ufb01cation losses. If \u03bbc is very large such that JSD(PX , QX ) become\nless effective, the generator will push the generated images away from the boundary. However, on real\ndatasets, we can also observe low diversity when \u03bbc is small, which cannot be explained by the analy-\nsis in [33]. We take a different perspective by constraining JSD(PX , QX ) to be small and investigate\nthe properties of the conditional QY |X. Our analysis suggests that even when JSD(PX , QX ) = 0,\nthe AC-GAN cross-entropy loss can still result in biased estimate of QY |X, reducing the support of\neach class in the generated distribution, compared to the true distribution. Furthermore, we propose a\nsolution that can remedy the low diversity problem based on our understandings.\n\nConnecting TAC-GAN with Projection cGAN. AC-GAN was once the state-of-the-art method\nbefore the advent of Projection cGAN [24]. Projection cGAN, AC-GAN, and our TAC-GAN share\nthe similar spirits in that image generation performance can be improved when the joint distribution\nmatching problem is decomposed into two easier sub-problems: marginal matching and conditional\nmatching [32]. Projection cGAN decomposes the density ratio, while AC-GAN and TAC-GAN\ndirectly decompose the distribution. Both Projection cGAN and TAC-GAN are theoretically sound\nwhen using the cross-entropy loss. However, in practice, hinge loss is often preferred for real data. In\nthis case, Projection cGAN loses the theoretical guarantee, while TAC-GAN is less affected, because\nonly the GAN loss is replaced by the hinge loss.\n\n4 Experiments\n\nWe \ufb01rst compare the distribution matching ability of AC-GAN, Projection cGAN, and our TAC-GAN\non Mixture of Gaussian (MoG) and MNIST [34] synthetic data. We evaluate the image generation\nperformance of TAC-GAN on three image datatest including CIFAR100 [27], ImageNet1000 [25] and\nVGGFace2 [28]. In our implementation, the twin auxiliary classi\ufb01ers share the same convolutional\nlayers, which means TAC-GAN only adds a negligible computation cost to AC-GAN. The detailed\nexperiment setups are shown in the SM. We implemented TAC-GAN in Pytorch. To illustrate\nthe algorithm, we submit the implementation on the synthetic datasets in SM. The source code to\nreproduce the full experimental results will be made public on GitHub.\n\n4.1 MoG Synthetic Data\n\nWe start with a simulated dataset to verify that TAC-GAN can accurately match the target distribution.\nWe draw samples from a one-dimensional MoG distribution with three Gaussian components, labeled\n\n5\n\n\fFigure 2: Comparison of sample quality on a synthetic MoG dataset.\n\nFigure 3: The MMD evaluation. The x-axis means the distance between the means of adjacent\nGaussian components (dm). Lower score is better.\n\nas Class_0, Class_1, and Class_2, respectively. The standard deviations of the three components\nare \ufb01xed to \u03c30 = 1, \u03c31 = 2, and \u03c32 = 3. The differences between the means are set to \u00b51 \u2212 \u00b50 =\n\u00b52 \u2212 \u00b51 = dm, in which dm ranges from 1 to 5. These values are chosen such that the supports of\nthe three distributions have different overlap sizes. We detail the experimental setup in Section S4 of\nthe SM.\nFigure 2 shows the ground truth density functions when \u00b50 = 0, \u00b51 = 3, \u00b52 = 6 and the estimated\nones by AC-GAN, TAC-GAN, and Projection cGAN. The estimated density function is obtained\nby applying kernel density estimation [35] on the generated data. When using cross-entropy loss,\nAC-GAN learns a biased distribution where all the classes are perfectly separated by the classi\ufb01cation\ndecision function, verifying our Theorem 1. Both our TAC-GAN and Projection cGAN can accurately\nlearn the original distribution. Using Hinge loss, our model can still learn the distribution well, while\nneither AC-GAN nor Projection cGAN can replicate the real distribution (see Supplementary S4\nfor more experiments). We also conduct simulation on a 2D dataset and the details are given in\nSupplementary S5. The results show that our TAC-GAN is able to learn the true data distribution.\nFigure 3 reports the Maximum Mean Discrepancy (MMD) [36] distance between the real data and\ngenerated data for different dm values. Here all the GAN models are trained using cross-entropy\nloss (log loss). The TAC-GAN produces near-zero MMD values for all dm\u2019s, meaning that the data\ngenerated by TAC-GAN is very close to the ground truth data. Projection cGAN performs slightly\nworse than TAC-GAN and AC-GAN generates data that have a large MMD distance to the true data.\n4.2 Overlapping MNIST\nFollowing experiments in [33] to show that AC-GAN learns a biased distribution, we use the\noverlapping MNIST dataset to demonstrate the robustness of our TAC-GAN. We randomly sample\nfrom MNIST training set to construct two image groups: Group A contains 5,000 digit \u20181\u2019 and\n5,000 digit \u20180\u2019, while Group B contains 5,000 digit \u20182\u2019 and 5,000 digit \u20180\u2019,to simulate overlapping\ndistributions, where digit \u20180\u2019 appears in both groups. Note that the ground truth proportion of digit\n\u20180\u2019, \u20181\u2019 and \u20182\u2019 in this dataset are 0.5, 0.25 and 0.25, respectively.\nFigure 4 (a) shows the generated images under different \u03bbc. It shows that AC-GAN tends to down\nsample \u20180\u2019 as \u03bbc increases, while TAC-GAN can always generate \u20180\u2019s in both groups. To quantitatively\nmeasure the distribution of generated images, we pre-train a \u201cperfect\u201d classi\ufb01er on a MNIST subset\nonly containing digit \u20180\u2019, \u20181\u2019, and \u20182\u2019, and use the classi\ufb01er to predict the labels of the generated data.\nFigure 4 (b) reports the label proportions for the generated images. It shows that the label proportion\nproduced by TAC-GAN is very close to the ground truth values regardless of \u03bbc, while AC-GAN\ngenerates less \u20180\u2019s as \u03bbc increases. More results and detail setting are shown in Section S6 of the SM.\n\n6\n\n0510150.00.10.20.30.40.5DensityTarget Distribution0510150.00.10.20.30.40.5DensityAC-GAN (Log Loss)0510150.00.10.20.30.40.5DensityTAC-GAN (Log Loss)0510150.00.10.20.30.40.5DensityProjection cGAN (Log Loss)0510150.00.10.20.30.40.5DensityAC-GAN (Hinge Loss)0510150.00.10.20.30.40.5DensityTAC-GAN (Hinge Loss)0510150.00.10.20.30.40.5DensityProjection cGAN (Hinge Loss)12345dm(Distance between means)010203040506070MMDMarginal12345dm (Distance between means)0.00.20.40.60.8MMDClass_0Projection cGANAC-GANTAC-GAN12345dm (Distance between means)0102030405060MMDClass_112345dm (Distance between means)0100200300400500600700MMDClass_2\f(a) Generated samples\n\n(b) Quantitative result\n\nFigure 4: (a) Visualization of the generated MNIST digits with various \u03bbc values. For each section,\nthe top row digits are sampled from group A and the bottom row digits are from group B. (b) The\nlabel proportion for generated digits of two methods. The ground truth proportion for digit 0,1,2 is\n[0.5, 0.25, 0.25], visualized as dashed dark lines.\n\nFigure 5: Generated images from \ufb01ve classes of CIFAR100.\n\nFigure 6: Impact of \u03bbc on the image generation quality on CIFAR100.\n\n4.3 CIFAR100\n\nCIFAR100 [27] has 100 classes, each of which contains 500 training images and 100 testing images\nat the resolution of 32 \u00d7 32. The current best deep classi\ufb01cation model achieves 91.3% accuracy on\nthis dataset [37], which suggests that the class distributions may have certain support overlaps.\nFigure 5 shows the generated images for \ufb01ve randomly selected classes. AC-GAN generates images\nwith low intra-class diversity. Both TAC-GAN and Projection cGAN generate visually appealing and\ndiverse images. We provide the generated images for all the classes in Section S6 of the SM.\nTo quantitatively compare the generated images, we consider the two popular evaluation criteria,\nincluding Inception Score (IS) [38] and Fr\u00e9chet Inception Distance (FID) [39]. We also use the\nrecently proposed Learned Perceptual Image Patch Similarity (LPIPS), which measures the perceptual\ndiversity within each class [40]. The scores are reported in Table 1. TAC-GAN achieves lower FID\nthan Projection cGAN, and outperforms AC-GAN by a large margin, which demonstrates the ef\ufb01cacy\nof the twin auxiliary classi\ufb01ers. We report the scores for all the classes in Section S7 of the SM. In\nSection S7.2 of the SM, we explore the compatibility of our model with the techniques that increase\ndiversity of unsupervised GANs. Speci\ufb01cally, we combine pacGAN [41] with AC-GAN and our\nTAC-GAN, and the results show that pacGAN can improve both AC-GAN and TAC-GAN, but it\ncannot fully address the drawbacks of AC-GAN.\nEffects of hyper-parameters \u03bbc. We study the impact of \u03bbc on AC-GAN and TAC-GAN, and\nreport results under different \u03bbc values in Figure 6. It shows that TAC-GAN is robust to \u03bbc, while\nAC-GAN requires a very small \u03bbc to achieve good scores. Even so, AC-GAN generates images with\nlow intra-class diversity, as shown in Figure 5.\n\n7\n\nReal\tDataAC-GAN,\t!\"=1TAC-GAN,!\"=1AC-GAN,\t!\"=2TAC-GAN,\t!\"=2(a)AAAB6nicbVBNS8NAEJ34WetX1aOXxSLUS0lU0GPRi8eK9gPaUDbbTbt0swm7E6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvSKQw6Lrfzsrq2vrGZmGruL2zu7dfOjhsmjjVjDdYLGPdDqjhUijeQIGStxPNaRRI3gpGt1O/9cS1EbF6xHHC/YgOlAgFo2ilhwo965XKbtWdgSwTLydlyFHvlb66/ZilEVfIJDWm47kJ+hnVKJjkk2I3NTyhbEQHvGOpohE3fjY7dUJOrdInYaxtKSQz9fdERiNjxlFgOyOKQ7PoTcX/vE6K4bWfCZWkyBWbLwpTSTAm079JX2jOUI4toUwLeythQ6opQ5tO0YbgLb68TJrnVe+i6t1flms3eRwFOIYTqIAHV1CDO6hDAxgM4Ble4c2Rzovz7nzMW1ecfOYI/sD5/AGJp41N(b)AAAB6nicbVBNS8NAEJ34WetX1aOXxSLUS0lU0GPRi8eK9gPaUDbbSbt0swm7G6GE/gQvHhTx6i/y5r9x2+agrQ8GHu/NMDMvSATXxnW/nZXVtfWNzcJWcXtnd2+/dHDY1HGqGDZYLGLVDqhGwSU2DDcC24lCGgUCW8Hoduq3nlBpHstHM07Qj+hA8pAzaqz0UAnOeqWyW3VnIMvEy0kZctR7pa9uP2ZphNIwQbXueG5i/Iwqw5nASbGbakwoG9EBdiyVNELtZ7NTJ+TUKn0SxsqWNGSm/p7IaKT1OApsZ0TNUC96U/E/r5Oa8NrPuExSg5LNF4WpICYm079JnytkRowtoUxxeythQ6ooMzadog3BW3x5mTTPq95F1bu/LNdu8jgKcAwnUAEPrqAGd1CHBjAYwDO8wpsjnBfn3fmYt644+cwR/IHz+QOLLI1O0.51.01.52.02.53.0Weight of Classifier c0.10.20.30.40.5Relative FrequencyAC-GAN, 0TAC-GAN, 0AC-GAN, 1TAC-GAN, 1AC-GAN, 2TAC-GAN, 2AppleFishTreeMotorcycleMountainTAC-GAN, \ud835\udf06\ud835\udc50=1.0Projection cGANAC-GAN, \ud835\udf06\ud835\udc50=0.20.20.40.60.81.0Weight of Classifier6810Inception ScoreTAC-GANAC-GAN0.20.40.60.81.0Weight of Classifier20406080FID0.20.40.60.81.0Weight of Classifier0.00.10.20.3LPIPS\fFigure 7: Comparison of generated face samples from three identities in VGGFace2 dataset.\n\nTable 1: The quantitative results of all models on three datasets.\n\nTAC-GAN (Ours) (\u03bbc = 1)\n\nProjection cGAN\n\nIS \u2191\n\nAC-GAN (\u03bbc = 1)\nFID \u2193\n5.37 \u00b1 0.064\n82.45\n7.26 \u00b1 0.113 184.41\n27.81 \u00b1 0.29\n95.70\n25.96 \u00b1 0.32\n31.90\n\nMethods\nMetrics\n\nCIFAR100\n\nImageNet1000\nVGGFace200\nVGGFace500\nVGGFace1000\nVGGFace2000\n\n4.4\n\nImageNet1000\n\nIS \u2191\n\n9.34 \u00b1 0.077\n28.86 \u00b1 0.298\n48.94 \u00b1 0.63\n77.76 \u00b1 1.61\n108.89 \u00b1 2.63\n109.04 \u00b1 2.44\n\nFID \u2193\n7.22\n23.75\n29.12\n12.42\n13.60\n13.79\n\nIS \u2191\n\n9.56 \u00b1 0.133\n38.05 \u00b1 0.790\n32.50 \u00b1 0.44\n35.96 \u00b1 0.62\n71.15 \u00b1 0.93\n79.51 \u00b1 1.03\n\nFID \u2193\n8.92\n22.77\n66.23\n43.10\n24.07\n22.42\n\nWe further apply TAC-GAN to the large-scale ImageNet dataset [25] containing 1000 classes, each of\nwhich has around 1,300 images. We pre-process the data by center-cropping and resizing the images\nto 128 \u00d7 128. We detail the experimental setup and attach generated images in Section S8 of the SM.\nTable 1 reports the IS and FID metrics of all models. Our TAC-GAN again outperforms AC-GAN\nby a large margin. In addition, TAC-GAN has lower IS than Projection cGAN. We hypothesize\nthat TAC-GAN has a chance to generate images that do not belong to the given class in the overlap\nregions, because it aims to model the true conditional distribution.\n4.5 VGGFace2\n\nVGGFace2 [28] is a large-scale face recognition dataset, with around 362 images for each person.\nIts main difference to CIFAR100 and ImageNet1000 is that this dataset is more \ufb01ne-grained with\nsmaller intra-class diversities, making the generative task more dif\ufb01cult. We resize the center-cropped\nimages to 64 \u00d7 64. To compare different algorithms, we randomly choose 200, 500, 1000 and 2000\nidentities to construct the VGGFace200, VGGFace500 VGGFACE1000 and VGGFACE2000 datasets,\nrespectively.\nFigure 7 shows the generated face images for \ufb01ve randomly selected identities from VGGFACE200.\nAC-GAN collapses to the class center, generating very similar images for each class. Though\nProjection cGAN generate diverse images, it has blurry effects. Our TAC-GAN generates diverse\nand sharp images. To quantitatively compare the methods, we \ufb01netune a Inception Net [42] classi\ufb01er\non the face data and then use it to calculate IS and FID score. We report IS and FID scores for all\nthe methods in Table 1. It shows that TAC-GAN produces much better/higher IS better/lower FID\nscore than Projection cGAN, which is consistent with the qualitative observations. These results\nsuggest that TAC-GAN is a promising method for \ufb01ne-grained datasets. More generated identities\nare attached in in section S9 of the SM.\n\n5 Conclusion\n\nIn this paper, we have theoretically analyzed the low intra-class diversity problem of the widely\nused AC-GAN method from a distribution matching perspective. We showed that the auxiliary\nclassi\ufb01er in AC-GAN imposes perfect separability, which is disadvantageous when the supports of\nthe class distributions have signi\ufb01cant overlaps. Based on the analysis, we further proposed the Twin\nAuxiliary Classi\ufb01ers GAN (TAC-GAN) method, which introduces an additional auxiliary classi\ufb01er\nto adversarially play with the players in AC-GAN. We demonstrated the ef\ufb01cacy of the proposed\n\n8\n\nID1ID2ID3TAC-GANProjection cGANAC-GAN\fmethod both theoretically and empirically. TAC-GAN can resolve the issue of AC-GAN to learn an\nunbiased distribution, and generate high-quality samples on \ufb01ne-grained image datasets.\n\nAcknowledgments\n\nThis work was partially supported by NIH Award Number 1R01HL141813-01, NSF 1839332\nTripod+X, and SAP SE. We gratefully acknowledge the support of NVIDIA Corporation with the\ndonation of the Titan X Pascal GPU used for this research. We were also grateful for the computational\nresources provided by Pittsburgh SuperComputing grant number TG-ASC170024.\n\nReferences\n[1] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio.\n\nGenerative adversarial nets. In NIPS, 2014.\n\n[2] Sebastian Nowozin, Botond Cseke, and Ryota Tomioka. f-GAN: Training generative neural samplers using\n\nvariational divergence minimization. In NIPS, pages 271\u2013279, 2016.\n\n[3] Chun-Liang Li, Wei-Cheng Chang, Yu Cheng, Yiming Yang, and Barnab\u00e1s P\u00f3czos. MMD GAN: Towards\n\ndeeper understanding of moment matching network. In NIPS, pages 2203\u20132213, 2017.\n\n[4] Mikolaj Binkowski, Dougal J. Sutherland, Michael Arbel, and Arthur Gretton. Demystifying MMD GANs.\n\nCoRR, abs/1801.01401, 2018.\n\n[5] Alec Radford, Luke Metz, and Soumith Chintala. Unsupervised representation learning with deep\n\nconvolutional generative adversarial networks. In ICLR, 2016.\n\n[6] Han Zhang, Ian J. Goodfellow, Dimitris N. Metaxas, and Augustus Odena. Self-attention generative\n\nadversarial networks. arXiv:1805.08318, 2018.\n\n[7] Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved\n\ntechniques for training GAN. In NIPS, pages 2234\u20132242, 2016.\n\n[8] Martin Arjovsky, Soumith Chintala, and L\u00e9on Bottou. Wasserstein generative adversarial networks. In\n\nICML, pages 214\u2013223, 2017.\n\n[9] Xudong Mao, Qing Li, Haoran Xie, Raymond YK Lau, Zhen Wang, and Stephen Paul Smolley. Least\n\nsquares generative adversarial networks. In ICCV, pages 2794\u20132802, 2017.\n\n[10] Ishaan Gulrajani, Faruk Ahmed, Martin Arjovsky, Vincent Dumoulin, and Aaron C Courville. Improved\n\ntraining of wasserstein GAN. In NIPS, pages 5767\u20135777, 2017.\n\n[11] Lars Mescheder, Sebastian Nowozin, and Andreas Geiger. Which training methods for GANs do actually\n\nconverge? In ICML, 2018.\n\n[12] Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for\n\ngenerative adversarial networks. In ICLR, 2018.\n\n[13] Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale GAN training for high \ufb01delity natural\n\nimage synthesis. In ICLR, 2019.\n\n[14] M. Mirza and S. Osindero. Conditional generative adversarial nets. arXiv preprint arXiv:1411.1784, 2014.\n\n[15] Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional\n\nadversarial networks. arxiv, 2016.\n\n[16] Anh Nguyen, Jeff Clune, Yoshua Bengio, Alexey Dosovitskiy, and Jason Yosinski. Plug & play generative\nnetworks: Conditional iterative generation of images in latent space. In CVPR, pages 4467\u20134477, 2017.\n\n[17] Augustus Odena, Christopher Olah, and Jonathon Shlens. Conditional image synthesis with auxiliary\n\nclassi\ufb01er GANs. In ICML, pages 2642\u20132651. JMLR. org, 2017.\n\n[18] Dingdong Yang, Seunghoon Hong, Yunseok Jang, Tianchen Zhao, and Honglak Lee. Diversity-sensitive\n\nconditional generative adversarial networks. In ICLR, 2019.\n\n[19] Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee.\nGenerative adversarial text to image synthesis. In Maria Florina Balcan and Kilian Q. Weinberger, editors,\nICML, pages 1060\u20131069, 2016.\n\n9\n\n\f[20] Han Zhang, Tao Xu, Hongsheng Li, Shaoting Zhang, Xiaogang Wang, Xiaolei Huang, and Dimitris N\nMetaxas. StackGAN: Text to photo-realistic image synthesis with stacked generative adversarial networks.\nIn ICCV, pages 5907\u20135915, 2017.\n\n[21] Emily L Denton, Soumith Chintala, Rob Fergus, et al. Deep generative image models using a laplacian\n\npyramid of adversarial networks. In NIPS, pages 1486\u20131494, 2015.\n\n[22] Guim Perarnau, Joost Van De Weijer, Bogdan Raducanu, and Jose M \u00c1lvarez. Invertible conditional gans\n\nfor image editing. arXiv preprint arXiv:1611.06355, 2016.\n\n[23] Vincent Dumoulin, Ishmael Belghazi, Ben Poole, Olivier Mastropietro, Alex Lamb, Martin Arjovsky, and\n\nAaron Courville. Adversarially learned inference. arXiv preprint arXiv:1606.00704, 2016.\n\n[24] Takeru Miyato and Masanori Koyama. cGANs with projection discriminator. In ICLR, 2018.\n\n[25] Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang,\nAndrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large\nScale Visual Recognition Challenge. IJCV, 115(3):211\u2013252, 2015.\n\n[26] Ayushman Dash, John Cristian Borges Gamboa, Sheraz Ahmed, Marcus Liwicki, and Muhammad Zeshan\nAfzal. Tac-gan-text conditioned auxiliary classi\ufb01er generative adversarial network. arXiv preprint\narXiv:1703.06412, 2017.\n\n[27] Alex Krizhevsky, Vinod Nair, and Geoffrey Hinton. CIFAR-100 (canadian institute for advanced research).\n\n[28] Qiong Cao, Li Shen, Weidi Xie, Omkar M. Parkhi, and Andrew Zisserman. Vggface2: A dataset for\n\nrecognising faces across pose and age. In FG. IEEE Computer Society, 2018.\n\n[29] Kumar Sricharan, Dennis Wei, and Alfred O Hero. Ensemble estimators for multivariate entropy estimation.\n\nIEEE transactions on information theory, 59(7):4374\u20134388, 2013.\n\n[30] Shashank Singh and Barnab\u00e1s P\u00f3czos. Finite-sample analysis of \ufb01xed-k nearest neighbor density functional\n\nestimators. In NIPS, pages 1217\u20131225, 2016.\n\n[31] Weihao Gao, Sreeram Kannan, Sewoong Oh, and Pramod Viswanath. Estimating mutual information for\n\ndiscrete-continuous mixtures. In NIPS, pages 5986\u20135997, 2017.\n\n[32] Chunyuan Li, Hao Liu, Changyou Chen, Yuchen Pu, Liqun Chen, Ricardo Henao, and Lawrence Carin.\n\nALICE: Towards understanding adversarial learning for joint distribution matching. In NIPS, 2017.\n\n[33] Rui Shu, Hung Bui, and Stefano Ermon. Ac-gan learns a biased distribution.\n\n[34] Yann LeCun. The MNIST database of handwritten digits. http://yann. lecun. com/exdb/mnist/.\n\n[35] Emanuel Parzen. On estimation of a probability density function and mode. The annals of mathematical\n\nstatistics, 1962.\n\n[36] Arthur Gretton, Karsten M. Borgwardt, Malte J. Rasch, Bernhard Sch\u00f6lkopf, and Alexander Smola. A\n\nkernel two-sample test. J. Mach. Learn. Res., 2012.\n\n[37] Yanping Huang, Yonglong Cheng, Dehao Chen, HyoukJoong Lee, Jiquan Ngiam, Quoc V Le, and Zhifeng\nChen. Gpipe: Ef\ufb01cient training of giant neural networks using pipeline parallelism. arXiv preprint\narXiv:1811.06965, 2018.\n\n[38] Tim Salimans, Ian J. Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved\n\ntechniques for training gans. CoRR, abs/1606.03498, 2016.\n\n[39] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, G\u00fcnter Klambauer, and Sepp\nHochreiter. Gans trained by a two time-scale update rule converge to a nash equilibrium. CoRR,\nabs/1706.08500, 2017.\n\n[40] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable\n\neffectiveness of deep features as a perceptual metric. In CVPR, pages 586\u2013595, 2018.\n\n[41] Zinan Lin, Ashish Khetan, Giulia Fanti, and Sewoong Oh. Pacgan: The power of two samples in generative\n\nadversarial networks. In NeurIPS, pages 1498\u20131507, 2018.\n\n[42] Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jonathon Shlens, and Zbigniew Wojna. Rethinking\n\nthe inception architecture for computer vision. CoRR, abs/1512.00567, 2015.\n\n10\n\n\f", "award": [], "sourceid": 770, "authors": [{"given_name": "Mingming", "family_name": "Gong", "institution": "University of Melbourne"}, {"given_name": "Yanwu", "family_name": "Xu", "institution": "University of Pittsburgh"}, {"given_name": "Chunyuan", "family_name": "Li", "institution": "Microsoft Research"}, {"given_name": "Kun", "family_name": "Zhang", "institution": "CMU"}, {"given_name": "Kayhan", "family_name": "Batmanghelich", "institution": "University of Pittsburgh"}]}