Generative AI in visual context

Tushar Sandhan(Indian) Professor (Indian Institute of Technology Kanpur)
Speaker

Tushar Sandhan(Indian)
| Professor (Indian Institute of Technology Kanpur)

Abstract

Generative adversarial networks (GAN) are generative models that require large amounts of training data to ensure a stable learning trajectory during the training phase. In the absence of sufficient data, GAN suffers from unstable training dynamics that adversely affect the quality of generated data. This behavior is attributed to the adversarial learning process and the classifier-like functioning of the discriminator. In data-deficient cases, the adversarial learning procedure leads to the discriminator memorizing the data instead of generalizing. Due to their wide applicability in several generative tasks, improving the GAN performance in the limited data paradigm will further advance their usage in data-scarce fields. Therefore to circumvent this issue, we propose a loss-regularized GAN, which improves the performance by forcing a strong regularization on the discriminator.
The problem of recovering missing data has garnered considerable attention due to its significance and challenges in recent times. In particular, the ability to recover clear face images from occluded face images has found applications in various domains. One prominent approach in this context is the utilization of autoencoders within the framework of Generative Adversarial Networks (GAN), such as the Context Encoder (CE). The CE is an unsupervised algorithm that leverages an autoencoder as its generator. It is designed to inpaint missing areas in an image based on the information present in the surrounding areas. By learning a compressed representation of the input image, the autoencoder can generate plausible and visually coherent predictions for the missing regions. We found that the initial values of the pixels in the missing area have a significant effect on the quality of the generated images. Careful selection of these initial values proved crucial in achieving accurate and visually appealing inpainted results.

List