Skip to content

Pytorch implementatiions of the Homeworks in course CS294-158

Notifications You must be signed in to change notification settings

CaptainE/Deep-Unsupervised-Learning

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

23 Commits
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Deep-Unsupervised-Learning

Pytorch implementatiions of the Homeworks in course CS294-158

Here you can find PyTorch implementations of a Masked Autoencoder for Distribution Estimation, PixelCNN, RealNVP and other architectures that are used in course.

Homework 1

For the first homework there is a PyTorch implementation of PixelCNN-MADE

That is a auto-regressive model, which mixes ideas from the two papers Pixel Recurrent Neural Networks (2016) and MADE: Masked Autoencoder for Distribution Estimation (2015) to produce colored 28x28 MNIST digits. Pixel intensities have been quantized to 2 bits (i.e. four intensities for each color channel). The dataset can be downloaded from here.

Training data

I have used 60,000 quantized 28x28 images from the colored MNIST dataset. Here's a few samples from the training set:

TrainingSet

Examples

After 50 epochs of training, a network consisting of 12 residual blocks - see Deep Residual Learning for Image Recognition (2015) - followed by a 3-layer MADE generates samples like the following:

Example

You are very welcome to extend the code however you like. If you produce anything cool, be sure to let me know!

Homework 2

For the second homework we implement RealNVP coupling layers for modelling flows

Training data

Here I have used 5000 datapoints sampled from this distribution:

TrainingSet

Example

After 500 epochs we can sample the following face

Example

with this fancy latent space

Latent space face

and density plot density plot

We also implement RealNVP(https://arxiv.org/abs/1605.08803) to achive these results:

realnvp

Homework 3

We implement a VAE with a gated shortcur connection(https://arxiv.org/pdf/1612.08083) and train it on the SVHN dataset.

The final results look like this: done_training

Homework 4

In the 4th homework we implement the Wasserstein GAN(https://arxiv.org/abs/1704.00028) and draw inspiration from the architecture used in SN-GAN(https://arxiv.org/abs/1802.05957)

This we for 80 K iteration and the training curves and mean inception score look like the following: Training curve Inception_Score

And the resulting samples:

Samples

Releases

No releases published

Packages

No packages published