The Diffusion Image Scaler

Convert time into pixels with the slowest generative neural network scaler anywhere on the web. Got a small image? Try it out below!

Compare below how your browser and the diffusion image scaler perform.

Before

After

How It Works

The details added by the generative model (a denoising diffusion probablistic model) are random and constrained to what the neural network has previously seen in real images. The diffusion technique repeatedly adds random noise then partially removes it with a neural network, gradually leaving behind only the components of the noise that the network cannot distinguish from real details. Applying this process while gradually decreasing noise variance produces a semi-random walk through the space of noisy high resolution images, ending at one of the many images are compatible with the low resolution input.

During sampling the image is repeatedly analysed, making the process much slower than single pass methods such as generative adversarial networks. To deal with this uploads are placed into a queue which prioritises tasks with smaller images and fewer iterations.

When It Doesn't Work

This projects restoration models are limited to inverting the damage caused by downsampling paired with a variety of low-pass filters and mild jpeg compression under a variety of encoders. As images can be damaged in vastly more ways than this, failure to recover a good output is expected in the presence of other degradation mechanisms, such as:

• Sensor noise
• Other lossy image compression
• Multiple compression passes
• Unsharp filters or high frequency exaggeration
• Artistic filters
• Motion blur

The Real-ESRGAN project has done excellent work in tackling the general problem of restoration by producing synthetic training data from a broad range of degradation models.

References

A great introduction to diffusion models is this overview by Lilian Weng, along with this excellent blog by Yang Song who developed the score matching perspective. Also see:

Score-Based Generative Modeling Through Stochastic Differential Equations.

Denoising Diffusion Probabilistic Models.

Denoising Diffusion Implicit Models.

Image Super-Resolution via Iterative Refinement.

Score-Based Generative Modeling with Critically-Damped Langevin Diffusion.