FontNN

https://github.com/shaunabanana/font-nn

Intro

Create fonts with the help of a neural network, by giving examples of what you want. Mix and match different styles to get things in between.

This project runs entirely in your browser, including the convolutional neural network (CNN) that processes the images. Powered by Candle.js.

Behind the scenes

FontNN is a project I did to try my hands at deep learning and bringing it to the web. You can upload or draw some characters, add them into the dataset, then watch the CNN extract your font style and apply it to the entire alphabet.

At the core of it lies a modified Variational Autoencoder (VAE). Typically, an autoencoder looks like this:

On the left is the encoder. An image is fed into it, and it outputs a bunch of numbers, effectively "describing" the image. On the right, the decoder gets the numbers, and tries to "reconstruct" the original image. The training goes on for all images, until the encoder is good enough at describing, and the decoder is good enough at reconstructing.

However, I did something different. Instead of picking one image at a time, I pick a pair of images with the same font style. I then feed one of them (A) to the encoder, thus getting a bunch of numbers for image A.

More numbers then gets attached to them, telling the decoder that it's supposed to generate the character B. Then, when it spits out an image, I compare the result with image B to see how close they are. Then training goes on, until all results are close enough.

This way, I kind of separated the style from the content. The encoder gets a font, and outputs its style. The decoder gets the style and the demanded character, then outputs that character in that style.

Finally, when given a set of examples, I can have the encoder look at each of them and generate a style. The average of all example styles are then sent to the decoder, along with each letter of the alphabet, to generate the complete font.

Next steps

I intended this project to be a one-shot thing. But another project that was born because of it,  Candle.js, is (hopefully) going to keep evolving into something useful.