Understanding Neural Style Transfer

1. Play. A series of posts exploring the visual and artistic side of convolutional neural networks.

Neural networks are fascinating and I'm really interested in how they can be leveraged for art or visual processing. In order to learn more about the topic I'm writing a series of articles exploring and trying to answer my own questions starting with neural style transfer.

I've started just by playing around. Generating these successful neural style transfer images took a lot of trial and error. The problem is a game of identifying good hyper-parameters and after playing around a bit I realized that unfortunately these are [content, style] specific and the output possibility space is huge.

Neural style transfer example image: Erin with round yellow brush strokes.

Neural style transfer example image: Erin with speckled paint.

Through the exploration it appears there is an affinity between certain styles and certain content. Not all combinations work, but when they do the results can look incredible.

Neural style transfer example image: Ilie portrait style 1

Neural style transfer example image: Ilie portrait style 2

Next is a progression showing the effect of scaling the style has on the output. Kandinsky's style is shining through and different details surface at various scale frequencies.

Neural style transfer example image: Kandinsky like at a large scale

Neural style transfer example image: Kandinsky like at a medium scale

Neural style transfer example image: Kandinsky like at a small scale

Neural style transfer example image: Kandinsky like at an extra small scale

Neural style transfer example image: Craig in barn

Finally two more Kandinsky transfers at a fairly large style scale.

Neural style transfer example image: Old Ford with Kandinsky

Neural style transfer example image: Jeep with Kandinsky

2. Watch it learn.