TensorFlow

TensorFlow Dev Summit 2017 cont...

in

Matt and I had a long day listening to some of the latest breakthroughs in deep learning, specifically those related to TensorFlow.  Some standouts included a Stanford student that had created a neural net to detect skin cancer. Also liked Doug Eck's talk about artificial creativity. Jeff Dean had a cool keynote, and got to learn about TensorBoard from Dandelion Mane.  One of my favorite parts of the summit was getting shout outs and later talking to both Jeff Dean and Doug Eck.  The shoutouts to cloudpainter during Jeff's Keynote and Eck Session and lots of pics can be seen below. This is mostly for my memories.

Hunter's Portrait

in ,

Inspired by our trip to the National Portrait Gallery, we started thinking to ourselves, what's so impressive about making our robot's paint like a famous artist.  Sure they are inspirational and a lot can be learned from them, but when you think about it, people are more interested in the art of their loved ones.  

So this morning, Hunter and I decided to do quick portraits of each other and then run the portraits through deep neural nets to see how well they applied to a photo we took of each other. As soon as we started, Corinne joined in so here is obligatory photo of her helping out.

Also in the above photo you can see my abstract portrait in progress.

Below you can see the finished paintings and how they were applied to the photos we took. If you have been following this blog recently, you will know that the images along the top are the source images from which style is taken and applied to the photos on the left. This is all being done via Style Transfer and TensorFlow. Also I should note that the painting on left is mine, while Hunter's is on right. 

Most interesting thing about all this is that the creative agent remains Hunter and I, but still something is going on here. For example even though we were the creative agents, we drew some of our stylistic inspiration from other artist's paintings that we saw at the National Portrait Gallery yesterday. Couldn't a robot do something similar?

More work to be done.

Channeling Picasso with Style Transfer and Google's TensorFlow

in ,

We are always jumping back and forth between hardware and software upgrades to our painting robot. This week it's the software. Pleased to report that we now have our own implemention of Dumoulin, Shlens, and Kudlar's Style Transfer. This of course is the Deep Learning algorithm that allows you to recreate content in the style of a source painting. 

The first image that we successfully created was made by transferring the style of Picasso's Guernica into a portrait of me in my studio.  

So here are the two images we started with. 

And the following is the image that the neural networks came up with.

I was able to get this neural net working thanks in large part to the step-by-step tutorial in this amazing blog post by LO at http://www.chioka.in/tensorflow-implementation-neural-algorithm-of-artistic-style. Cool thing about the Deep Learning community, is that I found half a dozen good tutorials. So if this one doesn't work out for you, just search out another.

Even cooler though, is that you don't even need to set up your own implementation. If you want to do your own Style Transfers, all you have to do is head on over to the Deep Dream Generator at deepdreamgenerator.com. On this site you can upload pictures and have their implementation generate your own custom Style Transfers.  There is even a way to upload your own source images and play with the settings.  

Below is a grid of images I created on the Deep Dream Generator site using the same content and source image that I used in my own implementation.  In them, I played around with the Style Scale and Style Weight settings. Top row has Scale set to 1.6, while second row is 1, and third is 0.4.  First column has the Weight set to 1, while second is at 5 and third is at 10.

So while I suggest you go through the pains of setting up your own implementation of Style Transfer, you don't even have to.  Deep Dream Generator lets you perform 10 style transfers an hour.

For us on the other hand, we need our own generator as this technology will be closely tied into all robot paintings going forward.

 

 

 

Brushstroke Maps for Three Famous Paintings

in ,

When you Google "Famous Artwork" a list of paintings is revealed, and at the top of that list is da Vinci, Van Gogh, and Munch. Here is a picture of the top ten actually...

Now that we have a stroke map of the Mona Lisa and The Scream, we decided to round up the top three by creating a mapping of The Starry Night.  Interestingly, The Starry Night is probably one of the best examples of the importance of brushstrokes in a painting.  There is nothing but flow in it.  And a major part of the composition is the movement made by the direction of the strokes. 

If we could somehow capture how Van Gogh used his strokes, well thats impossible, but if we could at least learn something from them. Well we will never know until we try.

As of 7:00 PM January 8, 2017, cloudpainter has just barely begun to explore the strokes of Van Gogh's The Starry Night.  We realize these first strokes are rudimentary, but its just laying down a background. Over the next several days we will attempt to copy as many of the strokes with as much detail as possible. These will be stored in an Elasticsearch database and shared for anyone to use in attempts to deconstruct Van Gogh's brushstroke.

Work Continues Mapping the Brushstrokes of Famous Masterpieces

in ,

Once I created a brushstroke map of Edvard Munch's The Scream, I thought it would be cool to have brush stroke mappings for more iconic artworks. So I googled "famous paintings" and was presented with a rather long list. Interestingly The Scream was in the top three along with da Vinci's Mona Lisa and Van Gogh's Starry Night. Well, why not do the top three.  So work has begun an creating a stroke map for The Mona Lisa.  In the following image, the AI has taken care of laying down an underpainting, or what would have been called a cartoon in da Vinci's time.

 

I am now going into it by hand and finger-swiping my best guess as to how da Vinci would have applied his brushstrokes.  Will post the final results as well as provide access to the Elasticsearch database with all the strokes as soon as it is finished. My hope is that the creation of the brushstroke mappings can be used to better understand these artists, and how artists create art in general.

Full Visibility's Machine Learning Sponsorship

in ,

Wanted to take a moment to publicly thank cloudpainter's most recent sponsor, Full Visibility. 

Full Visibility is a Washington D.C. based software consulting boutique that I have been lucky enough to become closely associated with. Their sponsorship arose from a conversation I had with one of their partners. Was telling him how I finally thought that Machine Learning, which has long been an annoying buzzword, was finally showing evidence of being mature. Next thing I knew Full Visibility bought a pair of mini-supercomputers for the partner and I to experiment with. One of the two boxes can be seen in the picture of my home based lab below. It's the box with the cool white skull on it. While nothing too fancy, it has about 2,500 more cores than any other machine I have ever been fortunate enough to work with. The fact that private individuals such as myself can now run ML labs in their own homes, might be the biggest indicator that a massive change is on the horizon.

Full Visibility joins the growing list of cloudpainter sponsors which now includes Google, 7Bot, RobotArt.org, 50+ Kickstarter Backers, and hundreds of painting patrons. I am always grateful for any help with this project that I can get from industry and individuals. All these fancy machines are expensive, and I couldn't do it without your help.

Pindar Van Arman