paint with ml
09/16/2020
Paint with ML was an ML art web application that lets you build your own Bob Ross style landscape painting. You build a semantic segmentation mask using a set of nine different semantic brushes, each one a different landscape painting element. Clicking on the "Run" button feeds this semantic map through an image-to-image translation network, generating a painting as output.
This was a freelance consulting project for Spell, an MLOps platform startup. Some of the finishing touches were made after I was hired by Spell full time.
The deep learning model powering the application is a PyTorch GauGAN. The model was trained on the ADE20K scene parsing benchmark dataset, then fine-tuned on a dataset of 250 Bob Ross paintings that I hand-labeled. Model training and deployment (to a Kubernetes-based model server endpoint) was via the Spell SDK. Here are some examples of things people built with it:
The code is available publicly on GitHub.