paint with ml09/16/2020
Paint with ML is an experimental ML art web application that lets you design your own Bob Ross style landscape painting. You build a semantic segmentation mask using a set of nine different semantic brushes, each one a different landscape painting element. Clicking on the "Run" button feeds this semantic map through an image-to-image translation network, generating a painting as output.
This project was initially a freelance consulting project for Spell, an end-to-end machine learning infrastructure platform startup. Some of the finishing touches, the application design especially, were made after I was hired onto Spell full time.
The deep learning model powering the application is a PyTorch GauGAN. The model was trained on the ADE20K scene parsing benchmark dataset, then fine-tuned on a custom dataset of 250 Bob Ross paintings. Model training and deployment (to a Kubernetes-based model server endpoint) was via the Spell SDK.