A ROBOTO
Robotic Painter developed by Data Science Lab
The human race is filled with passion. And medicine, law, business, engineering, these are noble pursuits and necessary to sustain life. But poetry, beauty, romance, love, these are what we stay alive for.
- John Keating
Our Goal
To enable any person to create a beautiful and emotional painting, with the help of AI and a robot.
Toward Robotart Competition
One of the major milestones of our team is the participation in Robotart Competition 2018. It was a rollercoaster of a journey!
Our goal was to develop a beautiful real painting by using a generating input image with specific style. When we finally got our main tool – the robot arm – we only had six weeks left until the deadline of the competition. Initially, we thought of painting a high-resolution painting. However, such idea was not practical, because we would need to use a small brush on a huge canvas, spending few days only to draw the initial sketch…
We all know that constraints develop creative thinking… therefore, we decided to change our approach. Our team implemented a technique that is similar to Chinese painting style. Such style uses only a small number of strokes, but it nevertheless able to portray many things. After many trials we succeeded in developing an algorithm that was able to generate simple and natural, but powerful stroke lines.
If you like our project, please vote:) →VOTE
Generate Natural Stroke
We used a real photo as a reference in order to generate natural and smooth strokes. By far, this was the most challenging part of the project. We knew that some of the common approaches to robotic art include 1. Painting with dots 2. Using straight lines (just like in printers) and 3. Using very small strokes on a big canvas.
Our goal was to capacitate the robot to draw natural and long (or short when necessary) strokes, just like a real human being. Therefore, we developed an original technique called Deep Stroke Line (DSL), which is able to automatically generate a trace of a natural human-like stroke line. Please take a peak to our video, where you can see the process of how our technology generates natural strokes.
Simple Line but Powerful
We combine 1)local brightness information, 2)edge feature and 3)global spatial flow direction to generate natural stroke line that well express sprits in original photo.
It can generate many patterns form one photo, we use these stroke lines to painting layer by layer.
The disadvantage of this approach is hard to generate stoke when there are lack of change in brightness. But it still can work well in many case in our experiments.Generate Image
with Style
Style Transfer (e.g. Prisma, deepart.io) is a popular technique in Deep Learning that can transfer painter’s style onto a photo and generate a completely new painting. One of the most famous state-of-art technologies in this field is called Deep Image Analogy. We enhanced Deep Image Analogy in order to make a high-resolution image. You can see an example of this in the way that we transferred Mucha’s painting style onto a photo.
Our future goal is to enable the robot to paint a physical painting exactly as it appears in the input image.
Base Image
Target Style (Mucha)
Women smelling flowers by Alfons Mucha, 1899
Transfer Mucha's Style to Base Image
About Us
We are members of technical community called Data Science Lab (DSL) from Japan. DSL focus is creating new things in the latest technology area(AI, Cloud, VR etc). We love art. Yet, not everybody is good at painting. Hence, the goal of our project is to enable people to paint beautiful real paintings with a help of a robot. Luckily, my friend from Yamaha Motor Company helped us to borrow a great robotic arm entirely for free. We started painting this March and continuously progressed, never forgetting our underlying motivation - to create something aesthetic and beautiful.
Team
Software engineer:
Yiru Lai/頼 伊汝(Captain), Yoshiki Yatsu/谷津 芳樹, Taiki Shimba/榛葉 大樹, Naoto Kimura/木村 直登, Shinichiro Baba/馬場 慎一郎
Mechanical engineer:
Yuji Yamaguchi/山口 勇二, Kaito Matsubara/松原 海人
Color mixing specialist:
Serina Miyake
https://serinamiyake.jimdo.com
Office:
Kazuko Sato/佐藤 和子, Tetsuya Kanai/金井 哲也
Document check:
Marija Uzkuraityte
Special thanks:
Yamaha Motor Co., Ltd.
- Media
# English
# Chinese
# Spanish
# Russian
behind the scenes
Take a look and enjoy!
March 1st, 2018 - Setup the robotic arm
Start career as robotic painter
March 10th, 2018 - First painting test
With unexpected rotation:)
- black line painting
- all lines are defined manuallyMarch 20th, 2018 - 3rd painting test
Ancient Egypt style
- she got color!
- small paint but with layers and outline
March 29th, 2018 - 5th painting test
Looks good!
- bigger
- use good transparent watercolor
- horizontal lines and outline
April 2nd, 2018 - 6th painting test
Too much layers will kill you
- new one with many layers
- fyi, this is completed version
April 4th, 2018 - 7th painting test
Never give up!
April 10th, 2018 - 11th painting test
Only 3 days left...
- finally we implemented paint fully with nice strokes
April 14th, 2018(JST) - 15th painting test
Today is the deadline for the contest
- tired
Fin
Progress of 45 days
- still in the middle of a journey
© 2018 Data Science Lab