Picassnake: The Painting Robot

Picassnake is our custom-made robot that paints pictures from music.

Picassnake consists of an arm and a head, embedded in a plush snake doll. The robot is connected to a laptop for control and music processing, which can be fed through a microphone or an MP3 file. To open the media source, an operator can use the graphical user interface or place a text QR code in front of a webcam. Once the media source is opened, Picassnake generates unique strokes based on the music and translates the strokes to physical movement to paint them on canvas.

To generate unique strokes, Picassnake does frequency analysis of the music. After taking some time to think, Picassnake decides the color of a stroke based on the loudness of the highest frequency range (14k-22k). The start point of a stroke is the end point of last stroke. Picassnake decides the end point of the stroke by calculating a distance and angle from the start point based on spectrum of the music. The code for this calculation is presented below. In addition to strokes, Picassnake also detects beats of the music to move its head up and down. The beat detection algorithm is retrieved from http://archive.gamedev.net/archive/reference/programming/features/beatdetection/.

Picassnake plans physical motions using precalibrated servo positions. There are 5 positions specifying the locations of three color paints, water to clean the brush, and paper towels to dry, and 13 positions on canvas (3x3 grid points which are 9 positions, and the center positions of 4 equally sized sub-rectangles of the canvas). To cover the entire canvas with the limited number of positions, we use triangular interpolation (pick three closet points and interpolate based on the distance from them to the destination).

With mechanical characteristics of the robot and the signature of the music, Picassnake draws an interesting and unique abstract painting of the music.

A map to calibrate the 13 canvas points (every intersection on the map is a point).

Robot arm without plush doll.

// The program is written in C#. // // spectrum range is 0, 60, 250, 500, 2k, 4k, 6k, 14k, 22k // spectrum value is ranged from 0.0 to 1.0, after calculation // spectrum window is 1024 // minimum decibel for spectrum is -90dB. // // _specAbsDiff: absolute value of diffrence between previous spectrum and current spectrum // _specDifference: diffrence between previous spectrum and current spectrum // // _lastAngle at the beginning is -90 degree // _lastPoint at the beginning is the center of canvas double diffAbsSum = 0.0; double diffSum = 0.0; double diffSumHighFreq = 0.0; double diffSumLowFreq = 0.0; for( int i = 0; i < _specAbsDiff.Length; ++i ) { diffAbsSum += _specAbsDiff<i>; diffSum += _specDifference<i>; if( i > _specAbsDiff.Length / 2 ) { diffSumHighFreq += _specAbsDiff<i>; } else { diffSumLowFreq += _specAbsDiff<i>; } } if( diffAbsSum != 0.0 && diffSum != 0.0 ) { _lastAngle += (Math.PI * 2.0) * (diffAbsSum / (diffSumLowFreq - diffSumHighFreq)); double distance = (diffSumHighFreq / diffAbsSum) * Math.Sqrt( 2.0 ); double pointX = distance * Math.Cos( _lastAngle ) + distance * -Math.Sin( _lastAngle ) + _lastPoint.X; double pointY = distance * Math.Sin( _lastAngle ) + distance * Math.Cos( _lastAngle ) + _lastPoint.Y; Point3D currPoint = BoundDrawingPoint( new Point3D( pointX, pointY, 0.0 ) ); LineEventArgs evtLine = new LineEventArgs( _nStartSamples ); int index = (int)(Math.Abs( _specDifference[_specDifference.Length - 1] ) * 10000.0 + 0.5); evtLine.Color = COLORS[index % COLORS.Length]; evtLine.Thickness = THICKNESS_BASE * Math.Pow( diffSumLowFreq / diffAbsSum, 2.0 ); evtLine.SetPoints( new Point( _lastPoint.X, _lastPoint.Y ), new Point( currPoint.X, currPoint.Y ) ); lock ( _lines ) { _lines.Add( evtLine ); } _lastPoint = currPoint; } _nStartSamples = _nProcessedSamples;

Video

Following videos are for RobotArt competition in 2016 (robotart.org)

Project Publications

• 
• 
• 
•