Nader Ahmed

Robotics engineer, currently pursuing MS at Northwestern University

In this project, robotics meets art! A 7 DOF Franka Panda arm, “BotROS”, was programmed to autonomously create pointilist-style artwork.

BotROS is provided with a canvas, paint brushes, and a palette. It will pick up a brush, dip it in a color, and will begin painting some dots! It will return for more paint every 8 dots. Once the first color is complete, it will return its brush and pick up a new one before initiating a second round of color.

Video Demo

Project Components

I. Motion Planning

A custom MoveIt wrapper frankastein was developed in Python for interfacing with the Franka. This API facilitated the planning and executing of robot trajectories by making requests to MoveIt’s GetPositionIK and GetCartesianPath services.

API Function: Input
plan_path_to_position goal position (x,y,z)
plan_path_to_orientation goal quaternion (x,y,z,w)
plan_path_to_position_orientation goal position(x,y,z) + quaternion (x,y,z,w)
plan_path_cartesian list of position (x,y,z) waypoints

II. Computer Vision

April tags are used to locate the brushes and palette. The listener node establishes the transformation between the camera and robot and publishes the paint brush and palette locations. The colordetection node then color thresholds the image to determine the specific paint dip locations with respect to the robot frame.

The PictureTaking package creates a service take_picture which performs the following operations:

  • Snaps a picture from an Intel Realsense (images can also be supplied)
  • Applies morphological operations (dilation + erosion)
  • Applies a Canny edge detector
  • Discretizes the edge map into points
  • Maps the points to the task space of the Franka end effector
"Swarthmore S"
Swarthmore S points.

III. Collage

Our team decided to make a collage representing all of our undergraduate institutions:
College logos: Northwestern, Swarthmore, Notre Dame, Georgia Tech, Tennessee, and Maryland

Team: Nader Ahmed, Demiana Barsoum, Shail Dalal, Fiona Neylon, Courtney Smith

Check out the project → GitHub