Simon the Robot: Creating Social Engagement

Using a Saliency Attention Mechanism to Synthesize Object Preference Based on a Visual Stimulant
Robotics and Intelligent Machines Lab, Georgia Tech | Aug 2010 – May 2011

The Challenge

Simon was a social robot at Georgia Tech's Robotics and Intelligent Machines Lab. My undergraduate research project was to program him as a greeter at the lab entrance, creating engaging behaviors that would make him approachable and interactive with visitors.

What I Built

I developed Simon's social interaction behaviors using his existing Saliency Attention Mechanism (S.A.M.). I programmed him to autonomously respond to visual stimuli—specifically colors and people—to create engaging, personality-driven interactions.

I built a graphical interface for color preference input and designed behavioral sequences that made Simon feel alive and expressive.

How It Worked

When Simon identified a salient object (using S.A.M.'s visual processing):

• He would fixate on it for 4 seconds

• Change his ear color to match the object's color

This turned abstract perception into a relatable personality, which gave Simon preferences and made AI tangible for lab visitors.

Impact

My first experience programming a robot. I later featured this work in my 2021 X-STEM presentation "Real Life Robotics" as an example of making AI accessible and understandable.

Team: Advised by Dr. Andrea Thomaz, mentored by Maya Cakmak