Gesture-Controlled Robot Arm

Two students in the AI and Data Workshop trained a model that recognises five hand gestures and connected it to a servo-driven robot arm. An open hand raises the arm, a fist closes the gripper, and a pointing finger turns it left or right.

The process

For each gesture the team built an anonymous dataset of 200 images of classmates’ hands, shot so that no faces were in frame. The first model only worked on a white table; once data with different backgrounds was added, accuracy rose from 71 to 93 per cent. The parts of the robot arm were printed on the 3D printer.

The project showed students in concrete terms how data diversity affects a model’s success. The team also reviewed the images the model misclassified and prepared a confusion matrix showing which gestures were being mixed up, then presented the results to the class. The work has become one of the examples used in next term’s lessons.