Hand-Tracking Robotic Arm: Live Demo

A Python + MediaPipe pipeline reads your webcam, extracts hand landmarks, and maps them to six joint angles on an Arduino-driven Braccio arm, right hand steers the arm, left hand opens and closes the gripper.

Looking for local service…View source ↗

Run it yourself

This demo drives real hardware, so it runs on your machine rather than a server. It needs a webcam; the Arduino and Braccio arm are optional, and without them the vision pipeline still runs and prints the joint angles it would send.

Clone the repository

git clone https://github.com/s3raphsembrace/mediapipe-braccio-control.git
cd mediapipe-braccio-control

2
Install dependencies
```
pip install -r requirements.txt
```
3
Start the service
```
python app.py
```

Once it's listening on localhost:8000, hit Re-check above and the live view will load here.

How it works

MediaPipe extracts 21 hand landmarks per frame from the webcam stream.
Right-hand pose maps to six Braccio joint angles; left-hand pinch drives the gripper.
Only changed values are transmitted over serial, which cuts jitter from frame-to-frame noise.
Arduino firmware validates every incoming angle against a safety table and steps in 3° increments to avoid mechanical strain.
Both hands open flat triggers a reset to the calibrated safe position.