LeRobot Hackathon

I took part in the worldwide LeRobot hackathon in 2025, where contestants had two days to train a robot on a novel task using vision-language-action models. I joined late, so I split my time between helping a friend get a robot to place bread in a toaster and building my own leader-follower arm.

The leader-follower arm was used to create a series of demonstration trajectories of the robot moving bread, which — along with the text prompt "place bread in the toaster" — was used to fine-tune a model with a new toasting behaviour. The leader arm was controlled by a person with a trigger to close the gripper, and the follower matched the leader's movement through space.

The robot successfully placed the bread by the end of the weekend, and I learned a lot about how sensitive these models are to their environment — small changes hurt performance substantially. So we created a fixed background, constant lighting, and tried to reduce vibration and knocks to the arms, and even used plastic bread instead of the real thing.

I also built my own set of SO-101 leader-follower arms from scratch. The video below is the pair I finally got working after an all-nighter, a reformatted hard drive, and plenty of debugging.