Wat AI

Reinforcement learning and a car made from pizza boxes.

At around 9 p.m., in the cold, we went outside E7 and grabbed pizza boxes and other cardboard from the recycling bins.

We slapped together some 90-degree angles, put on some tape, made holes for the motors and screwed them straight into the cardboard. We stuck in the Raspberry Pi, added a camera and wheels, and got it to work. We called it Carboard.

Carboard, with a Raspberry Pi, camera, motor driver and wheels on a cardboard chassis — *Carboard. Motors screwed into cardboard, with the Raspberry Pi and camera on top.*

This was Wat AI, the student design team I joined in my first term at Waterloo. We wanted a miniature race car that learned to drive using reinforcement learning. Train it in simulation, where crashing is free, then transfer what it learned to a real car.

The cardboard got us something physical to test without waiting for a finished chassis. Alongside the hardware, I worked on a PyBullet environment and a separate 2D racing simulator built with Gymnasium and Pymunk.

In the 2D simulator, PPO and SAC policies control steering and throttle. They can observe simulated LiDAR distances or a perspective-projected occupancy grid processed by a CNN. An interactive track editor lets us change the layouts we train on. I also put together a CartPole example for teammates getting started with PyBullet.

The 2D driving and track-editing demos.

We later built a longer chassis with steering and a lower center of mass.

Steering and movement tests on the long-chassis prototype.

We got the agent driving in simulation and the hardware moving, but didn't test the trained policy on the physical car before the project ended. Our team presented the work at CUCAI, the Canadian Undergraduate Conference on AI. I really enjoyed getting to build a robot with my peers.