The Robotics Platform Used in This Course
Throughout the course, we used a 1:8 scale autonomous racecar as our hands-on robotics platform. Designed for research, education, and experimentation in autonomous mobility, it combines powerful onboard computing, advanced sensor technology, and a realistic race-inspired design.
Because its setup closely reflects the architecture and behavior of real-world autonomous vehicles, the platform creates a strong connection to industry practice. This made it especially valuable for students, allowing us to explore perception, control, navigation, and autonomous driving in a practical, application-oriented way.
Applied Autonomous Driving at Cal Poly
William Engel · Spring 2026
The Racetrack
With the help of my students, we created a precise and advanced racetrack in the High Bay Lab, located in the Bonderson Engineering Projects Center.
Featuring intersections, curves, and a roundabout, the track became a realistic testing environment for autonomous driving concepts and a central part of our hands-on course experience.
Applied Autonomous Driving at Cal Poly
William Engel · Spring 2026
Teaching the Car to See
The first challenge was computer vision: teaching the robot to recognize its surroundings.
Students worked through the complete pipeline every engineer should know: data collection, image labeling, neural network training, evaluation, and deployment on the car.
Using a latest-generation YOLO-based segmentation architecture, they trained a high-performance, real-time model similar to those used in modern robotics, autonomous driving, and industrial vision systems.
This allowed the robot to detect key parts of the racetrack in real time and make driving decisions based on its visual understanding of the environment.
Applied Autonomous Driving at Cal Poly
William Engel · Spring 2026
Teaching the Car to Steer
For the midterm challenge, students combined perception and control to make the robot drive autonomously.
Using the YOLO segmentation model to detect the road and a PID controller to steer, the car was able to follow the lane smoothly and react to the track in real time.
Applied Autonomous Driving at Cal Poly
William Engel · Spring 2026
Teaching the Car to Follow Traffic Rules
After teaching the car how to steer and follow the road, we moved from lateral control to speed control.
Students learned how the robot could react to traffic rules and real-world driving situations, including speed limit signs, stop signs, traffic lights, and other vehicles. This allowed the car to make smarter driving decisions and behave more like an autonomous vehicle in a realistic traffic environment.
Applied Autonomous Driving at Cal Poly
William Engel · Spring 2026
Teaching the Car to Localize Itself
Students learned how IMU sensors and vehicle kinematics can be used to estimate motion without GPS. Using acceleration, angular velocity, magnetometer data, and control commands, they built an odometry algorithm from scratch to reconstruct the path driven by the racecar.
On the right, we see the car following the racetrack. On the left, we see how the algorithm reconstructs the path and shape of the track from motion data. The result is not perfect, as expected, because small sensor errors accumulate over time.
Applied Autonomous Driving at Cal Poly
William Engel · Spring 2026
Smart Parking with Ultrasonic Sensors
Students learned how ultrasonic sensors can be used for autonomous parking, where short-range distance measurements are essential.
In the top right, we see the racecar scanning for an open parking spot, positioning itself, and then starting the parking maneuver. In the top left, the side ultrasonic sensor measurements show how the distance changes when a parking space is blocked by another vehicle.
By combining these measurements with the odometry from the previous project, the car can create a map of the parking area and localize itself within it. In the bottom left, the surrounding ultrasonic measurements are shown as a PDC-style visualization and compared with the LiDAR scan data in blue, showing how well the sensor readings align with the real environment.
Applied Autonomous Driving at Cal Poly
William Engel · Spring 2026
Camera–LiDAR Fusion for Smarter Driving
Students explored how camera, LiDAR, and radar can be combined to understand objects, distance, and motion in front of the car.
On the right, we see the car driving. On the left, we see the laser scan data. At first glance, the purple dots in front of the front axle do not look very meaningful, but they measure the distance to the vehicle ahead. By fusing this distance data with the camera image, the car can detect the vehicle in front, measure how far away it is, and build a powerful adaptive cruise control system.
Applied Autonomous Driving at Cal Poly
William Engel · Spring 2026
Building Neural Networks from Scratch
Students then spent a longer phase learning the fundamentals of machine learning and neural networks. They built and trained a perceptron from scratch to understand the core ideas behind learning, weights, and optimization.
As a first applied deep learning project, they implemented a U-Net in TensorFlow and trained it from scratch on the dataset they had created earlier, gaining real hands-on experience in model building, training, and computer vision workflows.
Applied Autonomous Driving at Cal Poly
William Engel · Spring 2026
Advanced End-to-End Autonomous Driving
Students explored an advanced technique used in modern autonomous driving: end-to-end learning.
Instead of manually designing every control rule, they trained a PilotNet model to predict steering commands directly from camera images.
They completed the full pipeline from data gathering and dataset balancing to model building, training, evaluation, and deployment on the racecar, gaining hands-on experience with a real autonomous driving workflow.
Applied Autonomous Driving at Cal Poly
William Engel · Spring 2026
Master Thesis: Comparing Autonomous Driving Controllers
As part of a master thesis I advised, we compared two autonomous driving controllers: Stanley Control and Model Predictive Control.
In the main video, we see both controllers driving against each other on the racetrack, with the leading vehicle using MPC. The bottom left shows a 3D mapping of the track, while the bottom right visualizes the steering angles predicted by the Stanley controller.
The results showed that both control algorithms were able to drive the vehicle in a stable way, which was an exciting and encouraging outcome.
Applied Autonomous Driving at Cal Poly
William Engel · Spring 2026
From Path Prediction to Overtaking
This slide shows the MPC path prediction overlaid directly on the video frames, making it easy to see how the controller plans the vehicle’s future motion.
On the top, we see the results achieved by my master student. On the bottom, I demonstrate an overtaking maneuver as inspiration for future projects and possible extensions of this work.
Applied Autonomous Driving at Cal Poly
William Engel · Spring 2026
Future Opportunity: Scaling Hands-On Learning
The strong feedback from students and faculty showed the value of a hands-on course that connects AI, robotics, control, and autonomous driving with real hardware experience. I would be excited to continue developing this format together with Cal Poly and help make it a recurring part of the curriculum.
Recurring Course Offering
Continue Applied Autonomous Driving as a regular hands-on course built around perception, control, machine learning, and real-world testing.
Continue Applied Autonomous Driving as a regular hands-on course built around perception, control, machine learning, and real-world testing.
Shared Robotics Platform
Use the autonomous racecars across classes, senior projects, graduate research, and faculty collaborations in robotics, controls, AI, and autonomous systems.
Use the autonomous racecars across classes, senior projects, graduate research, and faculty collaborations in robotics, controls, AI, and autonomous systems.
More Time on Real Hardware
Additional vehicles would allow more student teams to test, debug, and iterate in parallel, giving them more direct experience with real autonomous driving hardware.
Additional vehicles would allow more student teams to test, debug, and iterate in parallel, giving them more direct experience with real autonomous driving hardware.
Applied Autonomous Driving at Cal Poly
William Engel · Spring 2026
Acknowledgements
I would like to sincerely thank the Cal Poly faculty, staff, and students who supported this course and helped make the hands-on autonomous driving experience possible.
Special thanks to Charlene Birdsong and Simon Xing for their mentorship and continued support, and to John Chen and Kim Shollenberger for supporting the opportunity to bring this course to Cal Poly.
I am also grateful to Siavash Farzan, Carlos Diaz Alvarenga, Eric Espinoza-Wade, Behnam Ghalamchi, and Charlie Refvem for their time, interest, and support, and for the opportunity to share and discuss the course and robotics platform with them.
Thank you.
Applied Autonomous Driving at Cal Poly
William Engel · Spring 2026