Here we can see the robot turning in response to the controller turning. The controller sends its estimated angular position from the gyro to the robot over the radio, and then the robot reacts and adjusts its angular position using a PID controller.
Posted on by Oren A

Don’t we love it when a project wraps up? This is one of those times where a project has been in the works forever and finally gets finished. Frank the differential drive robot (the name I’ve given him) is finally to a point where I can do just about anything! For this post I’ll cover what I did most recently with a custom remote controller and real time operating system (RTOS). However, I plan on adding some functionalities for position estimation as well as path planning using something like A* or rapidly-exploring random trees (RRT). Everything I’ve finished has been uploaded to my GitHub and I plan to add some user friendly features as I expand the python library I’m designing.

Here is a full render of Frank. The bottom layer houses the battery, motors, and some voltage regulators. The second layer holds the Raspberry Pi, BNO055 IMU and gyro, and a servo hat for the Pi. The top layer is to attach the camera to.
Here is a full render of Frank. The bottom layer houses the battery, motors, and some voltage regulators. The second layer holds the Raspberry Pi, BNO055 IMU and gyro, and a servo hat for the Pi. The top layer is to attach the camera.

Getting the Radios Working

The first thing I did was get the two nrf24l01 RF radios communicating. This took awhile because I had to rewrite part of the driver for the TM4C123GH6PM processor on the Tiva Launchpad I was using. It was not ready to be used with the i2c port that I wanted to use, so I had to modify the driver for this. I then got the module working on the Raspberry Pi on Frank. This was using the RF24 library I found. This took some time as I had to build python bindings for their C++ library. Their provided instructions worked quite well though, so I definitely recommend this library for others in the future.

Here we can see two Arduino nanos communicating using the nrf24l01 modules.
Here we can see two Arduino nanos communicating using the nrf24l01 modules.

After I got that working I got the other modules working with their respective device. That included a servo hat and BNO055 IMU/gyro for the raspberry PI, and an LED controller and IMU/gyro for the Tiva. Luckily for me, I only had to write the driver for one of these, the LED controller on the supplemental Tiva board.

Me moving the joystick to cause the LEDs to move. My driver allows me to dim each color and change the clock update rate.
Me moving the joystick to cause the LEDs to move. My driver allows me to dim each color and change the clock update rate.

The Real Time Operating System and Remote Controller

After I got all of the hardware working, I starting writing code to combine them in different ways. This is where the RTOS concepts came in. For my Microprocessor Applications 2 (EEL 4745C) class I coded a custom RTOS in C on the Tiva Launchpad. The RTOS handled the generation of periodic, aperiodic, or background threads and had functions for sleeping, blocking/semaphores, and thread safe queues. For instance, I made two background threads, one for receiving and one for transmitting data from the Radio. I also used a thread safe queue to manage data access to the joystick and some of the incoming radio data.

The Tiva Launchpad connected to the course designed expansion PCB. The touch function for the LCD on the board was a pain, guess that's what happens when you accidently get chinese knockoffs.
The Tiva Launchpad connected to the course designed expansion PCB. The touch function for the LCD on the board was a pain, guess that’s what happens when you accidently get Chinese knockoffs.

I will not go into much more detail on the implementation as there was a lot of code specific to the processor I was using. However, I will say that I got an intimate understanding of stack manipulation as a result of the project. This was the source of most of my issues, so if you happen to be working on your own RTOS, definitely check the stack during debugging.

This is the Tiva expansion board with the radio hat attached. There is a joystick and buttons and their outputs are sent over the radio.
This is the Tiva expansion board with the radio hat attached. There is a joystick and buttons and their outputs are sent over the radio.

The Raspberry Pi Robot (Frank)

Frank only took a few days to design and print, and with shipping only taking about a week, this part of the project moved very quickly. Below I have included a wiring diagram and the some build pictures. I can provide a BOM or CAD if requested, but Frank cost approximately $300.

This is the second version of the wiring diagram for anyone who wants to try to replicate the electronics.
This is the second version of the wiring diagram for anyone who wants to try to replicate the electronics.
Here we see some old Frank assembly pictures. The Pi is mounted to a 3D printed baseplate with standoff to support the above support structure. The bottom potion is mounted with three standoffs with threaded inserts. The continuous turn servos attach to this section too.
Here we see some old Frank assembly pictures. The Pi is mounted to a 3D printed baseplate with standoffs to support the above support structure. The bottom portion is mounted with three standoffs that have threaded inserts. The continuous turn servos attach to this section too.
This is Frank's battery. It was made using the same techniques as the longboard battery but using a smaller BMS PCB. It's a little sketchy, but it was made with what I had. I have added a blue heat shrunk shielding around it for safety.
This is Frank’s battery. It was made using the same techniques as the longboard battery but using a smaller BMS PCB. It’s a little sketchy, but it was made with what I had. I have added a blue heat shrunk shielding around it for safety.
Here is Frank right after I built him. In the newer pictures he is now yellow and blue. This is because I had to reprint some parts due to the original print breaking and not having enough standoffs.
Here is Frank right after I built him. In the newer pictures he is now yellow and blue. This is because I had to reprint some parts due to the original print breaking and not having enough standoffs.

Programming Frank’s Motion Controls and Facial Tracking

The programming for Frank was a culmination of projects from class as well as concepts I learned on my own time. Fortunately, for the raspberry pi there was far less custom code. I was able to use the existing threading libraries to do everything. I had threads for each peripheral on Frank. Because the data was not used for anything high performance this worked well, but in the future I’ll probably have to synchronize these systems in some way for accurate control. This worked pretty well for motion controls from the Tiva board to Frank.

Here we can see the robot turning in response to the controller turning. The controller sends its estimated angular position from the gyro to the robot over the radio, and then the robot reacts and adjusts its angular position using a PID controller.
Here we can see the robot turning in response to the controller turning. The controller sends its estimated angular position from the gyro to the robot over the radio, and then the robot reacts and adjusts its angular position using a PID controller.

Frank can also do facial tracking! This was easy to implement but was a bit of trouble due to some of my python libraries being old or missing (I’m looking at you 2019 NumPy). After I got that fixed I was able to get it working! To track faces, I calculate the average face position in the image. I then offset this value by half the width of the image. This new value can be considered as the error that the robot has to correct for. It then figures out what to do using a PID controller. Due to the camera latency, it has some difficultly tracking, but with some small changes, specifically by adding angle estimation or some type of feedforward term, I’m sure I could improve performance.

This is a video of Frank tracking my face! I had to set up the lighting just right, but it did it quite well under these conditions.
This is a video of Frank tracking my face! I had to set up the lighting just right, but it did it quite well under these conditions.

My Frank Project Summary

I have learned so much throughout this project! I already felt comfortable with Python, but this has made me feel far more confident with C, so I’m really happy I did all of this. Additionally, combining my knowledge of electronics, mechanics, and programming to make a robot has been a really rewarding experience. I am definitely going to have to dive deeper into the programming and libraries I have been developing to make something more reusable. Looking forward to future development!

Leave a Reply