handshake: how to update the software on the devices

Summary

Using esptool.py we can load new firmware onto the two devices that are used as part of the handshake project. These devices use the ESP32-S3 microcontroller.

The problem

I’m developing an assistive technology project called handshake. This uses a programmable smart watch and a receiver unit. Both of these devices use an ESP32-S3 as the main processor. The system is now ready for testing. I’ll inevitably need to improve the code running on the devices after feedback from the testers. How do I enable the good folk who are testing the system to load my “new and improved” code onto the devices?

I use VS Code with the PlatformIO plugin to write the code that then produces the firmware that is loaded onto the devices. The firmware files are loaded onto the devices through a USB cable ‘automagically’ by the PlatformIO plugin. It took me long enough to figure out the programming toolchain. I don’t want to subject anybody else to this learning step!

The solution

Looking at the tiny printout on my VS Code screen, I see that the firmware is uploaded to the ESP32-S3 using the esptool.py program. A little light background reading and I found that this tool can be installed separately from VS Code and can be run from the command line. This also allows the potential to write a script to automate loading new firmware onto target devices.

I tested esptool from the command line on both Linux and Windows 11 and found it worked to load new firmware onto the receiver unit! Hopefully, this method of loading firmware will not be too off-putting to the testers.

Setting up the tools

To install and run esptool I followed these intructions: https://www.flamingo-tech.nl/2021/03/21/installing-and-using-esptools/

In case this link disappears, here is the summary of the guide on this site:

We need to install the Python programming language to enable esptools.py to run. Then we need to set up a ‘virtual environment’ to install esptools into.

A ‘virtual environment’ means that whatever you install using Python does not affect anything outside of the virtual environment. Not so many versions of Python ago, we didn’t need to run a virtual environment to install new software in Python. Using virtual environments is a safer way of doing things even if it is more fiddly.

To create a virtual environment called ‘python’ type:

python -m venv python

I’ve named the virtual environment ‘python’ for no particular reason. You can call the virtual environment whatever you like.

We need to activate the virtual environment before we can use it. Activate the virtual environment using:

Windows

cd python/Scripts
activate.bat

Linux

cd python\bin
./activate

Install esptools inside of your virtual environment using this command:

pip3 install esptool

We now have all the tools that we need to upload new firmware to the ESP32-S3 devices!

Loading new firmware to the ESP32-S3 devices

Connect the ESP32-S3 device to your PC using a USB connector. In the Scripts (or bin directory on Linux) try:

python esptool.py flash_id

This displays information on the serial port that the ESP32-S3 is connected to through the USB cable as well as information on the device, such as the MAC address. A right rivetting read. The important bit is which serial port we are connected to. On Windows this will be something like ‘COM4’. The number after ‘COM’ will change depending on the machine and what else has been plugged in prior to attaching the ESP32-S3. On Linux, the serial port will be something like ‘/dev/ttyACM0’ or /dev/ttyUSB0’. Again, the number in the port name may be different.

There are three firmware files that need to be loaded: bootloader.bin, partitions.bin and firmware.bin. Each of these is loaded to a different memory location in the ESP32-S3 device. What are these memory locations? I looked at the printout from loading firmware using the PlatformIO plugin in VSCode to find out. Otherwise, I’d read the data sheet for the chip to find out. Then I looked at example commands for esptool from t’web. A little bit of cobbling together and I got newly made firmware to load using the following commands:

Windows

python esptool.exe --baud 115200 --port COM4 --chip esp32-s3 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size detect 0x0000 bootloader.bin 0x8000 partitions.bin 0x10000 firmware.bin

Linux

esptool.py --baud 115200 --port /dev/ttyACM0 --chip esp32-s3 --before default_reset --after hard_reset write_flash -z --flash_mode dio --flash_freq 40m --flash_size detect 0x0000 bootloader.bin 0x8000 partitions.bin 0x10000 firmware.bin

What do all the other options in the command do? Magic incantations?

There is a ‘default_reset’ to put the device into a state where it can be programmed prior to loading the firmware.

There is a ‘hard_reset’ after loading the firmware to restart the device and tell it to load the new code.

–baud says how fast to load the firmware to the device.

–port defines the serial port name that the device is programmed through.

0x0000m 0x8000 and 0x1000 are the memory locations to load each firmware file to.

bootloader.bin is the firmware that sets up the ESP32-S3 chip when we start it up so that it can do ‘something’.

The partitions.bin firmware tells the ESP32-S3 how the internal memory is arranged.

The firmware.bin file has the code that we wrote to tell the device what to do in a format that can be loaded onto the ESP32-S3.