Automating loading micropython code to the BBC micro:bit in Linux

Introduction

When I edit micropython for a microbit project, I want the code to automagically load onto the microbit each time that I save the code. Then I want to see any messages from the microbit displayed. I worked out a toolchain that does this under Linux.

I summarise the toolchain in the next section, then go through each step in more detail.

For the examples in this post, I am editing a file called 'feeder.py'.

Toolchain

Use your favourite editor for creating your code with. The toolchain to load the code to the microbit kicks off when you save the file.

There are four tools needed to automate the process.

  • A script to locate and mount the microbit. I wrote a blog post about this script here.
  • The uflash utility to write the micropython file to the microbit.
  • screen terminal to display messages from the microbit
  • inotifywait command to monitor the micropython file being edited

uflash

Use pip to install this. Details are in the link given earlier in this post. To load a_file.py to an attached microbit, simply type:

uflash a_file.py

Using virtual environments makes my life easier, so I use one for my microbit programming. There are plenty of blog posts detailing how to set up and use these.

script to attach a microbit

I wrote a blog post on how to create a script that automatically locates and mounts a microbit here. This script is aliased to mm and md in my .bashrc file to mount a microbit and dismount a microbit.

screen

screen is built in to most Linux distros, so I use it here. This connects to the port that is in use by the microbit and displays any output from the microbit.

You need to know what port your microbit is attached to though. On my laptop this is often /dev/ttyACM3, which I use in the example below.

I present a script in the next section to automatically find the port that the microbit is connected to and fire up a screen connection using this.

To fire up the connection manually, if you know the port (in this example /dev/ttyACM3) use:

screen /dev/ttyACM3 115200

Remember to kill screen without leaving it running in the background by using:

control-a k

One potential problem with screen, is that it is easy to 'detach' from screen instead of ending the process, which leaves an instance of screen running invisibly which is attached to the same port as the microbit. When you come to try and run screen or connect to the microbit again, you will see an error like the one below:

can't open device "/dev/ttyACM3" 

To find and kill the errant screen instance:

lsof | grep /dev/ttyACM3

If you get something like:

screen  8610  elm    5u   CHR  166,3      0t0 5599015 /dev/ttyACM3

You still have a screen instance attached to the port. The PID is the second value in the list, in this case 8610. Kill it using e.g.

kill 8610

One way to prevent this potential problem is to replace screen with another terminal display tool such as gtkterm.

script to automate finding and connecting to the microbit

The python3 script below automates finding a connected microbit. The script starts a screen session on the port that the microbit is connected to.

'''
Find pyboard or microbit and open a terminal.
Created on 4 Nov 2015.
From http://wdi.supelec.fr/boulanger/MicroPython/
Usage: find_device [device]
Calls screen on "device". If device is not given, looks for /dev/ttyACM* devices
then /dev/tty.usbmodem* devices (for MacOS).
'''
#!/usr/bin/python
import sys
import os
import fnmatch

BAUD = 115200

def main():
    if len(sys.argv) > 1:
        ttymodems = [sys.argv[1]]
    else:
        ttymodems = fnmatch.filter(os.listdir('/dev'), 'ttyACM*')
        if len(ttymodems) == 0:
            ttymodems = fnmatch.filter(os.listdir('/dev'), 'tty.usbmodem*')
        if len(ttymodems) == 0:
            print('Error: no pyboard found.')
            sys.exit(1)
        ttymodems[0] = '/dev/' + ttymodems[0]
        print('connection at: {}'.format(ttymodems[0]))
    os.system('screen '+ttymodems[0] + ' {}'.format(BAUD))

if __name__ == "__main__":
    main()

inotifywait

inotifywait is a command line tool that can be configured to watch a file for a change, then trigger an action. Here we use it to watch for a change in the micropython file being edited. When this file is saved, inotifywait mounts the microbit and flashes the file to it.

If infotifywait is not already installed, then you need to install inotify-tools.

The command to watch a file called feeder.py for a change, then run the alias mm (to mount the microbit) and to flash the file to the microbit is:

while inotifywait -e modify feeder.py ; do mm && uflash feeder.py ; done

I would have formatted the above line as the other code in this post but found that the syntax highlighter plug-in refuses to show the ampersand symbol correctly.

notes

Some editors have the 'feature' of 'kind of' saving the file you are editing while you are editing it. This creates a new timestamp for the file and activates inotifywait without your saving the file. mu is one of these editors. In this case, I do not use inotify and run the following command from the command line each time I want to flash the updated file. In this example the updated file is called feeder.py:

mm && uflash feeder.py

but why not use a Makefile

Makefiles are typically used to build programming projects, detailing the dependencies of each component. The file is run when 'make' is typed on the command line.

The one-liner command using inotifywait is fine for this simple example where we are editing a single file. Which is most of what I do with the microbit and micropython. Makefile comes into its own for more complex builds involving multiple files. It allows for extra run-time options to be added, e.g. to clean files created by earlier builds.

For completeness, I present a simple Makefile below for this project. Makefile is a useful tool that is worth learning to use for when we have larger projects to build.

Makefile

I created a Makefile containing commands to mount the microbit to the Linux filesystem, then flash a micropython file to the attached microbit.

An example Makefile is below. Note: Use tabs to create the indents after the line 'run:'. Or the World Ends. The supporting file microbit_mount.sh is in the same directory as feeder.py to keep it simple.

MICROPYTHON_FILE=feeder.py

run:
	./microbit_mount.sh mount
	uflash $(MICROPYTHON_FILE)

In this example, microbit_mount.sh is the script to attach a microbit, detailed above.

uflash is the tool used to write the .py file to the microbit. This is detailed above.

To automate the Makefile being run each time that we save new code, we can use the inotifywait command below:

while inotifywait -e close_write feeder.py ; do make; done

Powering the BBC micro:bit with a 5V power bank

I had an interesting question to a blog post I made about running a DC motor from a microbit. The blog post is here. I am pleased that anybody reads my blogs.

Doesn't applying 5V to the microbit board exceed the voltage spec?

When we connect the board to our laptops or PCs, we apply 5V to the board through the USB connector. Sometimes I run the board from a USB power bank. This supplies 5V to the microbit. The advantage of using the power bank over AAA batteries is that I can use the power bank to also run e.g. a DC motor. So far I have not cooked off a microbit doing this.

But I still only measure 3.2V on the pads on the board edge connector.

Why is this?

Let's look at the board circuit schematics. Please find a screengrab showing the USB connector on the left and the microcontroller on the microbit on the right.

BBC micro:bit schematic fragment, showing USB connector and the microcontroller.

Let's zoom in on the connector:

Schematic showing the USB connector.

The 5V input voltage on the USB connector is given the net label VBUS_IF. This label reappears next to the little number 5 - which means this is where the track connects to on the PCB. The zener diode to the left of the number 5 and the 10 Ohm resistor R31 to the right combine to protect the board in the case of reverse polarity - if somehow the voltage is applied to the USB connector backwards, the current flows to ground, not into the microcontroller.

The capacitor C31 and C33 are there to act as little charge stores for the microcontroller for when it needs a sudden 'oomph' of charge that the USB power supply cannot supply quickly enough. These are called decoupling capacitors.

The +5V signal is renamed VBUS_IF_2 to the right of R31. Lets look at where this goes into the microcontroller. I labelled the net VBUS_IF_2:

So our +5V USB voltage connects to the microcontroller on a dedicated pin VREGIN. This is the input to a regulator inside of the microcontroller. The output of this regulator is a DC voltage called +3.3V_IF. This output supplies the +3.3V seen elsewhere in the board and on the board edge connector pads.

Now, if the input to VREGIN is below 3.3V, we are not going to see 3.3V on the output pads. In fact, I see 3.2V with an input of 5V, so there is a 0.1V drop somewhere on the board.

So, in summary, we can apply 5V through the USB connector.

One last schematic fragment. This shows the battery connector and where this supply connects with the voltage generated by the internal regulator on the microcontroller:

Microbit schematic showing the battery connector and how it connects to the board voltage tracks.

The battery voltage goes through a low drop BAT60A schottky diode labelled D2. This then connects to V_TGT. This is the main voltage rail for powering most of the board components, such as the magnetometer and accelerometer. This is also the rail that connects to the board edge connector on the pad labelled '3V'.

Both the output from the microcontroller regulator, labelled 3.3V_IF and the voltage from the battery labelled VBAT connect to this rail through BAT60A schottky diodes. The diodes mean that the battery voltage and the regulator output voltages are isolated from each other, but either can produce V_TGT.

What if both a battery and a USB connector are connected? Whichever is the higher of VBAT or 3.3V_IF, minus a small voltage drop created by the BAT60A diode, will produce V_TGT. The voltage drop across the diode is around 0.1-0.2V according to the datasheet for the BAT60A.

It would be dangerous to the board to apply 5V to the battery connector. I am not about to try this. The battery connector bypasses the regulator in the microcontroller, so applying a high voltage to this connector applies the same high voltage to the components on the microbit board, minus a small drop across the BAT60A diode. Many of these are not rated to work at 5V.

The board is designed to take 5V only through the USB connector.

Configuring a Python virtualenv in Debian

Python virtual environments are a good idea. Naturally, I had a few problems getting mine to work properly. I found that my globally installed libraries were visible the activated venv. The whole point of a venv is to isolate libraries. The globally installed libraries that are not also explicitly installed in the venv should not be usable.

I use Debian and the bash shell.

So how did I fix what I found is a common problem?

Summary:

# blank the PYTHONPATH environment variable
export PYTHONPATH=
virtualenv my_venv
./my_venv/bin/source/activate
# use python3, not python as a REPL or the interpreter in e.g. VSCode
# use pip install, not python -m pip install

I have this line in my .bashrc file:

PYTHONPATH="${PYTHONPATH}:~/.local/lib/python3.7/site-packages/"

This means that when I create my venv, this path gets added to the PYTHONPATH environment variable in the venv. This allows the venv to see the packages in ~/.local/lib/python3.7/site-packages/. Which we do not want.

To prevent this, blank the PYTHONPATH before creating the venv by typing:

export PYTHONPATH=

We can check on what the PYTHONPATH environment variable is by entering:

echo $PYTHONPATH

This should now be blank.

Go ahead and create a new venv by typing e.g.

virtualenv my_venv

This is where things continued to be a little.... non-standard. I should be able to use:

python -m pip freeze

To see only the packages that I install in the venv. But I see all the global packages.

Using only:

pip

shows me what I should see. I would like to know why.

Similarly, I should get a REPL with access to only the packages installed in the venv using:

python

but typing this starts a REPL that allows me to import globally installed packages.

To get around this, use:

python3

Now things behave as they should. I cannot import a package that is globally installed but not installed in the venv. Which is the correct behaviour for a venv. I use the path to the python3 executable in the my_venv/bin directory as the interpreter in e.g. VSCode when I want to use a venv.

I found I had several installations of pip in various locations. I deleted all of them except for the one in /usr/bin.

If you want to learn more about virtual environments, I recommend this series of short videos: https://calmcode.io/virtualenv/intro.html

The videos recommend using the command:

python -m pip

in the venv, which I found to be unreliable.

Instead I just use:

pip

Check that this is pointing to the correct executable by typing:

which pip

Running handShake in administrator mode to operate Grid 3

Sensory Software's Grid 3 is a popular communication software package, running in Windows. Naturally, I would like handShake to be able to operate this software through the software keystrokes that handShake generates.  To get Grid 3 to respond to a software keystroke, I have to 'elevate' the base.py script which runs on the communication device to run as an Administrator.

There is a second solution. I can use a Freetronics Leostick USB dongle as a pretend keyboard and have this generate keystrokes that appear as coming from a physical keyboard. I did this for a while, but this adds a layer of complexity and expense to the project. The simplest solution is to run handShake as an Administrator when using Grid 3, or other software that requires software keystrokes to come from an elevated source.

I tested out adding the functionality for handShake to detect when Grid3 was running, then automatically try to elevate the base.py script to run in Administrator mode. I got this running. Then removed the functionality. Why? Security. Software running as Administrator can damage your system if incorrectly or maliciously written.

Now handShake detects if Grid 3 is running and advises that this requires the software to be restarted as an Administrator, but does not try to automate this restart. The decision is left to the user.

The Grid 3 software is detected using the code shown below. The code looks through the titles of the windows for any that match the ones in the list called ADMIN_SOFTWARE. At the moment there is only one title to check for - 'grid'. This is the title for a window running Grid3. As I find other packages that demand that my script runs as Administrator to have the ability to interact with it, then I will add their titles to the ADMIN_SOFTWARE list.

ADMIN_SOFTWARE = ['grid']

def target_admin_sware(software=ADMIN_SOFTWARE):
    ''' Check if target software requires this script to run as Administrator. '''
    toplist, winlist = [], []
    logging.info('Looking for software that requires elevation to Aministrator: {}'
        .format(ADMIN_SOFTWARE))
    def _enum_cb(hwnd, results):
        winlist.append((hwnd, win32gui.GetWindowText(hwnd)))
    win32gui.EnumWindows(_enum_cb, toplist)
    for sware in software:
        # winlist is a list of tuples (window_id, window title)
        for hwnd, title in winlist:
            if sware in title.lower():
                logging.info('found software requiring Administrator mode {}'
                    .format(title))
                return True
    return False

Running handShake as Administrator is a choice that the user makes and implements if he or she deems necessary.  As all of the code is on the project GitHub site, the code can be reviewed to check that it is safe to run as an Administrator.

One of the advantages of open source projects is that they are open to this kind of scrutiny to find security flaws.

For interest, I detail the code I added and then removed from the base.py script to enable it to detect if it is running as an administrator and if not, request to be restarted as an Administrator.

I added the option to run the script as an Administrator from the command line using the click library.

click.command()

@click.option('-a', '--admin', default=False, 
    help='Run as administrator. Required for Grid 3.')

@click.option('-k', '--keystroke', default='F1',
     help='Keystroke to send. Default is F1.')

def main(admin, keystroke):
    logging.info('software keystroke is {}'.format(keystroke))
    logging.debug('admin flag is {}'.format(admin))
    if is_admin():
        logging.info('running as administrator')
    else:
        logging.info('not running as administrator')
        if admin:
            logging.info("restarting as administrator")
            elevate()
    service_microbit(keystroke) 

To test if the script is running as an administrator I added this method:

def is_admin():
    ''' Is the script running as an Administrator? '''
    try:
        return windll.shell32.IsUserAnAdmin()
    except:
        return False