Fixing relative and absolute links in Word, Microsoft Office 2013

This article explains how to replace absolute hyperlink references with relative ones, which makes your Microsoft Word document and linked files portable. So when you send them to a different computer, clicking on the hyperlink will open the file that you shipped with the Word document and not try to open the file that is on the computer where you wrote the document.

Say you are writing a final report for a survey and you have a link to the original contract, which is in a folder in the same directory as your report.You burn the Word document and the folder with the contract file on to a DVD and send it to out to the survey’s project manager. Then the project manager complains that the link doesn’t work. Instead of trying to open the file on the DVD, the link is trying to open the file that is on the computer that you wrote the Word report on.

We need to make sure that the links are ‘relative’, meaning relative to where the Word file is, not ‘absolute’, meaning where the original file was. In Office 2010 this was relatively straight forwards. With Office 2013 it is not.
 
To prevent this issue happening in the first place there is an option to set in Word, hidden under a sub menu of a sub menu:
 
  • Select “File > Options”
  • Select “Advanced”
  • Scroll down to “General” and select “Web Options”
  • Select the Files tab
This is great if you are in sole control of the document. But as soon as somebody else edits the file without this option being set correctly, you will inherit a Word file with hyperlinks pointing all over the place. Time to edit the underlying field codes.
The commands we are going to use are:
alt-f9 – this shows all of the hyperlinks as field codes. It toggles.
ctrl-f9 – click on a hyperlink and use this shortcut to toggle a single hyperlink to being the field code.
 
 A field code for a hyperlink will have curly brackets and the word HYPERLINK . For example, under your nice blue underlined link called ‘8.16 Water bottom Horizon’, using alt-f9 you might see something like:
 
{HYPERLINK “08_Supporting_Documents/8.16%20Waterbottom%20Horizon%20ASCII”}
 
This is a nicely formed relative hyperlink. However, if you start to see unpleasantness like:
 

{HYPERLINK “file:///\\\\v07-fnp\\Ship\\Projects\\03_Past_Surveys\\survey\\11_Reports\\04_End_of_Job\\FT_Processing_Report_v2\\Appendix\\13%20Post%20Processing%20after%20PreSTM\\Survey_Report_v1.pdf”}

You have an absolute link, which will try to open a file on v07-fnp\\Ship, which is unfortunate as this is a server on a survey ship. We need to use Word’s search and replace function to get rid of the bumf at the start.

Search on:

file:///\\\\v07-fnp\\Ship\\Projects\\03_Past_Surveys\\survey\\11_Reports\\04_End_of_Job\\FT_Processing_Report_v2\\

replace with

<intentionally left blank>

One more search and replace should be done. For whatever reason, we need to replace all ‘\\’ with ‘/’.

The information in this post should help you to get your links working correctly in your Microsoft Word documents in Office 2013. I am required to use Microsoft products at work. Life is simpler with Linux. But that is a topic for a different post.

python – how to communicate between threads using pydispatcher

The pydispatcher module makes it straight forwards to communicate between different threads in the same process in python.

Why would I want to do this?

I am collecting and processing sensor data from an accelerometer and want to display this real-time. The interface has some controls to save the data and to change the sampling rate of the sensor. Naturally, I want to interact with the user interface without having to wait for the sensor data to be collected and processed. I also want the sensor to be continuously sampled, not having to wait for the real-time display to update.

I run the the graphical user interface (GUI) in one thread and use a separate thread to handle getting data from the sensor. This way the sensor is continuously sampled and the display remains responsive.

I use pydispatcher to send sensor measurements from the sensor thread the display thread. I also use pydispatcher to communicate from the display thread back to the sensor thread to control the rate that the sensor collects data or to stop data collection. So I have two way communication between the threads. I pass numpy arrays from the sensor thread to the display and send text from the display thread to the sensor thread. The text is then interpreted by the sensor thread to alter the sensor sampling rate, or stop sampling. Pydispatcher does not seem to mind what kind of data is sent as messages.

The application that I have described takes up quite a lot of code and is split over several classes. So I will present the code for a simpler example, which shows how to set up and apply pydispatcher and introduces some of the features that makes the library versatile.

Here is an example python 3 script that creates two threads and has them communicate. When the script is executed, as it will have the __name__ as __main__, so lines 46-50 will be the first to execute. A thread that instigates the Alice class is defined and created in lines 47-48 and a separate thread that instigates the Bob class is defined then started in lines 49-50.

In line 26 the alice_thread thread prints out a message ‘Alice is procrastinating’ every second.

In line 43 the bob_thread sends a message to the alice_thread every three seconds using a dispatcher. The alice_thread reacts to this dispatcher message by returning a message of her own to the bob_thread using a separate dispatcher.

If we look at line 15 in the Alice class, a dispatcher listener is set up:

dispatcher.connect(self.alice_dispatcher_receive, signal=BOB_SIGNAL, sender=BOB_SENDER)

This means that when a dispatcher.send statement with the signal BOB_SIGNAL and sender BOB_SENDER is executed anywhere else in the process, the method alice_dispatcher will be triggered so long as an instance of the Alice class has been created. In line 43, the Bob class sets up a dispatcher sender, which is designed to trigger the dispatcher listener in the Alice class described above.

dispatcher.send(message='message from Bob', signal=BOB_SIGNAL, sender=BOB_SENDER)

Having signal and sender names for each dispatcher listener and sender is a little confusing at first. Why do we have to define two identifiers for the dispatcher? Being able to define two identifiers allows us to group dispatchers from the same sender, using the sender identifier. Then we can have the same sender class sending different types of signal, for example data from different sensors, each one with the same sender identifier but each one with different signal identifier. This is verbose, but this verbosity makes for unambiguous easy to maintain code.

Lines 6-9 define the names of the signals and senders for Alice and Bob.

When the alice_thread receives a dispatch from the bob_thread thread, she replies with a dispatch sender of her own (line 21). The corresponding dispatch listener is defined in the Bob class in line 33.

''' demonstrate the pydispatch module '''
from pydispatch import dispatcher
import threading
import time

ALICE_SIGNAL='alice_signal'
ALICE_SENDER='alice_sender'
BOB_SIGNAL='bob_signal'
BOB_SENDER='bob_sender'

class Alice():
''' alice procrastinates and replies to bob'''
def __init__(self):
print('alice instantiated')
dispatcher.connect(self.alice_dispatcher_receive, signal=BOB_SIGNAL, sender=BOB_SENDER)
self.alice()

def alice_dispatcher_receive(self, message):
''' handle dispatcher'''
print('alice has received message: {}'.format(message))
dispatcher.send(message='thankyou from Alice', signal=ALICE_SIGNAL, sender=ALICE_SENDER)

def alice(self):
''' loop and wait '''
while(1):
print('Alice is procrastinating')
time.sleep(1)

class Bob():
''' bob contacts alice periodically '''
def __init__(self):
print('Bob instantiated')
dispatcher.connect(self.bob_dispatcher_receive, signal=ALICE_SIGNAL, sender=ALICE_SENDER)
self.bob()

def bob_dispatcher_receive(self, message):
''' handle dispatcher '''
print('bob has received message: {}'.format(message))

def bob(self):
''' loop and send messages using a dispatcher '''
while(1):
dispatcher.send(message='message from Bob', signal=BOB_SIGNAL, sender=BOB_SENDER)
time.sleep(3)

if __name__ == '__main__':
alice_thread = threading.Thread(target=Alice)
alice_thread.start()
bob_thread = threading.Thread(target=Bob)
bob_thread.start()
Output:
alice instantiated
Alice is procrastinating
Bob instantiated
alice has received message: message from Bob
bob has received message: thankyou from Alice
Alice is procrastinating
Alice is procrastinating
Alice is procrastinating
alice has received message: message from Bob
bob has received message: thankyou from Alice
Alice is procrastinating
Alice is procrastinating
alice has received message: message from Bob
bob has received message: thankyou from Alice
Alice is procrastinating
Alice is procrastinating
Alice is procrastinating
alice has received message: message from Bob
bob has received message: thankyou from Alice

To conclude. There are different ways to communicate between threads in python. I choose pydispatcher as the library allows me to write code that I can understand when I come back to it 6 months later and I don’t have to worry about the type of message that I am passing between the threads.

One minus alpha filter

I’ve got some real time accelerometer and gyroscope data coming in on my project to recognise hand gestures here. Naturally, I would like to be able to remove jitter and noise from the data as painlessly as possible. So we are into the world of real time digital filtering. Many books have been written on this subject and it is easy to dive ‘down the rabbit hole’ and lose a lot of your life testing filters. I want something that is ‘good enough’ quickly. From this stackoverflow answer, I got the idea for a simple implementation of a moving average filter:
x’ ←x’+α(x−x’)

where α<1

The first term, x’, is the new filtered value, calculated by taking the last filtered value and adding α.(last measured value-last filtered value).
So how to simulate and implement this?

Simulation

First of all, I will simulate the idea. Filters are implemented by using convolution. The input data is convolved with a filter operator. So I tried out a filter operator:

(1-α, α)

Here’s a simple script I ran using a Jupyter notebook and Python 3. I lifted the base code from the matplotlib examples page. I generated a sine wave and add some random noise in line 6. The filter function is defined in line 5, I am using alpha = 0.5 for this example. The function np.convolve in line 7 implements the filter. I had to knock off the last element of the filtered and difference data to get them all to plot, as one of the characteristics of a filter is that it will elongate the data set. Really, you need to ‘pad’ a data set at each end before applying convolution to remove ‘edge effects’ of the filter. But we are looking to quickly test and implement a filter here, not get bogged down in the technical minutia of filter design. Rabbit hole. Avoid.

import numpy as np
import matplotlib.pyplot as plt
ALPHA = 0.5
x = np.linspace(0, 2 * np.pi, 100)
filter = (1-ALPHA,ALPHA*1)
y = 2 * np.sin(x) + 0.1 * np.random.normal(x)
y_filt = np.convolve(y, filter)
y_diff = y - y_filt[:-1]
 
print(y)
print(y_filt[:-1])
print(y_diff[:-1])
 
fig, (ax0, ax1, ax2) = plt.subplots(nrows=3)
 
ax0.plot(x, y)
ax0.set_title('input')
 
ax1.plot(x, y_filt[:-1])
ax1.set_title('output')
 
ax2.plot(x, y_diff)
ax2.set_title('difference')
 
# Hide the right and top spines
ax1.spines['right'].set_visible(False)
ax1.spines['top'].set_visible(False)
# Only show ticks on the left and bottom spines
ax1.yaxis.set_ticks_position('left')
ax1.xaxis.set_ticks_position('bottom')
 
# Tweak spacing between subplots to prevent labels from overlapping
plt.subplots_adjust(hspace=0.5)
 
plt.show()

For an input, filtered output and difference plot, see below. Note that the difference plot is on a different scaling to the input and filtered output. Looks to have the same amplitude and that some of the random noise has been removed. It is not perfect, but it has helped and was fast and easy to implement.

Implementation

I am using micropython v1.7 on a pyboard v1.0 with an mpu6050 accelerometer/gyroscope for my hardware platform – see the diagram below. So how hard could it be to implement a simple one point filter? Errrr….

The filter code is straightforwards, see the snippet below. The function filter takes the latest sensor value as new_value and uses the last filtered value as old_value, returning the latest filtered value. I am using ALPHA as 0.5 for this test.

def filter(self, old_value, new_value):
        ''' simple moving average filter '''
        return (new_value*ALPHA + (1-ALPHA)*old_value)

This function is called from the main sensor scan and process while loop for each of the x,y and z accelerometers, shown in the snippet below.

        while(True):
            if (self.run_flag):
                if(self.acc_read_flag):
                    self.counter+= 1
                    (delta, x, y, z) = self.read_acc()
                    x_acc = x
                    x = self.filter(old_x, x)
                    y = self.filter(old_y, y)
                    z = self.filter(old_z, z)
                    print(START, self.counter, delta, x_acc, x, x-x_acc, END)
                    old_x, old_y, old_z = x, y, z

Have a look at the plot below. This shows the x-axis from an imu6050 module being sampled through a pyboard v1.0. at 100Hz. I wrote the firmware for this board using micropython v1.7 and the display software using python 3.4 with the pyqtgraph library. The x scale shows samples, the y scale shows acceleration in g.

So what can we see? The raw data looks jittery, the filtered data looks smoother and we can see the jitter that has been taken out in the difference plot. To characterise this filter properly I would need to start looking at the frequency spectrum of the raw and filtered data. But this is heading down the rabbit hole again.
I’ve quickly implemented a filter that looks to be doing what I want it to – removing noise from data. I can play with the alpha value to change the amount of smoothing. ‘The proof is in the eating’. If I can get my gesture recognition system to work with this simple filter implemented, then it is good enough.