Relative imports in Jupyter notebooks

How do we import a module from a .py or a .ipynb file into a Jupyter notebook from a different directory?
I wrote this post after answering a question on stackoverflow:
https://stackoverflow.com/questions/49282122/import-modules-in-jupyter-notebook-path/

For example, if we have the directory structure:

analysis.ipynb
/src/configuration.py
/src/configuration_nb.ipynb

How do we access the file configuration.py or the notebook configuration_nb.ipynb?

The nbimporter module helps us here:

pip install nbimporter

/src/configuration.py

class Configuration():
    def __init__(self):
        print('hello from configuration.py')

analysis.ipynb:

import nbimporter
from src import configuration

new = configuration.Configuration()

output:

hello from configuration.py

We can also import and use modules from other notebooks. If you have configuration_nb.ipynb in the /src module:

src/configuration_nb.ipynb:

class Configuration_nb():
    def __init__(self):
        print('hello from configuration notebook')

analysis.ipynb:

import nbimporter
from src import configuration_nb

new = configuration_nb.Configuration_nb()

output:

Importing Jupyter notebook from ......\src\configuration_nb.ipynb
hello from configuration notebook

Sending parameters to a Jupyter Notebook cell using click

Using libraries such as click and optparse we can send parameters to Python scripts when we run them from the command line. For example, passing a parameter called count with a value of 2 to a script called hello.py:

hello.py --count=2

How can I replicate this functionality in a cell of a Jupyter notebook? I like to run the same code in the notebook so that I can easily copy it to a stand alone script. Using sys.argv to pass parameters to the main function seemed one way to go and works with optparse:

from optparse import OptionParser
import sys


def main():
    parser = OptionParser()
    parser.add_option('-f', '--fake',
                      default='False',
                help='Fake data')
    (options,args) = parser.parse_args()
    print('options:{} args: {}'.format(options, args))
    if options.fake:
        print('Fake detected')
    
def test_args():
    
    print('hello')
    
if __name__ == '__main__':

    sys.argv = ['--fake', 'True' '--help']
    main()

output:

options:{'fake': 'False'} args: ['True--help']

Fake detected

Click seems to be flavor of the month, but I kept on getting a screen full of errors when I tried to run click through a Jupyter notebook cell. If we consider the Click example code:

import click

@click.command()
@click.option('--count', default=1, help='Number of greetings.')
@click.option('--name', prompt='Your name',
            help='The person to greet.')
def hello(count, name):
    """Simple program that greets NAME for a total of COUNT times."""
    for x in range(count):
        click.echo('Hello %s!' % name)

if __name__ == '__main__':
    hello()

If this file is called hello.py, then running from a command line:

hello.py 'Max' --count=3 

Gives this output:

Hello Max!

Hello Max!

Hello Max!

But using the same sys.argv trick that works with optparse produces a screen full of errors when the same code is run from a Jupyter notebook cell. The solution is to put the %%python magic at the start of the cell:

%%python

import sys
import click

@click.command()
@click.option('--count', default=1, help='Number of greetings.')
@click.option('--name', prompt='Your name',
            help='The person to greet.')
def hello(count, name):
    """Simple program that greets NAME for a total of COUNT times."""
    with open('echo.txt', 'w') as fobj:
        for x in range(count):
            click.echo('Hello %s!' % name)

if __name__ == '__main__':
    # first element is the script name, use empty string instead
    sys.argv = ['', '--name', 'Max', '--count', '3']
    hello()

A small tip, but one which cost me an hour or two of pondering. Finally I asked the hive mind of Stackoverflow. Please see this stackoverflow solution.