Python Journal - The Curious Case of Python Imports

Table of Contents

• 1. The import statement
• 2. Selective imports
• 3. Investigating the imports
• 4. The __import__ function

1 The import statement

Python has a module system for easier maintenance of large codebases. This enables us to put Python statements and definitions in different files and 'import' them whenever needed in another script. Consider following script names 'constants.py'

constants.py

PI = 3.14
EXP = 2.7182

Our script 'constants.py' contains constants PI, and EXP (natural logarithmic base). If we ever need to use these constants, we can just import them from 'constants.py'

constants_tester.py

import constants

def get_area(circle_radius):
    return constants.PI * (circle_radius ** 2)

Another way to import constants is

from constants import PI

def get_area(circle_radius):
    return PI * (circle_radius ** 2)

This example only imports PI from our constants.py module. However, there's one question that I asked myself, if Python runs the module script (constants.py script) while importing it in another script, does the statement 'from constants import PI' also imports EXP?

2 Selective imports

from constants import PI

def get_area(circle_radius):
    return constants.PI * (circle_radius ** 2)

print(EXP)

We get following error

Traceback (most recent call last):
  File "constants_tester.py", line 11, in <module>
    print(EXP)
NameError: name 'EXP' is not defined

Which means it does not import EXP. Let's see if we can import EXP using constants module name

from constants import PI


def get_area(circle_radius):
    return PI * (circle_radius ** 2)

print(constants.EXP)

This also gives error as following

Traceback (most recent call last):
  File "constants_tester.py", line 10, in <module>
    print(constants.EXP)
NameError: name 'constants' is not defined

Interesting! This means that when we say 'from constants import PI', it just imports PI in the current namespace. It doesn't even recognize 'constants'

But what exactly happens under the hood?

3 Investigating the imports

We could look into globals(), locals(), and sys.modules to see what exactly Python imports

from constants import PI
import sys
from pprint import pprint

def get_area(circle_radius):
    return PI * (circle_radius ** 2)


pprint(globals()['PI'])
pprint(locals()['PI'])
pprint(sys.modules['constants'])

This gives following output

3.141592653589793
3.141592653589793
<module 'constants' from '/home/chinmaybhoir/PycharmProjects/python-practice/constants.py'>

You can see that sys.modules has an entry for 'constants', so the module is loaded. Why can not we access it then?

4 The __import__ function

Python has a built-in function __import__ that is called whenever the statement import is executed.

Following is taken verbatim from __import__ documentation

For example, the statement import spam results in bytecode resembling the following code:

spam = __import__('spam', globals(), locals(), [], 0)

The statement import spam.ham results in this call:

spam = __import__('spam.ham', globals(), locals(), [], 0)

Note how __import__ returns the toplevel module here because this is the object that is bound to a name by the import statement.

On the other hand, the statement from spam.ham import eggs, sausage as saus results in

_temp = __import__('spam.ham', globals(), locals(), ['eggs', 'sausage'], 0)
eggs = _temp.eggs
saus = _temp.sausage

You can see that the last usecase imports the module in a temporary variable _temp, and the variables (PI/EXP in our usecase) are assigned differently. This is the reason we can not access 'constants', or 'EXP' (since it does not appear in the fromlist attribute of __import__)

Author: Chinmay.Bhoir

Created: 2019-05-17 Fri 17:20

Emacs 24.5.1 (Org mode 8.2.10)

Validate

Python Journal - Generators

Table of Contents

• 1. The return statement
• 2. The yield statement

Generators are often not well understood by programmers who are used to normal execution flow of functions and are seeing the yield keyword in Python. Before moving on to generators, we need to understand how functions are executed normally. Consider a function that gives us the even numbers in given range

def even_numbers(start_number, end_number):
    # First creating an empty list that will be populated 
      as we iterate till the end_number
    even_nums = []
    for i in range(start_number, end_number+1):
        if i % 2 == 0:
            # append the number to our list
            even_nums.append(i)
    return even_nums

Above code gives us the required list. We can even use list comprehension for this purpose

def even_numbers(start_number, end_number):
    return [i for i in range(start_number, end_number + 1) if i % 2 == 0]

The main difference that we see in a generator and a normal function returning a value is the keyword yield. So what's the difference?

1 The return statement

Every function performs three actions (in a very informal way):

• To take inputs
• Perform operations on those inputs
• Return a result

When we say 'return a result', what actually happens is that the function, once completed, returns the control of execution to its caller. Our previous even_numbers() function can be called as follows

even_nos = even_numbers(5, 10)

When the program execution reaches this statement, the control is given to the function even_numbers and when the function is done with its work (in this case, creating a list of even numbers), it passes the control back to the caller statement along with the result. Short and simple! So what exactly are generators and why do we need them when we get what we want in normal function return statement? This has to do with 'storing the entire result in memory and then returning that result'. The list that is produced in even_numbers is stored in memory, and only when the function calculates all the elements does it returns that result. This produces a problem when the result that we want is too big to fit in memory. What if the range given to even_numbers doesn't fit in the memory? Doesn't it sound better if we could produce the numbers one at a time? This is where yield comes into picture

2 The yield statement

Instead of calculating an entire list of even numbers, we calculate the 'next even number' by using yield statement. Our function can be modified as:

def even_numbers(start, end):
    for i in range(start, end + 1):
        if i % 2 == 0:
            yield i

>>> even_numbers(5, 15)
<generator object even_nos at 0x7fec94571bf8>

You can see now that the function that we wrote isn't an ordinary function, rather it returns a generator object. Generator object can be iterated through by using next() method

>>> g = even_numbers(5, 10)
>>> next(g)
6
>>> next(g)
8
>>> next(g)
10
>>> next(g)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

You can see that after returning (or yielding) all values, when we called next on the generator, it raised the 'StopIteration' exception, which tells the program that generator has no more values to produce. We can iterate through the generator using normal for-loop

>>> g = even_numbers(5, 10)
>>> for i in g:
...    print(i)
...
6
8
10

You can see that the for loop printed all the values (by calling next() implicitly), and when the generator raises 'StopIteration', that's the cue for the loop to stop.

Author: Chinmay.Bhoir

Created: 2019-05-17 Fri 17:17

Emacs 24.5.1 (Org mode 8.2.10)

Validate