Python Journal - The Curious Case of Python Imports

Table of Contents

• 1. The import statement
• 2. Selective imports
• 3. Investigating the imports
• 4. The __import__ function

1 The import statement

Python has a module system for easier maintenance of large codebases. This enables us to put Python statements and definitions in different files and 'import' them whenever needed in another script. Consider following script names 'constants.py'

constants.py

PI = 3.14
EXP = 2.7182

Our script 'constants.py' contains constants PI, and EXP (natural logarithmic base). If we ever need to use these constants, we can just import them from 'constants.py'

constants_tester.py

import constants

def get_area(circle_radius):
    return constants.PI * (circle_radius ** 2)

Another way to import constants is

from constants import PI

def get_area(circle_radius):
    return PI * (circle_radius ** 2)

This example only imports PI from our constants.py module. However, there's one question that I asked myself, if Python runs the module script (constants.py script) while importing it in another script, does the statement 'from constants import PI' also imports EXP?

2 Selective imports

from constants import PI

def get_area(circle_radius):
    return constants.PI * (circle_radius ** 2)

print(EXP)

We get following error

Traceback (most recent call last):
  File "constants_tester.py", line 11, in <module>
    print(EXP)
NameError: name 'EXP' is not defined

Which means it does not import EXP. Let's see if we can import EXP using constants module name

from constants import PI


def get_area(circle_radius):
    return PI * (circle_radius ** 2)

print(constants.EXP)

This also gives error as following

Traceback (most recent call last):
  File "constants_tester.py", line 10, in <module>
    print(constants.EXP)
NameError: name 'constants' is not defined

Interesting! This means that when we say 'from constants import PI', it just imports PI in the current namespace. It doesn't even recognize 'constants'

But what exactly happens under the hood?

3 Investigating the imports

We could look into globals(), locals(), and sys.modules to see what exactly Python imports

from constants import PI
import sys
from pprint import pprint

def get_area(circle_radius):
    return PI * (circle_radius ** 2)


pprint(globals()['PI'])
pprint(locals()['PI'])
pprint(sys.modules['constants'])

This gives following output

3.141592653589793
3.141592653589793
<module 'constants' from '/home/chinmaybhoir/PycharmProjects/python-practice/constants.py'>

You can see that sys.modules has an entry for 'constants', so the module is loaded. Why can not we access it then?

4 The __import__ function

Python has a built-in function __import__ that is called whenever the statement import is executed.

Following is taken verbatim from __import__ documentation

For example, the statement import spam results in bytecode resembling the following code:

spam = __import__('spam', globals(), locals(), [], 0)

The statement import spam.ham results in this call:

spam = __import__('spam.ham', globals(), locals(), [], 0)

Note how __import__ returns the toplevel module here because this is the object that is bound to a name by the import statement.

On the other hand, the statement from spam.ham import eggs, sausage as saus results in

_temp = __import__('spam.ham', globals(), locals(), ['eggs', 'sausage'], 0)
eggs = _temp.eggs
saus = _temp.sausage

You can see that the last usecase imports the module in a temporary variable _temp, and the variables (PI/EXP in our usecase) are assigned differently. This is the reason we can not access 'constants', or 'EXP' (since it does not appear in the fromlist attribute of __import__)

Author: Chinmay.Bhoir

Created: 2019-05-17 Fri 17:20

Emacs 24.5.1 (Org mode 8.2.10)

Validate

Python Journal - Generators

Table of Contents

• 1. The return statement
• 2. The yield statement

Generators are often not well understood by programmers who are used to normal execution flow of functions and are seeing the yield keyword in Python. Before moving on to generators, we need to understand how functions are executed normally. Consider a function that gives us the even numbers in given range

def even_numbers(start_number, end_number):
    # First creating an empty list that will be populated 
      as we iterate till the end_number
    even_nums = []
    for i in range(start_number, end_number+1):
        if i % 2 == 0:
            # append the number to our list
            even_nums.append(i)
    return even_nums

Above code gives us the required list. We can even use list comprehension for this purpose

def even_numbers(start_number, end_number):
    return [i for i in range(start_number, end_number + 1) if i % 2 == 0]

The main difference that we see in a generator and a normal function returning a value is the keyword yield. So what's the difference?

1 The return statement

Every function performs three actions (in a very informal way):

• To take inputs
• Perform operations on those inputs
• Return a result

When we say 'return a result', what actually happens is that the function, once completed, returns the control of execution to its caller. Our previous even_numbers() function can be called as follows

even_nos = even_numbers(5, 10)

When the program execution reaches this statement, the control is given to the function even_numbers and when the function is done with its work (in this case, creating a list of even numbers), it passes the control back to the caller statement along with the result. Short and simple! So what exactly are generators and why do we need them when we get what we want in normal function return statement? This has to do with 'storing the entire result in memory and then returning that result'. The list that is produced in even_numbers is stored in memory, and only when the function calculates all the elements does it returns that result. This produces a problem when the result that we want is too big to fit in memory. What if the range given to even_numbers doesn't fit in the memory? Doesn't it sound better if we could produce the numbers one at a time? This is where yield comes into picture

2 The yield statement

Instead of calculating an entire list of even numbers, we calculate the 'next even number' by using yield statement. Our function can be modified as:

def even_numbers(start, end):
    for i in range(start, end + 1):
        if i % 2 == 0:
            yield i

>>> even_numbers(5, 15)
<generator object even_nos at 0x7fec94571bf8>

You can see now that the function that we wrote isn't an ordinary function, rather it returns a generator object. Generator object can be iterated through by using next() method

>>> g = even_numbers(5, 10)
>>> next(g)
6
>>> next(g)
8
>>> next(g)
10
>>> next(g)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
StopIteration

You can see that after returning (or yielding) all values, when we called next on the generator, it raised the 'StopIteration' exception, which tells the program that generator has no more values to produce. We can iterate through the generator using normal for-loop

>>> g = even_numbers(5, 10)
>>> for i in g:
...    print(i)
...
6
8
10

You can see that the for loop printed all the values (by calling next() implicitly), and when the generator raises 'StopIteration', that's the cue for the loop to stop.

Author: Chinmay.Bhoir

Created: 2019-05-17 Fri 17:17

Emacs 24.5.1 (Org mode 8.2.10)

Validate

The greatest crime scene on Earth

Note: I wrote this article about 3 years ago. The numbers and theories are subjected to change as new data comes in.

As of August 2011, 8.7 million species (give or take 1.3 million) survive on this wonderful pale blue planet of ours. Fighting to survive against the odds of being hunted or caught in a natural disaster. Today, human beings have dominated this planet in every aspect. Empowered with amazing technologies, we can survive almost anywhere on this planet. Where the blue whales rule, deep in the ocean, we took our high resolution cameras and shot their routines. Where the penguins huddle up in groups to warm their bodies in negative temperatures (in degree celcius), we have set up research camps. Where the complex net of species survive in rainforests of Amazon, we have made science fiction movies. Needless to say, today this planet belongs to human beings. But this wasn't always the case. Some time in past, till about 66 million years from today, creatures with sized ranging from small to gigantic, both herbivores and carnivores existed on this planet and ruled.

Dinosaurs fascinate me. Partly because of Steven Spielberg, partly because it's the closest creature to dragons that really existed and partly because of the coolest things that paleontologists get to do. Dinosaurs existed on this planet from about 200 million years ago (the start of Jurassic period)  up until 66 million years ago (end of Cretaceous period).

Scientists have collected the fossils from all around the world that strongly supports what I just said. But there was a mystery: dinosaurs were so powerful that it should have been impossible for mammals like us to have evolved alongside them. How did that happen? What caused the end of an era of those huge creatures?

Paleontologists and geologists study the Earth and the remains of the lost worlds by carefully examining the rocks. That’s what they do! It is, however, a matter of practice to differentiate between a fossil and a rock. After millions of years, bones lose their minerals and tend to look almost like rocks. In 1981, scientist Luis Walter Alvarez put forth his theory of a large body impact on Earth, based on the evidence he found in rocks. He found that there was a layer of clay consisting iridium and shocked quartz radioactively dated about 66 million years ago. Iridium is one of the rarest elements on Earth. It was hardly plausible to find it uniformly in a layer dated about 66 million years ago. But guess what contains iridium in rich content - extraterrestrial bodies, asteroids! This was his first clue. The second clue was shocked quartz. Shocked quartz are special type of quartz that is only created after a huge amount of pressure. The layer in the quartz gets deformed by the pressure. And exactly how much pressure is needed to do that? Well, even a volcano eruption can not deform a quartz! The first shocked quartz was found in underground nuclear bomb testing. So to deform a plane in quartz, you need energy compared to nuclear bomb! This kind of pressure can also be created by a fairly big asteroid impact. Now Alvarez had formed his theory. But he lacked the most important thing - the crime scene - the crater. He was in search of a crater that might have been created by such huge impact.

Let's go back to 1978. Geophysicists Antonio Camargo and Glen Penfield had found a gravity potential map with a curious ring of about 70 km across at gulf of Mexico. They were working for an oil company Petroleos Mexicanos. Penfield then dug deeper in published papers. He found a similar gravity map that was obtained a decade ago, but wasn't published due to corporate policy back then. Penfield then published his results and possibility of a geological even near the gulf of Mexico in a conference. But he didn't get much attention. Scientists were busy in attending conferences on Earth impact elsewhere in the world.

Later in 1990, a reporter for Houston Chronicles told a scientist about Penfield's discovery. Then the search for more evidence led to the conclusion that indeed an extraterrestrial body had hit the Earth near gulf of Mexico about 66 million years ago. Finally, the crime scene was found! Recent findings show that the crater created by asteroid is about 300 km wide. Due to it's geography, it's named 'The Chicxulub Crater'.

However, it is still hard to believe that one asteroid strike wiped out the dinosaurs across the planet. So let's see what might have happened after the asteroid strike.

The estimated size of the asteroid that hit Earth is about 10 km or more. The resulting energy due to impact is estimated around 240,000 gigatons of TNT (FYI the 2004 Indian Ocean earthquake that led to tsunami had an estimated energy of 9560 gigatons of TNT. 240,000 gigatons of TNT is about 8 billion times stronger than energy released by each atomic bomb dropped on Hiroshima and Nagasaki). This was A LOT of energy. Far more powerful than any major event in history of Earth for hundreds of millions of years. Such amount of Energy would power today's world for hundreds of years. But still, it wasn't enough to kill every dinosaur on Earth. It would, however, roast every living being present within 100 km radius instantly. What killed dinosaurs wasn't the impact, but it's after effects.

The impact caused huge amount of heat released in atmosphere. Saying that it might have caused fireballs thrown around as far as 200 km would not be an exaggeration. This impact must have caused a huge shock wave that spread around the half of the globe. It caused many largest mega tsunamis that Earth has ever seen. It released a huge amount of hot ash and gas that later covered almost the globe. This led to climate change. It brought some of the hottest years on Earth. It wiped out vegetation in huge areas. Herbivores didn't have much to eat, and so carnivores fell short of herbivores. The impact must have released huge amount of sulphur and carbon dioxide in atmosphere. Carbon dioxide will be balanced after many years. But sulphur stays longer. So first, it was a global warming, caused by carbon dioxide in atmosphere. And later, it was mini ice age, caused by sulphur. Animals as big as dinosaurs would not survive such impact and it's after effects. The only species that had a chance at survival were those who stayed close to ground, or even underground - mammals. After hundreds of years, Earth started to glow again, in the light of living beings that ultimately led to existence of human beings.

Chicxulub impact was HUGE. It was a hell on Earth. But ironically, we owe our existence to it.

Further reading:
[1] Wikipedia page for Chicxulub crater and related expedition