Python Modules and Packages — Organizing Your Code

Learning Objectives

By the end of this tutorial, you will be able to:

• Understand what modules and packages are and why they matter
• Use import, from...import, and as aliases effectively
• Navigate Python's module search path and sys.path
• Create packages with __init__.py files
• Choose between relative and absolute imports
• Understand namespace packages and PEP 420
• Avoid circular imports and other common pitfalls
• Use dynamic imports with __import__() and importlib.import_module()
• Implement lazy imports for performance optimization

What Are Modules?

A module is any .py file. Modules let you break code into reusable, logically grouped pieces.

# math_utils.py
def add(a, b):
    return a + b

def subtract(a, b):
    return a - b

PI = 3.14159

# main.py
import math_utils
print(math_utils.add(5, 3))       # 8
print(math_utils.subtract(10, 4)) # 6
print(math_utils.PI)              # 3.14159

Python ships with a standard library of built-in modules:

import os, sys, json
print(os.getcwd())
print(sys.version)
print(json.dumps({"key": "value"}))

Module Types

Import Statements

Basic import

import math
print(math.sqrt(16))  # 4.0

from...import

Pull specific names directly into your namespace:

from math import sqrt, pi
print(sqrt(25))  # 5.0

import...as Aliases

Give modules or functions shorter names:

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

from math import sqrt as square_root
print(square_root(49))  # 7.0

Wildcard Imports (Use Sparingly)

from math import *
print(sqrt(100))  # 10.0

Best Practice: Avoid from module import *. It pollutes your namespace and makes it unclear where names originate.

Import Comparison Table

The Import System

When you run import math_utils, Python:

1. Checks if the module is already in sys.modules
2. Searches sys.path for a matching file
3. Compiles and executes the module code
4. Stores it in sys.modules
5. Binds the name to the current namespace

sys.path

import sys
for path in sys.path:
    print(path)

/home/user/projects/myapp
/usr/lib/python3.11
/usr/lib/python3.11/site-packages

import sys
sys.path.append("/home/user/my_modules")
import my_custom_module

__name__

Every module has a special __name__ variable. When run directly, it equals "__main__":

# calculator.py
def add(a, b):
    return a + b

def subtract(a, b):
    return a - b

if __name__ == "__main__":
    print(add(2, 3))      # 5
    print(subtract(10, 4)) # 6

This pattern lets modules work both as importable libraries and standalone scripts.

Module Execution Flow

# When Python imports a module, it:
# 1. Creates a new namespace
# 2. Executes the module code in that namespace
# 3. Binds the module name to the namespace
# 4. Caches the module in sys.modules

import sys
print("math" in sys.modules)  # False before import
import math
print("math" in sys.modules)  # True after import

Packages

A package is a directory containing Python modules and an __init__.py file.

__init__.py

The __init__.py file marks a directory as a Python package:

my_package/
    __init__.py
    module_a.py
    module_b.py

# my_package/__init__.py
from .module_a import some_function

# my_package/module_a.py
def some_function():
    print("Function from module_a")

# main.py
from my_package import module_a
module_a.some_function()  # Function from module_a

Flat vs Regular Packages

Flat Package:

my_package/
    __init__.py
    a.py
    b.py

Regular (Nested) Package:

my_project/
    __init__.py
    models/
        __init__.py
        user.py
    utils/
        __init__.py
        validators.py

# my_project/models/user.py
class User:
    def __init__(self, name, email):
        self.name = name
        self.email = email

# my_project/utils/validators.py
def validate_email(email):
    return "@" in email

# main.py
from my_project.models.user import User
from my_project.utils.validators import validate_email

user = User("Alice", "alice@example.com")
print(validate_email(user.email))  # True

Sub-packages

# my_project/models/__init__.py
from .user import User
from .product import Product

# main.py
from my_project.models import User
user = User("Bob")
print(user.name)  # Bob

Package Structure Best Practices

my_project/
+-- pyproject.toml
+-- README.md
+-- src/
|   +-- my_project/
|       +-- __init__.py
|       +-- core.py
|       +-- models/
|       |   +-- __init__.py
|       |   +-- user.py
|       +-- utils/
|           +-- __init__.py
|           +-- helpers.py
+-- tests/
|   +-- __init__.py
|   +-- test_core.py
|   +-- test_models.py
+-- docs/

Relative vs Absolute Imports

Absolute Imports

Absolute imports specify the full path from the project root:

# my_project/services/auth.py
from my_project.models.user import User
from my_project.utils.validators import validate_email

class AuthService:
    def login(self, username, email):
        return validate_email(email)

Relative Imports

Relative imports use dots to specify position relative to the current file:

# my_project/services/auth.py
from ..models.user import User
from ..utils.validators import validate_email

One dot (.) means the current directory. Two dots (..) means the parent directory.

# my_project/models/user.py
from . import helpers  # Same directory

# my_project/services/payment.py
from ..models.product import Product  # Parent dir, then into models

Import Syntax Rules

# Relative imports use dots
from . import module          # current package
from .. import module         # parent package
from ...package import name   # grandparent package

# Absolute imports start from root
from my_package.subpackage.module import name

Namespace Packages (PEP 420)

Namespace packages let you split a package across multiple directories. They do not require __init__.py.

my_package/           # Traditional - needs __init__.py
    __init__.py
    module_a.py

my_namespace/         # Namespace - no __init__.py needed
    module_a.py

Splitting Across Directories

# /path/to/project_a/my_namespace/utils.py
def helper():
    return "Helper from project A"

# /path/to/project_b/my_namespace/models.py
class Model:
    pass

# main.py
import sys
sys.path.extend(["/path/to/project_a", "/path/to/project_b"])

from my_namespace.utils import helper
from my_namespace.models import Model
print(helper())  # Helper from project A

Namespace Package Benefits

Module Search Path

Python searches for modules in order: the current directory, directories in PYTHONPATH, and default installation-dependent directories.

import sys
for i, path in enumerate(sys.path):
    print(f"{i}: {path}")

Third-party packages from pip install into site-packages.

Search Order

1. Current directory (or script directory)
2. PYTHONPATH environment variable
3. Installation-dependent defaults
4. site-packages directory

# Add custom paths
import sys
sys.path.insert(0, '/custom/path')

The __all__ Variable

__all__ controls what gets exported when using from module import *.

# module_with_all.py
__all__ = ["public_func", "PublicClass"]

def public_func():
    return "I'm public"

def _private_func():
    return "I'm private"

class PublicClass:
    pass

from module_with_all import *
print(public_func())  # I'm public
# _private_func()    # NameError

Without __all__, from module import * imports everything not starting with _.

__all__ Best Practices

# Define explicit public API
__all__ = [
    'ClassName',
    'function_name',
    'CONSTANT_VALUE',
]

# Don't include private helpers
__all__ = []  # Empty for modules with no public API

Circular Imports

A circular import happens when two modules try to import each other, causing an ImportError.

# module_a.py  |  # module_b.py
import module_b |  import module_a

Solutions:

1. Extract shared code to a third module
2. Use lazy imports inside functions: def func(): from module_b import func_b

Circular Import Patterns

# BAD: Direct circular import
# module_a.py
import module_b

# module_b.py
import module_a

# GOOD: Lazy import
# module_a.py
def func_a():
    import module_b
    module_b.func_b()

# GOOD: Extract shared code
# shared.py
def shared_function():
    pass

# module_a.py
from shared import shared_function

# module_b.py
from shared import shared_function

Lazy Imports

Lazy imports delay module loading until the module is actually needed, improving startup time.

Basic Lazy Import

# Instead of importing at module level:
# import numpy as np  # Loaded immediately

# Use lazy import inside function:
def compute_array():
    import numpy as np  # Loaded only when function is called
    return np.array([1, 2, 3])

Lazy Import with importlib

import importlib

def get_numpy():
    return importlib.import_module('numpy')

# Usage
np = get_numpy()
arr = np.array([1, 2, 3])

Conditional Lazy Import

def process_data(data):
    # Only import pandas if needed
    if needs_dataframe(data):
        import pandas as pd
        return pd.DataFrame(data)
    return data

Lazy Import Benefits

Dynamic Imports

__import__() Built-in Function

The __import__() function is the low-level import mechanism:

# Basic usage
module = __import__('math')
print(module.sqrt(16))  # 4.0

# Import specific attribute
math_module = __import__('math', fromlist=['sqrt'])
print(math_module.sqrt(25))  # 5.0

# Dynamic module name
module_name = 'json'
module = __import__(module_name)
data = module.dumps({"key": "value"})

importlib.import_module()

The importlib module provides a more Pythonic interface:

import importlib

# Basic import
math_module = importlib.import_module('math')
print(math_module.pi)

# Import with package
module = importlib.import_module('os.path')
print(module.exists('/tmp'))

# Dynamic import based on configuration
def load_backend(backend_name):
    """Dynamically load a backend based on configuration."""
    module = importlib.import_module(f'backends.{backend_name}')
    return module.Backend()

Import Comparison

Common Patterns

Re-export via __init__.py

# my_package/__init__.py
from .module_a import ClassA
from .module_b import function_b

Entry Point Pattern

def main():
    print("Application starting...")

if __name__ == "__main__":
    main()

Private Module Convention

Modules starting with _ are internal: _internal.py is not part of the public API.

Plugin System Pattern

# plugin_loader.py
import importlib
import pkgutil

def load_plugins(package_name):
    """Dynamically load all plugins from a package."""
    package = importlib.import_module(package_name)
    plugins = {}
    
    for importer, modname, ispkg in pkgutil.walk_packages(
        path=package.__path__,
        prefix=package.__name__ + '.',
    ):
        module = importlib.import_module(modname)
        if hasattr(module, 'register'):
            plugins[modname] = module.register()
    
    return plugins

Common Mistakes

1. Forgetting import executes code: Top-level code in a module runs on every import. Guard side effects with if __name__ == "__main__".
2. Wildcard imports: from module import * pollutes your namespace. Use explicit imports instead.
3. Shadowing module names: math = "hello" shadows the math module. Use aliases like import math as m.
4. Forgetting __init__.py: Always include __init__.py for traditional packages.
5. Circular imports: Extract shared code to a third module, or use lazy imports inside functions.
6. Not using if __name__ == "__main__": Code runs on import, not just when executed directly.
7. Mixing relative and absolute imports: Stick to one style per project for consistency.

Practice Exercises

Exercise 1: Create a Calculator Package

Create a calculator/ package with basic.py and advanced.py modules.

Solution:

# calculator/basic.py
def add(a, b): return a + b
def subtract(a, b): return a - b
def multiply(a, b): return a * b
def divide(a, b):
    if b == 0:
        raise ValueError("Cannot divide by zero")
    return a / b

# calculator/advanced.py
import math
def power(base, exponent): return base ** exponent
def square_root(n): return math.sqrt(n)
def factorial(n): return math.factorial(n)

# calculator/__init__.py
from .basic import add, subtract, multiply, divide
from .advanced import power, square_root, factorial

# main.py
from calculator import add, multiply, square_root, factorial
print(add(5, 3))         # 8
print(multiply(4, 7))    # 28
print(square_root(16))   # 4.0
print(factorial(5))      # 120

Exercise 2: Build a Logging Utility

Create a package with a ConsoleLogger and FileLogger. Structure:

logger_utils/
    __init__.py
    console_logger.py
    file_logger.py

Solution:

# logger_utils/console_logger.py
import datetime

class ConsoleLogger:
    def __init__(self, prefix="LOG"):
        self.prefix = prefix

    def info(self, message):
        ts = datetime.datetime.now().strftime("%H:%M:%S")
        print(f"[{self.prefix}] [{ts}] INFO: {message}")

    def error(self, message):
        ts = datetime.datetime.now().strftime("%H:%M:%S")
        print(f"[{self.prefix}] [{ts}] ERROR: {message}")

# logger_utils/file_logger.py
import datetime

class FileLogger:
    def __init__(self, filename):
        self.filename = filename

    def log(self, level, message):
        ts = datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")
        with open(self.filename, "a") as f:
            f.write(f"[{ts}] {level}: {message}\n")

# logger_utils/__init__.py
from .console_logger import ConsoleLogger
from .file_logger import FileLogger

# main.py
from logger_utils import ConsoleLogger, FileLogger
console = ConsoleLogger("APP")
console.info("Application started")
file_logger = FileLogger("app.log")
file_logger.log("INFO", "User logged in")

Exercise 3: Resolve a Circular Import

Fix this circular import between order.py and customer.py:

# order.py (BROKEN)
from customer import get_customer

def create_order(customer_id):
    customer = get_customer(customer_id)
    return {"order_id": 1, "customer": customer}

# customer.py (BROKEN)
from order import create_order

def get_customer(customer_id):
    return {"id": customer_id, "name": "Alice"}

def place_order(customer_id):
    return create_order(customer_id)

Solution: Move imports inside functions:

# order.py (FIXED)
def create_order(customer_id):
    from customer import get_customer
    return {"order_id": 1, "customer": get_customer(customer_id)}

# customer.py (FIXED)
def get_customer(customer_id):
    return {"id": customer_id, "name": "Alice"}

def place_order(customer_id):
    from order import create_order
    return create_order(customer_id)

Key Takeaways

Modules and packages are the foundation of well-organized Python code. Mastering them lets you build maintainable, scalable applications that grow without becoming unwieldy.