Python PIP — Package Installation and Management

Learning Objectives

By the end of this tutorial, you will be able to:

• Understand what PIP is and how it works with PyPI
• Install, upgrade, and uninstall Python packages using PIP
• Use version specifiers to pin or constrain package versions
• Create and manage requirements.txt files for reproducible environments
• Configure PIP via pip.conf for custom indexes and mirrors
• Diagnose and resolve common PIP installation errors
• Audit packages for known security vulnerabilities
• Follow best practices for production-ready dependency management
• Publish packages to PyPI
• Use build systems for package distribution

What is PIP?

PIP is the recursive acronym for "PIP Installs Packages." It is the standard package manager for Python, downloading libraries from the Python Package Index (PyPI) — a public repository with over 400,000 packages.

PIP is bundled with Python 3.4+ and handles:

• Downloading packages from PyPI or other indexes
• Resolving and installing dependencies automatically
• Uninstalling and upgrading packages

Verify PIP is installed:

$ pip --version
pip 23.3.1 from /usr/lib/python3/dist-packages (python 3.11)

If PIP is missing, bootstrap it:

$ python -m ensurepip --upgrade
$ python -m pip install --upgrade pip

Installing Packages

Basic Installation

$ pip install requests
Collecting requests
  Downloading requests-2.31.0-py3-none-any.whl (62 kB)
Installing collected packages: urllib3, idna, charset-normalizer, certifi, requests
Successfully installed certifi-2023.11.17 charset-normalizer-3.3.2 idna-3.6 requests-2.31.0 urllib3-2.1.0

PIP automatically resolves and installs all required dependencies.

Installing a Specific Version

$ pip install requests==2.28.2

Installing from a Requirements File

$ pip install -r requirements.txt

Installing in Editable Mode

During development, install a local package so changes take effect immediately:

$ pip install -e .

This reads setup.py or pyproject.toml from the current directory and links the package into your environment.

Installing with Optional Dependencies

Many packages provide optional feature groups:

$ pip install "requests[security]"
$ pip install "pandas[performance]"
$ pip install "black[jupyter]"

pip install Command Options

Version Specifiers

PIP supports several operators for constraining versions:

Practical Examples

$ pip install "numpy>=1.24,<2.0"   # constrain to numpy 1.x
$ pip install "Django~=4.2"        # Django 4.2.x
$ pip install "flask!=2.3.1"       # exclude a known buggy version

The compatible release operator ~= constrains the package to bugfix releases:

requests~=2.31.0  ->  >=2.31.0, <2.32.0
flask~=3.0        ->  >=3.0.0, <4.0.0

Version Specifier Examples

# Exact version
requests==2.31.0

# Minimum version
requests>=2.28.0

# Range
requests>=2.28,<3.0

# Compatible release (tilde equals)
requests~=2.31.0

# Exclude version
requests!=2.29.0

# Multiple conditions
requests>=2.28,!=2.29.0,<3.0

# Arbitrary equality (for special cases)
requests===2.31.0

Uninstalling Packages

Remove a package and its unneeded dependencies:

$ pip uninstall -y requests
Successfully uninstalled requests-2.31.0

Uninstall multiple packages at once:

$ pip uninstall -y requests flask django

pip uninstall Options

Listing Packages

List All Installed Packages

$ pip list
Package            Version
------------------ -------
beautifulsoup4     4.12.2
certifi            2023.11.17
flask              3.0.0
requests           2.31.0
urllib3            2.1.0

List Outdated Packages

$ pip list --outdated
Package    Version Latest   Type
---------- -------- -------- -----
pip        23.3.1   24.0     wheel
urllib3    2.1.0    2.2.1    wheel

Show Package Info

$ pip show requests
Name: requests
Version: 2.31.0
Summary: Python HTTP for Humans.
Requires: certifi, charset-normalizer, idna, urllib3

pip list Options

pip freeze

pip freeze outputs installed packages in requirements.txt format:

$ pip freeze
beautifulsoup4==4.12.2
certifi==2023.11.17
flask==3.0.0
requests==2.31.0
urllib3==2.1.0
werkzeug==3.0.1

Capture exact versions with pip freeze > requirements.txt and recreate with pip install -r requirements.txt. Exclude editable packages with --exclude-editable.

pip freeze Options

Requirements Files

A requirements.txt file lists every package your project needs, pinned to specific versions.

Example requirements.txt

# Web framework
flask==3.0.0
gunicorn==21.2.0

# HTTP client
requests==2.31.0

# Database
sqlalchemy==2.0.23
psycopg2-binary==2.9.9

# Testing
pytest==7.4.4
pytest-cov==4.1.0

# Linting
black==24.2.0
ruff==0.2.2

Installing from Requirements

$ pip install -r requirements.txt

Dev vs Production Dependencies

Create separate files for different environments:

# requirements-dev.txt
-r requirements.txt
pytest==7.4.4
pytest-cov==4.1.0
black==24.2.0
mypy==1.8.0

$ pip install -r requirements.txt          # production
$ pip install -r requirements-dev.txt      # development

Requirements File Syntax Reference

# Comments start with #

# Exact version pinning
package==1.0.0

# Minimum version
package>=1.0.0

# Maximum version
package<=1.0.0

# Compatible release
package~=1.0.0

# Exclude version
package!=1.0.0

# Multiple versions
package>=1.0,<2.0

# Install from URL
package @ https://example.com/package.tar.gz

# Install from Git repository
package @ git+https://github.com/user/repo.git@main#egg=package

# Install from local path
package @ file:///path/to/package.whl

# Editable install from Git
-e git+https://github.com/user/repo.git@main#egg=package

# Include another requirements file
-r base-requirements.txt

# Options
--no-binary :all:           # Force source distribution
--only-binary :all:         # Force wheel only
--find-links ./local-packages  # Additional package sources
--index-url https://pypi.org/simple  # Package index URL

Upgrading Packages

Upgrade a Single Package

$ pip install --upgrade requests
$ pip install -U requests                  # short form

Upgrade pip Itself

$ python -m pip install --upgrade pip
Successfully installed pip-24.0

Upgrade All Outdated Packages

PIP has no built-in flag for this. Use a one-liner:

# Linux/macOS
$ pip list --outdated --format=freeze | grep -v '^\-e' | cut -d = -f 1 | xargs -n1 pip install -U

# Windows PowerShell
$ pip list --outdated --format=freeze | ForEach-Object { $_.split('==')[0] } | ForEach-Object { pip install --upgrade $_ }

Downgrade a Package

$ pip install requests==2.28.0

Package Indexes (PyPI)

By default, PIP downloads from PyPI (https://pypi.org). You can override this with alternative indexes.

$ pip install --index-url https://pypi.tuna.tsinghua.edu.cn/simple requests
$ pip install --extra-index-url https://pypi.mycompany.com/simple/ mypackage
$ pip install --index-url https://pypi.mycompany.com/simple/ \
  --trusted-host pypi.mycompany.com mypackage

PyPI Commands

# Search PyPI (deprecated - use web search)
pip search package_name

# Download package without installing
pip download package_name

# Wheel creation
pip wheel package_name

Private Registries

Corporate PyPI Mirrors

# Using --index-url flag
pip install --index-url https://pypi.mycompany.com/simple/ package

# Using environment variable
export PIP_INDEX_URL=https://pypi.mycompany.com/simple/
pip install package

pip.conf for Private Registries

[global]
index-url = https://pypi.mycompany.com/simple
extra-index-url = https://pypi.org/simple
trusted-host = pypi.mycompany.com

Authentication

# Using username:password in URL
pip install --index-url https://user:pass@pypi.mycompany.com/simple/ package

# Using .netrc file
# ~/.netrc
machine pypi.mycompany.com
    login myuser
    password mypassword

pip.conf

The pip.conf file configures PIP globally or per user, eliminating repetitive flags.

File Locations

Example pip.conf

[global]
index-url = https://pypi.tuna.tsinghua.edu.cn/simple
trusted-host = pypi.tuna.tsinghua.edu.cn
timeout = 60

[install]
no-warn-script-location = true

[freeze]
exclude-editable = true

For a private index, set index-url and add extra-index-url to fall back to PyPI:

[global]
index-url = https://pypi.mycompany.com/simple
extra-index-url = https://pypi.org/simple
trusted-host = pypi.mycompany.com

pip.conf Options

Build Systems

Building Packages

# Install build tools
pip install build

# Build source distribution and wheel
python -m build

# This creates:
# dist/
#   myproject-0.1.0.tar.gz      (source distribution)
#   myproject-0.1.0-py3-none-any.whl  (wheel)

pyproject.toml Build Configuration

[build-system]
requires = ["setuptools>=68.0", "wheel"]
build-backend = "setuptools.backends._legacy:_Backend"

setup.py (Legacy)

from setuptools import setup, find_packages

setup(
    name="myproject",
    version="0.1.0",
    packages=find_packages(),
    install_requires=[
        "requests>=2.28",
        "flask>=2.3",
    ],
    python_requires=">=3.9",
)

Building and Publishing

# Build
python -m build

# Upload to TestPyPI
twine upload --repository testpypi dist/*

# Upload to PyPI
twine upload dist/*

PyPI Publishing

Package Structure

myproject/
+-- pyproject.toml
+-- README.md
+-- LICENSE
+-- src/
|   +-- myproject/
|       +-- __init__.py
|       +-- core.py
+-- tests/
    +-- test_core.py

Publishing Workflow

# 1. Install build tools
pip install build twine

# 2. Build the package
python -m build

# 3. Check the package
twine check dist/*

# 4. Upload to PyPI
twine upload dist/*

# 5. Verify
pip install myproject

TestPyPI

# Upload to TestPyPI first
twine upload --repository testpypi dist/*

# Install from TestPyPI
pip install --index-url https://test.pypi.org/simple/ myproject

Common Errors and Fixes

1. ModuleNotFoundError

PIP installed the package but Python cannot find it.

Fix: Install into the same Python interpreter you are using:

$ python -m pip install requests

2. Permission Denied

You lack write access to the system-wide site-packages directory.

Fix: Install with --user or use a virtual environment:

$ pip install --user requests

3. ERROR: Could not find a version that satisfies the requirement

The requested version does not exist or is incompatible with your Python version.

Fix: Check available versions on PyPI and relax the constraint:

$ pip install "requests>=2.28,<3.0"

4. ERROR: No matching distribution found

The package may not support your platform or Python version.

Fix: Verify your Python version and architecture:

$ python --version
$ python -c "import struct; print(struct.calcsize('P') * 8)"

5. SSL: CERTIFICATE_VERIFY_FAILED

Corporate firewalls intercept HTTPS traffic.

Fix: Set the REQUESTS_CA_BUNDLE environment variable:

$ set REQUESTS_CA_BUNDLE=/path/to/corporate-ca-bundle.crt

6. Build Failure for C Extensions

Some packages require a C compiler. Install build tools:

$ winget install Microsoft.VisualStudio.2022.BuildTools   # Windows
$ sudo apt-get install python3-dev build-essential          # Linux
$ xcode-select --install                                   # macOS

7. Dependency Conflict

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed.

Fix: Use pip check to identify conflicts and update or downgrade as needed.

Security (pip-audit)

pip-audit scans installed packages against the OSV database for known vulnerabilities.

Installation and Usage

$ pip install pip-audit
$ pip-audit
Package    Version  ID            Fix Versions
---------- -------- ------------- -------------
jinja2     3.1.2    PYSEC-2024-10 3.1.4
urllib3    2.1.0    PYSEC-2023-212 2.1.1
Found 2 known vulnerabilities in 16 packages.

Auto-Fix Vulnerabilities

$ pip-audit --fix

Audit a Requirements File

$ pip-audit -r requirements.txt

CI/CD Integration

- name: Audit dependencies
  run: |
    pip install pip-audit
    pip-audit

Best Practices

1. Always use virtual environments — isolate project dependencies from system Python
2. Pin exact versions in requirements.txt for reproducible builds
3. Separate dev and production dependencies into different files
4. Run pip-audit regularly as part of your workflow and CI pipeline
5. Use python -m pip instead of bare pip to target the correct interpreter
6. Run pip check to verify your environment has no broken dependency chains
7. Keep pip updated with python -m pip install --upgrade pip
8. Use --dry-run to preview changes before installing
9. Document your build process in README or CONTRIBUTING files
10. Use lock files (poetry.lock, Pipfile.lock) for deterministic builds

Common Mistakes

Mistake 1: Installing Packages Globally Without a Virtual Environment

Problem: System-wide installs cause version conflicts between projects.

Solution: Always activate a virtual environment first:

$ python -m venv myproject-env
$ source myproject-env/bin/activate
$ pip install flask

Mistake 2: Committing __pycache__ and .pyc Files

Problem: Compiled bytecode files are platform-specific and should not be version-controlled.

Solution: Add to .gitignore:

__pycache__/
*.pyc
*.pyo
.venv/
*.egg-info/
dist/
build/

Mistake 3: Using pip install Without Version Constraints

Problem: Installing the latest version may break your code after a future release.

Solution: Pin at least major versions:

$ pip install "requests>=2.31,<3.0"

Mistake 4: Mixing PIP Versions Across Python Interpreters

Problem: Running pip install installs into whichever Python pip is linked to, which may not be the one you expect.

# On a system with Python 3.10 and Python 3.11
$ pip install requests          # installs into Python 3.10
$ python3.11 -c "import requests"  # ModuleNotFoundError

Solution: Always use python -m pip:

$ python3.11 -m pip install requests

Mistake 5: Forgetting to Update requirements.txt

Problem: Teammates run pip install -r requirements.txt and get a stale environment.

Solution: Regenerate the requirements file whenever you change dependencies:

$ pip install <new-package>
$ pip freeze > requirements.txt

Practice Exercises

Exercise 1: Create a Requirements-Based Project Setup

Task: Create a project with a virtual environment and a requirements.txt containing requests, flask, and pytest.

Solution:

$ mkdir myproject && cd myproject
$ python -m venv .venv
$ source .venv/bin/activate           # Linux/macOS
$ .venv\Scripts\activate              # Windows
$ pip install requests flask pytest
$ pip freeze > requirements.txt
$ cat requirements.txt

Expected output:

blinker==1.7.0
certifi==2023.11.17
charset-normalizer==3.3.2
click==8.1.7
flask==3.0.0
idna==3.6
jinja2==3.1.2
markupsafe==2.1.3
pytest==7.4.4
requests==2.31.0
urllib3==2.1.0
werkzeug==3.0.1

Exercise 2: Audit and Upgrade Vulnerable Packages

Task: Install pip-audit, scan your environment, and upgrade any package with a known vulnerability.

Solution:

$ pip install pip-audit
$ pip-audit                              # scan for vulnerabilities
$ pip-audit --fix                        # auto-upgrade vulnerable packages
$ pip-audit                              # re-scan to confirm

Exercise 3: Build a Multi-Environment Requirements Setup

Task: Create requirements.txt (production) and requirements-dev.txt (with black, ruff, mypy).

Solution:

$ python -m venv .venv && source .venv/bin/activate
$ pip install flask requests sqlalchemy
$ pip freeze > requirements.txt
$ pip install black ruff mypy

Write requirements-dev.txt:

-r requirements.txt
black==24.2.0
ruff==0.2.2
mypy==1.8.0

Verify in a clean environment:

$ python -m venv test-env && source test-env/bin/activate
$ pip install -r requirements-dev.txt
$ pip list

Exercise 4: Create a Private Package Index

Task: Set up a local package directory and configure pip to use it.

Solution:

# Create local packages directory
mkdir local-packages
pip download flask -d local-packages/

# Create requirements file with local source
echo "--find-links ./local-packages" > requirements.txt
echo "--only-binary :all:" >> requirements.txt
echo "flask" >> requirements.txt

# Test installation
pip install -r requirements.txt

Key Takeaways

• PIP is Python's standard package installer, downloading from PyPI
• Use python -m pip to ensure you install into the correct Python environment
• Pin versions in requirements.txt for reproducible builds
• Use version specifiers (==, >=, ~=) to control which versions get installed
• Always use a virtual environment to isolate project dependencies
• Run pip-audit regularly to scan for known security vulnerabilities
• Separate production and development dependencies into different requirements files
• Configure PIP via pip.conf for custom indexes, mirrors, and global settings
• Run pip check to verify your environment has no broken dependency chains
• Keep pip updated with python -m pip install --upgrade pip
• Use python -m build and twine for publishing packages to PyPI
• Consider using Poetry or Pipenv for complete dependency management workflows