Python packaging has evolved significantly, with modern tools like pyproject.toml replacing the fragmented landscape of setup.py, setup.cfg, and requirements.txt. This guide provides a comprehensive overview of the Python packaging ecosystem and connects you to detailed tutorials for each topic.
Why Packaging Matters
Poor packaging practices lead to common problems:
| Problem | Symptom | Solution |
|---|---|---|
| "Works on my machine" | Different behavior across environments | Virtual environments + pinned dependencies |
| Dependency conflicts | Package A needs X>=2.0, B needs X<2.0 | Proper dependency specification |
| Broken installations | pip install fails | Correct package structure + metadata |
| Security vulnerabilities | Outdated dependencies | Automated updates + version policies |
| Lost code | Can't reproduce old version | Proper versioning + PyPI publishing |
The Python Packaging Landscape
┌─────────────────────────────────────────────────────────────────────┐
│ Python Packaging Ecosystem │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ CONFIGURE │ │ BUILD │ │ DISTRIBUTE │ │
│ │ │ │ │ │ │ │
│ │ pyproject.toml │───▶│ wheel (.whl) │───▶│ PyPI │ │
│ │ requirements.txt│ │ sdist (.tar.gz)│ │ Private repos │ │
│ │ setup.py (old) │ │ │ │ GitHub releases│ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
│ │ │ │ │
│ ▼ ▼ ▼ │
│ ┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐ │
│ │ Build Backends │ │ Build Tools │ │ Install Tools │ │
│ │ │ │ │ │ │ │
│ │ - hatchling │ │ - build │ │ - pip │ │
│ │ - setuptools │ │ - pip wheel │ │ - pipx │ │
│ │ - flit-core │ │ - twine │ │ - uv │ │
│ │ - poetry-core │ │ │ │ │ │
│ └─────────────────┘ └─────────────────┘ └─────────────────┘ │
│ │
│ ┌─────────────────────────────────────────────────────────────────┐│
│ │ Environment Management ││
│ │ venv | virtualenv | conda | pyenv | poetry | pipenv ││
│ └─────────────────────────────────────────────────────────────────┘│
│ │
└─────────────────────────────────────────────────────────────────────┘
Quick Start Decision Guide
Which Configuration File?
Starting a new project?
│
├── Library (publish to PyPI)
│ └── Use: pyproject.toml (primary)
│ + requirements-dev.txt (dev dependencies)
│
├── Application (deploy to server)
│ └── Use: pyproject.toml (metadata + deps)
│ + requirements.txt (pinned versions)
│
└── Script or notebook
└── Use: requirements.txt (simple dependency list)
For detailed guidance, see our pyproject.toml vs requirements.txt vs setup.py comparison.
Which Dependency Tool?
Project Type?
│
├── Simple project, familiar with pip
│ └── Use: pip + pip-tools
│
├── Want all-in-one solution
│ └── Use: Poetry
│
├── Data science / scientific computing
│ └── Use: conda
│
└── Enterprise with strict requirements
└── Use: pip-tools or Poetry with lock files
Core Topics
1. Declaring Dependencies
The foundation of Python packaging is declaring what your project needs.
requirements.txt provides simple dependency lists:
requests>=2.28.0
click>=8.0.0
pydantic>=2.0
For a complete guide to requirements.txt syntax, version specifiers, and best practices, see our requirements.txt Complete Guide.
pyproject.toml is the modern standard:
[project]
dependencies = [
"requests>=2.28.0",
"click>=8.0.0",
"pydantic>=2.0",
]
[project.optional-dependencies]
dev = ["pytest", "black", "mypy"]
For comprehensive pyproject.toml configuration, see our pyproject.toml Complete Guide.
2. Environment Isolation
Virtual environments prevent dependency conflicts between projects.
# Create environment
python -m venv .venv
# Activate (macOS/Linux)
source .venv/bin/activate
# Activate (Windows)
.venv\Scripts\activate
# Install dependencies
pip install -r requirements.txt
For in-depth coverage of venv, virtualenv, conda, and pyenv, see our Virtual Environments Guide.
3. Dependency Resolution
Lock files capture exact versions for reproducible builds:
# With pip-tools
pip-compile requirements.in -o requirements.txt
# With Poetry
poetry lock
# With Pipenv
pipenv lock
Compare dependency management tools in our Dependency Management Tools Guide.
4. Building Packages
Transform your source into distributable packages:
# Install build tool
pip install build
# Build wheel and sdist
python -m build
# Result:
# dist/my_package-1.0.0-py3-none-any.whl
# dist/my_package-1.0.0.tar.gz
5. Publishing to PyPI
Share your package with the world:
# Upload to PyPI
twine upload dist/*
# Or use trusted publishing (no tokens needed)
# Configure in GitHub Actions workflow
For complete publishing instructions including trusted publishing setup, see our Publishing to PyPI Guide.
6. Versioning
Communicate changes through version numbers:
# Semantic versioning
version = "1.2.3" # MAJOR.MINOR.PATCH
# Calendar versioning
version = "2026.01.15" # YYYY.MM.DD
Learn versioning strategies and changelog management in our Package Versioning Guide.
7. Testing Packages
Ensure your package works across Python versions:
# Run tests with tox
tox
# Or nox (Python-native)
nox
For multi-version testing strategies, see our Testing Python Packages Guide.
8. CLI Tools
Create installable command-line tools:
# pyproject.toml
[project.scripts]
my-tool = "my_package.cli:main"
Build CLI interfaces with Click or Typer. See our Building CLI Tools Guide.
9. Documentation
Generate beautiful docs from your code:
# Sphinx
sphinx-quickstart docs
# MkDocs
mkdocs new .
Learn documentation best practices in our Package Documentation Guide.
10. Private Repositories
Host packages internally for your organization:
# Configure pip for private repo
pip install --extra-index-url https://pypi.company.com/simple/ private-package
Set up private repositories with our Private Package Repositories Guide.
Recommended Learning Path
Beginner Level
Start with understanding how dependencies work:
1. requirements.txt Guide → Understand basic dependency management
2. Virtual Environments Guide → Isolate project dependencies
3. pyproject.toml Guide → Learn modern configuration
Intermediate Level
Build and share your own packages:
4. Comparison Guide → Choose the right approach
5. Dependency Tools Guide → Advanced dependency management
6. Versioning Guide → Version your packages properly
7. Publishing Guide → Share on PyPI
Advanced Level
Professional package development:
8. Testing Guide → Multi-version testing
9. CLI Tools Guide → Build command-line tools
10. Documentation Guide → Generate professional docs
11. Private Repos Guide → Enterprise distribution
Quick Reference: All Guides
| Guide | Topic | Best For |
|---|---|---|
| requirements.txt Guide | Basic dependency files | Beginners, simple projects |
| pyproject.toml Guide | Modern configuration | All projects |
| Comparison Guide | Choosing config files | Decision making |
| Virtual Environments Guide | venv, conda, pyenv | Environment setup |
| Dependency Tools Guide | pip-tools, Poetry, Pipenv | Dependency management |
| Versioning Guide | SemVer, CalVer, changelogs | Release management |
| Publishing Guide | PyPI, twine, CI/CD | Distribution |
| Testing Guide | pytest, tox, nox | Quality assurance |
| CLI Tools Guide | Click, Typer, argparse | Command-line tools |
| Documentation Guide | Sphinx, MkDocs | Documentation |
| Private Repos Guide | devpi, CodeArtifact | Enterprise |
Modern Package Structure
The recommended structure for a Python package in 2026:
my-package/
├── src/
│ └── my_package/
│ ├── __init__.py
│ ├── core.py
│ └── cli.py
├── tests/
│ ├── __init__.py
│ ├── test_core.py
│ └── test_cli.py
├── docs/
│ ├── index.md
│ └── api.md
├── .github/
│ └── workflows/
│ ├── test.yml
│ └── publish.yml
├── pyproject.toml
├── README.md
├── CHANGELOG.md
└── LICENSE
Minimal pyproject.toml
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"
[project]
name = "my-package"
version = "1.0.0"
description = "A useful Python package"
readme = "README.md"
license = {text = "MIT"}
requires-python = ">=3.9"
authors = [{name = "Your Name", email = "[email protected]"}]
dependencies = [
"requests>=2.28",
]
[project.optional-dependencies]
dev = ["pytest", "black", "mypy"]
[project.urls]
Homepage = "https://github.com/user/my-package"
[project.scripts]
my-command = "my_package.cli:main"
Key Takeaways
-
Use pyproject.toml as your single source of configuration for all new projects.
-
Always use virtual environments to isolate project dependencies.
-
Lock your dependencies with pip-tools, Poetry, or similar for reproducible builds.
-
Follow semantic versioning for libraries, calendar versioning for applications.
-
Test across Python versions with tox or nox before releasing.
-
Automate publishing with GitHub Actions and trusted publishing.
-
Document your package with Sphinx or MkDocs from the start.
Next Steps
-
Start with our pyproject.toml Complete Guide to set up your project
-
Set up proper environments with our Virtual Environments Guide
-
When ready to share, follow our Publishing to PyPI Guide
Additional Resources
- Python Packaging User Guide - Official documentation
- PyPI - Python Package Index
- Test PyPI - Testing package uploads
- PEP 517 - Build system specification
- PEP 621 - Project metadata standard
For more Python development guides, explore our complete Python packaging series.