Python dependency management has evolved beyond pip install. Modern tools provide lock files, dependency resolution, and reproducible builds. This guide compares the major tools and helps you choose the right one.
The Dependency Problem
Simple dependency declarations can lead to complex problems:
┌─────────────────────────────────────────────────────────────┐
│ The Dependency Diamond │
├─────────────────────────────────────────────────────────────┤
│ │
│ your-project │
│ / \ │
│ / \ │
│ package-A package-B │
│ needs X>=2.0 needs X<2.5 │
│ \ / │
│ \ / │
│ package-X │
│ (which version?) │
│ │
│ Valid range: X>=2.0,<2.5 → Could be 2.0, 2.1, 2.2, etc. │
│ Without locking: Different installs get different versions │
│ │
└─────────────────────────────────────────────────────────────┘
Lock files solve this by recording exact versions.
Tool Comparison
| Feature | pip-tools | Poetry | Pipenv | PDM |
|---|---|---|---|---|
| Lock file | Yes (txt) | Yes (poetry.lock) | Yes (Pipfile.lock) | Yes (pdm.lock) |
| Dependency resolution | Good | Excellent | Good | Excellent |
| Virtual env management | No | Yes | Yes | Yes |
| Build & publish | No | Yes | No | Yes |
| Config file | requirements.in | pyproject.toml | Pipfile | pyproject.toml |
| Learning curve | Low | Medium | Medium | Medium |
| Popularity | High | Very High | Medium | Growing |
pip-tools: Simple and Effective
pip-tools adds lock files to the standard pip workflow. It's minimal, fast, and fits existing projects.
Installation
pip install pip-tools
Basic Workflow
- Write requirements.in (what you want):
# requirements.in
django>=4.0
celery>=5.0
redis
- Compile to requirements.txt (exact versions):
pip-compile requirements.in
Output:
# requirements.txt
# This file is autogenerated by pip-compile with Python 3.11
#
django==4.2.8
# via -r requirements.in
celery==5.3.4
# via -r requirements.in
redis==5.0.1
# via -r requirements.in
amqp==5.2.0
# via kombu
billiard==4.2.0
# via celery
# ... (all transitive dependencies)
- Install pinned versions:
pip-sync requirements.txt
# Or: pip install -r requirements.txt
Separating Dev Dependencies
# requirements.in
django>=4.0
celery>=5.0
# requirements-dev.in
-c requirements.txt
pytest>=7.0
black
mypy
The -c requirements.txt ensures dev deps use the same versions as production.
# Compile both
pip-compile requirements.in
pip-compile requirements-dev.in
# Install for development
pip-sync requirements.txt requirements-dev.txt
Upgrading Dependencies
# Upgrade all packages
pip-compile --upgrade requirements.in
# Upgrade specific package
pip-compile --upgrade-package django requirements.in
# Upgrade with constraints
pip-compile --upgrade-package "django<5.0" requirements.in
pip-sync vs pip install
# pip install: adds packages, doesn't remove old ones
pip install -r requirements.txt
# pip-sync: matches environment exactly to requirements
pip-sync requirements.txt
# Removes packages not in requirements.txt!
pip-tools Best Practices
# Add hashes for security
pip-compile --generate-hashes requirements.in
# Output:
django==4.2.8 \
--hash=sha256:abc123... \
--hash=sha256:def456...
# Resolve for specific Python version
pip-compile --python-version 3.11 requirements.in
# Include annotations showing which package needs what
pip-compile --annotate requirements.in
Poetry: All-in-One Solution
Poetry handles dependencies, environments, building, and publishing in one tool.
Installation
# Official installer (recommended)
curl -sSL https://install.python-poetry.org | python3 -
# Or via pipx
pipx install poetry
Creating a New Project
# New project with src layout
poetry new my-package
# Or initialize in existing directory
cd existing-project
poetry init
pyproject.toml Structure
[tool.poetry]
name = "my-package"
version = "1.0.0"
description = "My awesome package"
authors = ["Your Name <[email protected]>"]
readme = "README.md"
[tool.poetry.dependencies]
python = "^3.9"
django = "^4.0"
celery = "^5.0"
[tool.poetry.group.dev.dependencies]
pytest = "^7.0"
black = "^23.0"
mypy = "^1.0"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
Managing Dependencies
# Add dependency
poetry add django
# Add with version constraint
poetry add "django>=4.0,<5.0"
# Add dev dependency
poetry add --group dev pytest
# Remove dependency
poetry remove celery
# Update all dependencies
poetry update
# Update specific package
poetry update django
Lock File (poetry.lock)
Poetry automatically maintains poetry.lock:
# Install from lock file (exact versions)
poetry install
# Update lock file without installing
poetry lock
# Install without dev dependencies
poetry install --without dev
# Show dependency tree
poetry show --tree
Virtual Environment Management
Poetry creates and manages environments automatically:
# Poetry creates .venv in project (or central location)
poetry install
# Run command in environment
poetry run python script.py
poetry run pytest
# Activate environment shell
poetry shell
# Show environment info
poetry env info
# Use existing environment
poetry env use /path/to/python
Building and Publishing
# Build wheel and sdist
poetry build
# Publish to PyPI
poetry publish
# Publish to Test PyPI
poetry publish -r testpypi
# Configure PyPI credentials
poetry config pypi-token.pypi your-token
Dependency Groups
[tool.poetry.dependencies]
python = "^3.9"
django = "^4.0"
[tool.poetry.group.dev.dependencies]
pytest = "^7.0"
black = "^23.0"
[tool.poetry.group.docs.dependencies]
sphinx = "^7.0"
sphinx-rtd-theme = "^1.0"
[tool.poetry.group.test.dependencies]
coverage = "^7.0"
pytest-cov = "^4.0"
# Install specific groups
poetry install --with docs
# Install without specific groups
poetry install --without test,docs
# Install only specific groups
poetry install --only dev
Pipenv: Application-Focused
Pipenv combines pip and virtual environments with a Pipfile format.
Installation
pip install pipenv
Creating a Project
cd my-project
pipenv install django celery
# Creates Pipfile and Pipfile.lock
Pipfile Format
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"
[packages]
django = ">=4.0"
celery = ">=5.0"
[dev-packages]
pytest = "*"
black = "*"
[requires]
python_version = "3.11"
Basic Commands
# Install from Pipfile.lock
pipenv install
# Install including dev packages
pipenv install --dev
# Add package
pipenv install requests
# Add dev package
pipenv install pytest --dev
# Update all packages
pipenv update
# Activate environment
pipenv shell
# Run command in environment
pipenv run python script.py
Lock and Sync
# Generate/update lock file
pipenv lock
# Install exactly what's in lock file
pipenv sync
# Check for security vulnerabilities
pipenv check
Export to requirements.txt
# For deployment to systems that need requirements.txt
pipenv requirements > requirements.txt
pipenv requirements --dev > requirements-dev.txt
PDM: Modern Standards
PDM follows the latest Python packaging standards (PEP 582, PEP 621).
Installation
# Via pipx (recommended)
pipx install pdm
# Or via pip
pip install pdm
Creating a Project
pdm init
# Answer prompts about project metadata
pyproject.toml (PEP 621)
[project]
name = "my-package"
version = "1.0.0"
description = "My package"
authors = [{name = "Your Name", email = "[email protected]"}]
dependencies = [
"django>=4.0",
"celery>=5.0",
]
requires-python = ">=3.9"
[project.optional-dependencies]
dev = ["pytest", "black"]
[tool.pdm]
distribution = true
[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"
Basic Commands
# Add dependency
pdm add django
# Add dev dependency
pdm add -dG dev pytest
# Install from lock file
pdm install
# Update packages
pdm update
# Run in environment
pdm run python script.py
PEP 582 Mode (Experimental)
PDM can use __pypackages__ instead of virtual environments:
pdm config python.use_venv false
# Packages install to __pypackages__/
# No activation needed—Python finds them automatically
Migration Guide
From requirements.txt to pip-tools
# Your existing requirements.txt becomes requirements.in
mv requirements.txt requirements.in
# Generate locked requirements.txt
pip-compile requirements.in
# Install
pip-sync requirements.txt
From pip to Poetry
# Initialize Poetry in existing project
poetry init
# Import existing requirements
cat requirements.txt | xargs poetry add
# Generate lock file
poetry lock
From Pipenv to Poetry
# Export Pipenv dependencies
pipenv requirements > requirements.txt
pipenv requirements --dev > requirements-dev.txt
# Initialize Poetry
poetry init
# Import dependencies
cat requirements.txt | xargs poetry add
cat requirements-dev.txt | xargs poetry add --group dev
# Remove Pipenv files
rm Pipfile Pipfile.lock
Decision Guide
Use pip-tools When:
- You want minimal changes to existing workflow
- Your team already knows pip well
- You don't need build/publish features
- You prefer standard requirements.txt format
Use Poetry When:
- Starting a new project
- Building a library for PyPI
- You want one tool for everything
- You need dependency groups
- You want better dependency resolution
Use Pipenv When:
- Building an application (not a library)
- You prefer Pipfile format
- Your team already uses it
Use PDM When:
- You want cutting-edge standards compliance
- You're interested in PEP 582
- You want Poetry-like features with standard pyproject.toml
Best Practices
1. Always Use Lock Files
# pip-tools
pip-compile requirements.in
git add requirements.txt
# Poetry
poetry lock
git add poetry.lock
# Commit lock files to version control!
2. Separate Production and Dev Dependencies
# pip-tools
requirements.in # Production
requirements-dev.in # Development
# Poetry
[tool.poetry.dependencies] # Production
[tool.poetry.group.dev] # Development
3. Pin Python Version
# Poetry
[tool.poetry.dependencies]
python = "^3.11"
# PDM/PEP 621
[project]
requires-python = ">=3.11"
4. Regular Updates
# Check for outdated packages
pip-tools: pip-compile --upgrade --dry-run
poetry: poetry show --outdated
pipenv: pipenv update --outdated
# Update regularly and test
5. Use Hashes for Security
# pip-tools
pip-compile --generate-hashes requirements.in
# Poetry (default in lock file)
poetry lock
Command Reference
pip-tools
| Command | Description |
|---|---|
pip-compile | Generate requirements.txt from .in |
pip-compile --upgrade | Upgrade all packages |
pip-sync | Install exactly what's in requirements |
pip-compile --generate-hashes | Add security hashes |
Poetry
| Command | Description |
|---|---|
poetry add PKG | Add dependency |
poetry remove PKG | Remove dependency |
poetry install | Install from lock |
poetry update | Update dependencies |
poetry lock | Regenerate lock file |
poetry build | Build wheel/sdist |
poetry publish | Publish to PyPI |
Pipenv
| Command | Description |
|---|---|
pipenv install PKG | Add dependency |
pipenv install --dev PKG | Add dev dependency |
pipenv lock | Generate lock file |
pipenv sync | Install from lock |
pipenv shell | Activate environment |
Next Steps
- Learn about virtual environments in our Virtual Environments Guide
- See our requirements.txt Guide for basic dependency files
- Explore the Python Packaging Complete Guide for the full ecosystem
For more Python development guides, explore our complete Python packaging series.