Python Packaging

Comprehensive guide to creating, structuring, and distributing Python packages using modern packaging tools, pyproject.toml, and publishing to PyPI.

When to Use This Skill

Creating Python libraries for distribution

Building command-line tools with entry points

Publishing packages to PyPI or private repositories

Setting up Python project structure

Creating installable packages with dependencies

Building wheels and source distributions

Versioning and releasing Python packages

Creating namespace packages

Implementing package metadata and classifiers

Core Concepts

1. Package Structure

Source layout

:

src/package_name/

(recommended)

Flat layout

:

package_name/

(simpler but less flexible)

Package metadata

pyproject.toml, setup.py, or setup.cfg

Distribution formats

wheel (.whl) and source distribution (.tar.gz)

2. Modern Packaging Standards

PEP 517/518

Build system requirements

PEP 621

Metadata in pyproject.toml

PEP 660

Editable installs

pyproject.toml

Single source of configuration

3. Build Backends

setuptools

Traditional, widely used

hatchling

Modern, opinionated

flit

Lightweight, for pure Python

poetry

Dependency management + packaging

4. Distribution

PyPI

Python Package Index (public)

TestPyPI

Testing before production

Private repositories

JFrog, AWS CodeArtifact, etc. Quick Start Minimal Package Structure my-package/ ├── pyproject.toml ├── README.md ├── LICENSE ├── src/ │ └── my_package/ │ ├── init.py │ └── module.py └── tests/ └── test_module.py Minimal pyproject.toml [ build-system ] requires = [ "setuptools>=61.0" ] build-backend = "setuptools.build_meta" [ project ] name = "my-package" version = "0.1.0" description = "A short description" authors = [ { name = "Your Name" , email = "you@example.com" } ] readme = "README.md" requires-python = ">=3.8" dependencies = [ "requests>=2.28.0" , ] [ project.optional-dependencies ] dev = [ "pytest>=7.0" , "black>=22.0" , ] Package Structure Patterns Pattern 1: Source Layout (Recommended) my-package/ ├── pyproject.toml ├── README.md ├── LICENSE ├── .gitignore ├── src/ │ └── my_package/ │ ├── init.py │ ├── core.py │ ├── utils.py │ └── py.typed # For type hints ├── tests/ │ ├── init.py │ ├── test_core.py │ └── test_utils.py └── docs/ └── index.md Advantages: Prevents accidentally importing from source Cleaner test imports Better isolation pyproject.toml for source layout: [ tool.setuptools.packages.find ] where = [ "src" ] Pattern 2: Flat Layout my-package/ ├── pyproject.toml ├── README.md ├── my_package/ │ ├── init.py │ └── module.py └── tests/ └── test_module.py Simpler but: Can import package without installing Less professional for libraries Pattern 3: Multi-Package Project project/ ├── pyproject.toml ├── packages/ │ ├── package-a/ │ │ └── src/ │ │ └── package_a/ │ └── package-b/ │ └── src/ │ └── package_b/ └── tests/ Complete pyproject.toml Examples Pattern 4: Full-Featured pyproject.toml [ build-system ] requires = [ "setuptools>=61.0" , "wheel" ] build-backend = "setuptools.build_meta" [ project ] name = "my-awesome-package" version = "1.0.0" description = "An awesome Python package" readme = "README.md" requires-python = ">=3.8" license = { text = "MIT" } authors = [ { name = "Your Name" , email = "you@example.com" } , ] maintainers = [ { name = "Maintainer Name" , email = "maintainer@example.com" } , ] keywords = [ "example" , "package" , "awesome" ] classifiers = [ "Development Status :: 4 - Beta" , "Intended Audience :: Developers" , "License :: OSI Approved :: MIT License" , "Programming Language :: Python :: 3" , "Programming Language :: Python :: 3.8" , "Programming Language :: Python :: 3.9" , "Programming Language :: Python :: 3.10" , "Programming Language :: Python :: 3.11" , "Programming Language :: Python :: 3.12" , ] dependencies = [ "requests>=2.28.0,<3.0.0" , "click>=8.0.0" , "pydantic>=2.0.0" , ] [ project.optional-dependencies ] dev = [ "pytest>=7.0.0" , "pytest-cov>=4.0.0" , "black>=23.0.0" , "ruff>=0.1.0" , "mypy>=1.0.0" , ] docs = [ "sphinx>=5.0.0" , "sphinx-rtd-theme>=1.0.0" , ] all = [ "my-awesome-package[dev,docs]" , ] [ project.urls ] Homepage = "https://github.com/username/my-awesome-package" Documentation = "https://my-awesome-package.readthedocs.io" Repository = "https://github.com/username/my-awesome-package" "Bug Tracker" = "https://github.com/username/my-awesome-package/issues" Changelog = "https://github.com/username/my-awesome-package/blob/main/CHANGELOG.md" [ project.scripts ] my-cli = "my_package.cli:main" awesome-tool = "my_package.tools:run" [ project.entry-points."my_package.plugins" ] plugin1 = "my_package.plugins:plugin1" [ tool.setuptools ] package-dir = { "" = "src" } zip-safe = false [ tool.setuptools.packages.find ] where = [ "src" ] include = [ "my_package" ] exclude = [ "tests" ] [ tool.setuptools.package-data ] my_package = [ "py.typed" , ".pyi" , "data/.json" ]

Black configuration

[ tool.black ] line-length = 100 target-version = [ "py38" , "py39" , "py310" , "py311" ] include = '.pyi?$'

Ruff configuration

[ tool.ruff ] line-length = 100 target-version = "py38" [ tool.ruff.lint ] select = [ "E" , "F" , "I" , "N" , "W" , "UP" ]

MyPy configuration

[ tool.mypy ] python_version = "3.8" warn_return_any = true warn_unused_configs = true disallow_untyped_defs = true

Pytest configuration

[ tool.pytest.ini_options ] testpaths = [ "tests" ] python_files = [ "test_*.py" ] addopts = "-v --cov=my_package --cov-report=term-missing"

Coverage configuration

[ tool.coverage.run ] source = [ "src" ] omit = [ "/tests/" ] [ tool.coverage.report ] exclude_lines = [ "pragma: no cover" , "def repr" , "raise AssertionError" , "raise NotImplementedError" , ] Pattern 5: Dynamic Versioning [ build-system ] requires = [ "setuptools>=61.0" , "setuptools-scm>=8.0" ] build-backend = "setuptools.build_meta" [ project ] name = "my-package" dynamic = [ "version" ] description = "Package with dynamic version" [ tool.setuptools.dynamic ] version = { attr = "my_package.version" }

Or use setuptools-scm for git-based versioning

[ tool.setuptools_scm ] write_to = "src/my_package/_version.py" In init .py:

src/my_package/init.py

version

"1.0.0"

Or with setuptools-scm

from importlib . metadata import version version = version ( "my-package" ) Command-Line Interface (CLI) Patterns Pattern 6: CLI with Click

src/my_package/cli.py

import click @click . group ( ) @click . version_option ( ) def cli ( ) : """My awesome CLI tool.""" pass @cli . command ( ) @click . argument ( "name" ) @click . option ( "--greeting" , default = "Hello" , help = "Greeting to use" ) def greet ( name : str , greeting : str ) : """Greet someone.""" click . echo ( f" { greeting } , { name } !" ) @cli . command ( ) @click . option ( "--count" , default = 1 , help = "Number of times to repeat" ) def repeat ( count : int ) : """Repeat a message.""" for i in range ( count ) : click . echo ( f"Message { i + 1 } " ) def main ( ) : """Entry point for CLI.""" cli ( ) if name == "main" : main ( ) Register in pyproject.toml: [ project.scripts ] my-tool = "my_package.cli:main" Usage: pip install -e . my-tool greet World my-tool greet Alice --greeting = "Hi" my-tool repeat --count = 3 Pattern 7: CLI with argparse

src/my_package/cli.py

import argparse import sys def main ( ) : """Main CLI entry point.""" parser = argparse . ArgumentParser ( description = "My awesome tool" , prog = "my-tool" ) parser . add_argument ( "--version" , action = "version" , version = "%(prog)s 1.0.0" ) subparsers = parser . add_subparsers ( dest = "command" , help = "Commands" )

Add subcommand

process_parser

subparsers . add_parser ( "process" , help = "Process data" ) process_parser . add_argument ( "input_file" , help = "Input file path" ) process_parser . add_argument ( "--output" , "-o" , default = "output.txt" , help = "Output file path" ) args = parser . parse_args ( ) if args . command == "process" : process_data ( args . input_file , args . output ) else : parser . print_help ( ) sys . exit ( 1 ) def process_data ( input_file : str , output_file : str ) : """Process data from input to output.""" print ( f"Processing { input_file } -> { output_file } " ) if name == "main" : main ( ) Building and Publishing Pattern 8: Build Package Locally

Install build tools

pip install build twine

Build distribution

python -m build

This creates:

dist/

my-package-1.0.0.tar.gz (source distribution)

my_package-1.0.0-py3-none-any.whl (wheel)

Check the distribution

twine check dist/* Pattern 9: Publishing to PyPI

Install publishing tools

pip install twine

Test on TestPyPI first

twine upload --repository testpypi dist/*

Install from TestPyPI to test

pip install --index-url https://test.pypi.org/simple/ my-package

If all good, publish to PyPI

twine upload dist/* Using API tokens (recommended):

Create ~/.pypirc

[ distutils ] index-servers = pypi testpypi [ pypi ] username = token password = pypi- .. .your-token .. . [ testpypi ] username = token password = pypi- .. .your-test-token .. . Pattern 10: Automated Publishing with GitHub Actions

.github/workflows/publish.yml

name : Publish to PyPI on : release : types : [ created ] jobs : publish : runs-on : ubuntu - latest steps : - uses : actions/checkout@v3 - name : Set up Python uses : actions/setup - python@v4 with : python-version : "3.11" - name : Install dependencies run : | pip install build twine - name : Build package run : python - m build - name : Check package run : twine check dist/ - name : Publish to PyPI env : TWINE_USERNAME : token TWINE_PASSWORD : $ { { secrets.PYPI_API_TOKEN } } run : twine upload dist/ For advanced patterns including data files, namespace packages, C extensions, version management, testing installation, documentation templates, and distribution workflows, see references/advanced-patterns.md