- Tue 17 September 2024
- python
- Hamish Croser
- #python, #development, #packaging
Poetry is a command-line dependency management and packaging tool for Python. It offers commands for creating a new project, adding and removing dependencies, and managing package metadata for publishing.
Getting started
Poetry requires Python 3.8+. It is multi-platform and the goal is to make it work equally well on Linux, macOS and Windows.
To install poetry, use python3 -m pip install poetry
in the command line. Note: the environment in which Poetry is installed should be distinct from the project environment to ensure there are no conflicts between Poetry's dependencies and those of the package.
Once installed, Poetry offers two commands for getting started: poetry new
and poetry init
:
poetry new <package-name>
will create a new project directory with a default layout. Use this command if you don't have an existing project.poetry init
will interactively create a pyproject.toml file. Use this command if you have an existing project.
The pyproject.toml file
The pyproject.toml file is the human-readable source of truth for your package metadata, dependencies, and compatible Python versions.
The poetry init
command will prompt you for the required information for your pyproject.toml file, and it will produce something like this:
[tool.poetry]
name = "example"
version = "0.1.0"
description = "An example project"
authors = ["Hamish Croser <h.croser1@gmail.com>"]
license = "MIT"
readme = "README.md"
[tool.poetry.dependencies]
python = "^3.10"
requests = "^2.32.3"
tqdm = "^4.66.5"
chardet = "^5.2.0"
[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"
Managing dependencies
To add or remove dependencies from your package, you can either directly edit the pyproject.toml file or use poetry add <dependency>
and poetry remove <dependency>
.
While the direct dependencies are stored in the pyproject.toml file, your package's downstream dependencies will be stored in a file called poetry.lock. This file is not intended to be edited manually and can be generated by calling poetry lock
(or poetry install
if no poetry.lock exists yet). The purpose of this file is to set an exact version of all dependencies to ensure the exact environment can be recreated.
The poetry install
command will install the dependencies specified in the poetry.lock file if it exists, or it will generate the poetry.lock file and install the dependencies if it doesn't exist.
Both the poetry lock
and poetry install
commands will resolve dependencies according to the constraints specified in the pyproject.toml file. If the dependency resolution fails, consider widening the dependency version constraints.
Publishing a package
File structure
Poetry assumes your package contains a package with the same name as tool.poetry.name located in the root of your project. If this is not the case, populate tool.poetry.packages to specify your packages and their locations. For example if your package name is 'example-project' but your source files are located in example_project, add packages = [{include = "example_project"}]
under the [tool.poetry]
section of your pyproject.toml file to ensure the source files are found and added to the package.
PyPI credentials
By default, Poetry publishes packages to PyPI (Python Package Index) which will allow you to use pip install <package_name>
in other environments.
In order to publish to PyPI, you will first need to create a user account here: https://pypi.org/account/register/ Once you have made an account, navigate to your account settings, scroll down to the 'API tokens' section, and add an API token.
Once you have obtained your PyPI API token, call the following in your Python environment, replacing '
poetry config pypi-token.pypi <TOKEN>
Publishing the package
Once your PyPI credentials are set, you can build and publish your package.
To build your package, use:
poetry build
To publish your package, use:
poetry publish
To combine the two steps into one, use:
poetry publish --build
Versioning
Optionally, you can bump the version of your package before publishing using poetry version <BUMP_RULE>
where
Dry-run and testing
Publishing a package can be risky, and it's a good idea to test things out before commiting.
When using the publish
command, you can add the --dry-run
argument like so: poetry publish --dry-run
. When called, this will perform all actions except uploading the package.
In addition to the above, before publishing to the main PyPI repository you can publish to PyPI's test repository: https://test.pypi.org/ To do this, first obtain a TestPyPI API token as described earlier under PyPI credentials. Then enter the following commands:
poetry config repositories.test-pypi https://test.pypi.org/legacy/
poetry config pypi-token.test-pypi <TEST_TOKEN>
replacing
The following will publish to TestPyPI:
poetry publish -r test-pypi
The poetry publish
command will still publish to the standard PyPI.
Further information
If you plan on using Poetry for packaging or just as a dependency manager for your projects, reference the documentation as much as possible: https://python-poetry.org/docs/ The documentation is very well laid out and contains much more information than can fit in this guide.