SIH Tech Tidbits

Useful tips, libraries and tools from the Sydney Informatics Hub team

Publishing Packages using Poetry


Poetry is a command-line dependency management and packaging tool for Python. It offers commands for creating a new project, adding and removing dependencies, and managing package metadata for publishing.

Getting started

Poetry requires Python 3.8+. It is multi-platform and the goal is to make it work equally well on Linux, macOS and Windows.

To install poetry, use python3 -m pip install poetry in the command line. Note: the environment in which Poetry is installed should be distinct from the project environment to ensure there are no conflicts between Poetry's dependencies and those of the package.

Once installed, Poetry offers two commands for getting started: poetry new and poetry init:

  • poetry new <package-name> will create a new project directory with a default layout. Use this command if you don't have an existing project.
  • poetry init will interactively create a pyproject.toml file. Use this command if you have an existing project.

The pyproject.toml file

The pyproject.toml file is the human-readable source of truth for your package metadata, dependencies, and compatible Python versions. The poetry init command will prompt you for the required information for your pyproject.toml file, and it will produce something like this:

[tool.poetry]
name = "example"
version = "0.1.0"
description = "An example project"
authors = ["Hamish Croser <h.croser1@gmail.com>"]
license = "MIT"
readme = "README.md"

[tool.poetry.dependencies]
python = "^3.10"
requests = "^2.32.3"
tqdm = "^4.66.5"
chardet = "^5.2.0"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Managing dependencies

To add or remove dependencies from your package, you can either directly edit the pyproject.toml file or use poetry add <dependency> and poetry remove <dependency>.

While the direct dependencies are stored in the pyproject.toml file, your package's downstream dependencies will be stored in a file called poetry.lock. This file is not intended to be edited manually and can be generated by calling poetry lock (or poetry install if no poetry.lock exists yet). The purpose of this file is to set an exact version of all dependencies to ensure the exact environment can be recreated.

The poetry install command will install the dependencies specified in the poetry.lock file if it exists, or it will generate the poetry.lock file and install the dependencies if it doesn't exist.

Both the poetry lock and poetry install commands will resolve dependencies according to the constraints specified in the pyproject.toml file. If the dependency resolution fails, consider widening the dependency version constraints.

Publishing a package

File structure

Poetry assumes your package contains a package with the same name as tool.poetry.name located in the root of your project. If this is not the case, populate tool.poetry.packages to specify your packages and their locations. For example if your package name is 'example-project' but your source files are located in example_project, add packages = [{include = "example_project"}] under the [tool.poetry] section of your pyproject.toml file to ensure the source files are found and added to the package.

PyPI credentials

By default, Poetry publishes packages to PyPI (Python Package Index) which will allow you to use pip install <package_name> in other environments.

In order to publish to PyPI, you will first need to create a user account here: https://pypi.org/account/register/ Once you have made an account, navigate to your account settings, scroll down to the 'API tokens' section, and add an API token.

Once you have obtained your PyPI API token, call the following in your Python environment, replacing '' with your PyPI API token:

poetry config pypi-token.pypi <TOKEN>

Publishing the package

Once your PyPI credentials are set, you can build and publish your package.

To build your package, use:

poetry build

To publish your package, use:

poetry publish

To combine the two steps into one, use:

poetry publish --build

Versioning

Optionally, you can bump the version of your package before publishing using poetry version <BUMP_RULE> where may be one of patch, minor, major, prepatch, preminor, premajor, prerelease. More details can be found here: https://python-poetry.org/docs/cli/#version

Dry-run and testing

Publishing a package can be risky, and it's a good idea to test things out before commiting.

When using the publish command, you can add the --dry-run argument like so: poetry publish --dry-run. When called, this will perform all actions except uploading the package.

In addition to the above, before publishing to the main PyPI repository you can publish to PyPI's test repository: https://test.pypi.org/ To do this, first obtain a TestPyPI API token as described earlier under PyPI credentials. Then enter the following commands:

poetry config repositories.test-pypi https://test.pypi.org/legacy/

poetry config pypi-token.test-pypi <TEST_TOKEN> replacing with your TestPyPI API token

The following will publish to TestPyPI:

poetry publish -r test-pypi

The poetry publish command will still publish to the standard PyPI.

Further information

If you plan on using Poetry for packaging or just as a dependency manager for your projects, reference the documentation as much as possible: https://python-poetry.org/docs/ The documentation is very well laid out and contains much more information than can fit in this guide.