Posted on Aug 10, 2024

[GitHub Actions] Automate Build and Deployment of Your Python Package to PyPI and GitHub Releases 🐍📦

When releasing a custom library, it can be tedious to manually build it locally, upload it to TestPyPI for verification, then upload it to PyPI, and finally tag and release it…

So, I decided to automate the deployment using GitHub Actions.

Objective of This Article

When you push a tag to the remote repository, this setup will automatically upload your package to PyPI and create a GitHub release.

After you push a tag from your local environment…

git tag vX.X.X git push origin vX.X.X

The following steps will be automatically executed in order:

Upload to TestPyPI
Upload to PyPI
Create a release on GitHub and distribute the source code

We will create a workflow as shown in the image below.

Steps

Preparation

Note: Instead of using setup.py as the packaging tool configuration file, we will use pyproject.toml. According to PEP-518, pyproject.toml is recommended for modern Python projects.

Prepare your Python package and upload it as a project to both PyPI and TestPyPI.

If you already have a custom library on PyPI, you can skip this step.

Follow the tutorial below to upload a sample package to TestPyPI and PyPI:

https://packaging.python.org/ja/latest/tutorials/packaging-projects/#creating-a-license

Creating a Workflow File in GitHub Actions

In your repository, go to Actions > Set up a workflow yourself > Create a YAML file with any name (in this case, main.yaml) and commit it.

It’s okay if the file is empty initially.

Setting Up Publishing in PyPI

Traditionally, to upload a package from GitHub Actions to PyPI, you would need to create an API token in PyPI and store that token as a secret in GitHub.

However, by using Publishing, you can automatically issue and authenticate a temporary token when connecting to pre-configured services (including specific users, repositories, etc.)!

Publishing is easier to set up, and since the token has a short expiration time, it also offers better security.

From your project page on PyPI, select Publishing > Add a new publisher.

Fill in the fields as shown in the image below and click Add.

Next, open TestPyPI and create a publisher in the same way.

Use a different environment name than the one used for PyPI.

Workflow in GitHub Actions

Now let’s write the workflow in the YAML file you just created.

1. Start the Workflow When a Tag is Pushed

name: Publish Python 🐍 distribution 📦 to PyPI and TestPyPI on: push: tags: - 'v*.*.*'

2. Build

To build the package, pyproject.toml must exist in the repository, so make sure it is not included in .gitignore.

jobs: build: name: Build distribution 📦 runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 # Checkout the code - name: Set up Python uses: actions/setup-python@v5 # Set up the Python environment with: python-version: "3.x" - name: Install pypa/build # Install the build tool run: >- python3 -m pip install build --user - name: Build a binary wheel and a source tarball run: python3 -m build # Build the package - name: Store the distribution packages # Temporarily store the build artifacts in the dist directory under the name python-package-distributions uses: actions/upload-artifact@v4 with: name: python-package-distributions path: dist/

3. Publish to TestPyPI

 publish-to-testpypi: name: Publish Python 🐍 distribution 📦 to TestPyPI needs: - build # Start the job only if the build job has completed runs-on: ubuntu-latest environment: name: testpypi # Enter the environment name set in the Publisher url: https://test.pypi.org/p/example-package-hanaosan0318 # Project URL permissions: id-token: write # Grant Publishing permissions steps: - name: Download all the dists # Download the build artifacts that were saved earlier uses: actions/download-artifact@v4 with: name: python-package-distributions path: dist/ - name: Publish distribution 📦 to TestPyPI # Publish to TestPyPI uses: pypa/gh-action-pypi-publish@release/v1 with: repository-url: https://test.pypi.org/legacy/

4. Publish to PyPI

 publish-to-pypi: name: >- Publish Python 🐍 distribution 📦 to PyPI needs: - publish-to-testpypi # Start the job only if the TestPyPI publishing job has completed runs-on: ubuntu-latest environment: name: pypi # Enter the environment name set in the Publisher url: https://pypi.org/p/example-package-hanaosan0318 # Project URL permissions: id-token: write steps: - name: Download all the dists uses: actions/download-artifact@v4 with: name: python-package-distributions path: dist/ - name: Publish distribution 📦 to PyPI uses: pypa/gh-action-pypi-publish@release/v1

5. Create a GitHub Release

 github-release: name: >- Create GitHub Release with source code needs: - publish-to-pypi # Start the job only if the PyPI publishing job has completed runs-on: ubuntu-latest permissions: contents: write # Grant permission to create a GitHub release steps: - name: Checkout code uses: actions/checkout@v4 # Checkout the code - name: Create GitHub Release env: GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} # A temporary token that is automatically generated each time the workflow is run run: >- gh release create '${{ github.ref_name }}' --repo '${{ github.repository }}' --notes "Release for version ${{ github.ref_name }}"

Finally, you have defined the workflow!

You can view the entire YAML file at the following link:

https://github.com/hanaosan/ci-cd-practice-python-library/blob/main/.github/workflows/main.yml

Deploying

Update the version in pyproject.toml to the release version and push the changes.

[project] name = "example_package_hanaosan0318" version = "2.0.3" # Change to the release version authors = [ { name="Example Author", email="author@example.com" }, ]

Create a tag and push it.