Automating a podman-based CI workflow using python

Question

I want to write a python3 script that will set up a pristine system (e.g: "fedora:42"), copy a local clone of a git repository, and run its tests. For continuous integration. I really would prefer to avoid Dockerfiles. A similar code in perl is https://github.com/thewml/website-meta-language/blob/master/CI-testing/docker-ci-run.pl . (Note that it uses podman on fedora.) I want the python3 equivalents of the methods in https://metacpan.org/pod/Docker::CLI::Wrapper::Container . Can anyone help?

https://github.com/shlomif/pysol-cards-in-C/blob/master/docker_ci.py gives me “podman.errors.exceptions.APIError: 500 Server Error: Internal Server Error (can only create exec sessions on running containers: container state improper)”.

How can i get .exec_run() to work? I want to be able to execute bash codes inside the container, see their stdout/stderr, and wait for them to finish. Synchronously.

I am on Fedora 42 x86-64 podman.

my code so far is:

#! /usr/bin/env python3
# -*- coding: utf-8 -*-
# vim:fenc=utf-8
#
# Copyright © 2025 Shlomi Fish < https://www.shlomifish.org/ >
#
# Licensed under the terms of the MIT license.

"""

"""

import json
# import time
from podman import PodmanClient

"""Demonstrate PodmanClient."""

# Provide a URI path for the libpod service.  In libpod, the URI can be a unix
# domain socket(UDS) or TCP.  The TCP connection has not been implemented in
# this package yet.

uri = "unix:///run/user/1000/podman/podman.sock"

with PodmanClient(base_url=uri) as client:
    version = client.version()
    if False:
        print("Release: ", version["Version"])
        print("Compatible API: ", version["ApiVersion"])
        print("Podman API: ",
              version["Components"][0]["Details"]["APIVersion"], "\n")

        # get all images
        for image in client.images.list():
            print(image, image.id, "\n")

    sysname = 'fedora:42'
    pull = client.images.pull(sysname)
    print(pull)
    image = client.images.get(sysname)
    # image = pull
    print(image)
    containers = client.containers
    # container = image.create()
    container = containers.create(image)
    print(container)
    # container.attach(eot=4)
    # container.attach()
    container2 = containers.run(image=image, detach=True,)
    print(container)
    print(container2)
    # time.sleep(5)
    ret = container.exec_run(
        cmd='echo helloworld\n',
        demux=True,
    )
    print(container)
    print(ret)
    # container.run()
    print('before exec_run', container)

    if False:
        # find all containers
        for container in client.containers.list():
            # After a list call you would probably want to reload the container
            # to get the information about the variables such as status.
            # Note that list() ignores the sparse option and assumes True
            # by default.
            container.reload()
            print(container, container.id, "\n")
            print(container, container.status, "\n")

            # available fields
            print(sorted(container.attrs.keys()))

        print(json.dumps(client.df(), indent=4))

Its output is:

$ python docker_ci.py
<Image: 'registry.fedoraproject.org/fedora:42'>
<Image: 'registry.fedoraproject.org/fedora:42'>
<Container: 875f2fd7e5>
<Container: 875f2fd7e5>
<Container: 754729039a>
Traceback (most recent call last):
  File "/home/shlomif/progs/python/pysol-cards-in-C/docker_ci.py", line 53, in <module>
    ret = container.exec_run(
        cmd='echo helloworld\n',
        demux=True,
    )
  File "/usr/lib/python3.13/site-packages/podman/domain/containers.py", line 211, in exec_run
    response.raise_for_status()
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^
  File "/usr/lib/python3.13/site-packages/podman/api/client.py", line 82, in raise_for_status
    raise APIError(cause, response=self._response, explanation=message)
podman.errors.exceptions.APIError: 500 Server Error: Internal Server Error (can only create exec sessions on running containers: container state improper)

I really would prefer to avoid Dockerfiles why? using python why with python? Why not just podman run fedora:42 bash -c "git clone this && cd this && do that"? Can anyone help? do you need a python client? How about just with subprocess.Popen("podman run -d fedora:42 sleep infintiy") as f: id = f.stdout.read(); subprocess.run("podman run {id} stuff to run")? — KamilCuk
– KamilCuk, Commented May 8 at 9:50

David Maze · Accepted Answer · 2025-05-08 10:20:21Z

I'd rebuild this to create a new container for each command you want to run. That would look something like:

container = containers.create(image, ["echo", "helloworld"])
container.start()
container.wait(condition="exited")
print(container.logs())
container.remove()

You could also use containers.run() as a shorthand for this. This is true more generally: docker run and the equivalent Podman containers.run() call is exactly equivalent to docker create immediately followed by docker start.

The "exec" command isn't what you want here. As the error message notes, it's a debugging tool that lets you run an additional command in a running container, which you don't have. If you were doing this without Docker, you'd subprocess.run(["pytest", ...]) or similar for each individual command; you wouldn't try to subprocess.Popen(["sleep", "infinity"]) and then somehow "go inside" that process to do your work, but that's the workflow that container.exec() would try to simulate.

Collectives™ on Stack Overflow

Automating a podman-based CI workflow using python

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related