Unable to find image 'ghcr.io/pypa/gh-action-pypi-publish:61da13deb5f5124fb1536194f82ed3d9bbc7e8f3'

[cpswan]

After bumping to v1.12.0 publishing to TestPyPI is failing:

Unable to find image 'ghcr.io/pypa/gh-action-pypi-publish:61da13deb5f5124fb1536194f82ed3d9bbc7e8f3' locally
docker: Error response from daemon: manifest unknown.

Full logs at the workflow run

Here's the relevant section of my workflow (which is pretty much boilerplate):

  publish-to-testpypi:
    name: Publish package to TestPyPI
    needs:
    - build
    runs-on: ubuntu-latest
    environment:
      name: testpypi
      url: https://test.pypi.org/p/atsdk
    permissions:
      id-token: write  # IMPORTANT: mandatory for trusted publishing
    steps:
    - name: Download all the dists
      uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
      with:
        name: python-package-distributions
        path: dist/
    - name: Publish distribution to TestPyPI
      uses: pypa/gh-action-pypi-publish@61da13deb5f5124fb1536194f82ed3d9bbc7e8f3 # v1.12.0
      with:
        skip-existing: true
        attestations: true
        repository-url: https://test.pypi.org/legacy/

If I was to take a wild guess at why this is happening, it would be permissions for the new Docker images, which might be available to testers, but not yet public.

[cpswan]

Ah... taking another look I've got a new theory...

I'm pinning the action to it's git hash, in order to keep OpenSSF Scorecard checks happy. But it's using the hash as a tag, and there isn't a container image with that tag.

[webknjaz]

Thanks for the report! We didn't take hashes into account in #230 and images with those versions aren't published, and the runtime logic doesn't fall back to local builds.

cc @br3ndonland

[webknjaz]

UPD: I published an image with another tag and in #287 (reply in thread) we confirmed that it fixes the immediate problem.

I'm going to keep this issue open for now to keep it on my radar and think about a more long-term solution.

[webknjaz]

UPD: while releasing v1.12.1, I remembered to also produce an image for the commit hash before pushing the tag/branches — #296.

Long-term, I want to integrate the disconnected part of the process more to avoid human factor-caused mistakes.

[webknjaz]

As for possibly, doing a fallback, here's what should help implement the required check: https://checkson.io/blog/monitoring-docker-images/.

[DinisCruz]

I had the same issue today with a GH Action that has been working for quite a while

the first error I got was a bit misleading since it said "docker: invalid reference format: repository name (owasp-sbot/OSBot-GitHub-Actions) must be lowercase." (see https://github.com/owasp-sbot/OSBot-Utils/actions/runs/11723153140/job/32656045035#step:4:209 )

once I made that lowercase, I got another error saying "Unable to find image 'ghcr.io/owasp-sbot/osbot-github-actions:dev' locally" (see https://github.com/owasp-sbot/OSBot-Utils/actions/runs/11723801117/job/32656257025#step:4:209 )

my temporary workaround was to change in my GH Action

uses: pypa/gh-action-pypi-publish@release/v1

into

uses: pypa/gh-action-pypi-publish@v1.11.0

[blink1073]

I'm seeing a similar situation when publishing to test pypi: "docker: Error response from daemon: Head "https://ghcr.io/v2/blink1073/drivers-github-tools/manifests/INTPYTHON-406": denied." My workaround was to pin the action as well.

https://github.com/blink1073/django-mongodb/actions/runs/11727412713/job/32668549951

[webknjaz]

These two last comments are actually a different issue and I saw two other reports filed separately (#291 / #299). I discovered that this was caused by calling the action from other composite actions.
I attempted fixing it in v1.12.1 and it didn't work because GitHub messes up the context variables we rely on to produce the image reference which consists of an action repo slug and the tag.
This was never encouraged, tested or intended use anyway. It just happened to work with the older structure.
I also discovered that people often misuse the action in an insecure/discouraged manner (like putting the build step into the same job where they elevate privileges with id-token: write).

So with that in mind, I'm thinking I'll document such use as known to be unsupported (as opposed to leading people to believe that it was ever encouraged in the first place).

[parth-kulkarni1]

I had the same issue today with a GH Action that has been working for quite a while

the first error I got was a bit misleading since it said "docker: invalid reference format: repository name (owasp-sbot/OSBot-GitHub-Actions) must be lowercase." (see https://github.com/owasp-sbot/OSBot-Utils/actions/runs/11723153140/job/32656045035#step:4:209 )

once I made that lowercase, I got another error saying "Unable to find image 'ghcr.io/owasp-sbot/osbot-github-actions:dev' locally" (see https://github.com/owasp-sbot/OSBot-Utils/actions/runs/11723801117/job/32656257025#step:4:209 )

my temporary workaround was to change in my GH Action

uses: pypa/gh-action-pypi-publish@release/v1

into

uses: pypa/gh-action-pypi-publish@v1.11.0

If anyone is also having an error when switching to v1.11.0 of ERROR InvalidDistribution: Unknown distribution format: 'XXXXX.whl.publish.attestation', make sure you set attestations: false within your release step. See #283 for more info.

[webknjaz]

@parth-kulkarni1 no, split the job into two instead.