"PANIC: runtime error" when pushing container to Gitea container registry

[perfectra1n]

Description

Hi there,

Upon pushing a container to Gitea (even something as simple as a hello-world container), I'm getting the attached kernel panic. I'm not able to replicate it on try.gitea.io, but I can reliably reproduce it on my Gitea instance.

Error

2023/11/16 22:42:51 ...eb/routing/logger.go:47:func1() [T] router: started   PUT /v2/<user>/<repo>/manifests/db5e586 for 10.233.81.22:48164
2023/11/16 22:42:51 ...eb/routing/logger.go:78:func1() [W] router: failed    PUT /v2/<user>/<repo>/manifests/db5e586 for 10.233.81.22:48164, panic in 521.6ms @ container/container.go:516(container.UploadManifest), err=runtime error: invalid memory address or nil pointer dereference
2023/11/16 22:42:51 ...s/process/manager.go:231:remove() [T] Done 65569aeb-31: PUT: /v2/<user>/<repo>/manifests/db5e586
2023/11/16 22:42:51 ...rs/common/errpage.go:26:RenderPanicErrorPage() [E] PANIC: runtime error: invalid memory address or nil pointer dereference

Example Dockerfile:

FROM viaductoss/ksops:v4.2.5 as ksops-builder
FROM quay.io/argoproj/argocd:v2.9.0
USER root
ARG PKG_NAME=ksops
COPY --from=ksops-builder /usr/local/bin/kustomize /usr/local/bin/kustomize
COPY --from=ksops-builder /usr/local/bin/ksops /usr/local/bin/ksops
USER argocd

Upon pushing the container, I get the following response:

...
e3ba1babc8bc: Layer already exists
74c0af6e0227: Layer already exists
received unexpected HTTP status: 500 Internal Server Error

I've deployed the Gitea Helm Chart, and there are no memory limitations, I'm using Redis, Elasticsearch, and Postgres. It was previously functional in 1.20.5. I've executed Gitea doctor:

gitea -c /data/gitea/conf/app.ini doctor check --fix --all

to no avail unfortunately. Please let me know if I can provide any additional information that might help.

Gitea Version

1.21.0

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

https://gist.github.com/perfectra1n/5b24f3c44dd77d3585036a474bb65b15

Screenshots

No response

Git Version

No response

Operating System

Linux

How are you running Gitea?

I've deployed the Gitea Helm Chart, and there are no memory limitations, I'm using Redis, Elasticsearch, and Postgres. I have the following values:

        image:
          repository: gitea/gitea
          # Overrides the image tag whose default is the chart appVersion.
          tag: 1.21.0
          pullPolicy: Always
          rootless: true
        deployment:
          annotations:
            reloader.stakater.com/auto: "true"
        strategy:
          type: Recreate
        clusterDomain: newcluster.local
        podSecurityContext:
          fsGroupChangePolicy: "OnRootMismatch"
        containerSecurityContext:
          capabilities:
            add: ["SYS_CHROOT"]
        redis-cluster:
          enabled: false
        postgresql:
          enabled: false
        postgresql-ha:
          enabled: false
        persistence:
          enabled: true
          mount: true
          create: false
          claimName: "gitea-data"
        gitea:
          additionalConfigSources:
            - secret: 
                secretName: gitea-config-secret
          admin:
            existingSecret: gitea-admin-secret
          config:
            APP_NAME: Gitea
            indexer:
              ISSUE_INDEXER_TYPE: elasticsearch
            
              REPO_INDEXER_ENABLED: true
              REPO_INDEXER_TYPE: elasticsearch
              REPO_INDEXER_REPO_TYPES: sources,mirrors,templates
              MAX_FILE_SIZE: 52428800
            repository:
              ENABLE_PUSH_CREATE_USER: true
              ENABLE_PUSH_CREATE_ORG: true
              DEFAULT_BRANCH: master
              DEFAULT_PRIVATE: true
            repository.upload:
              FILE_MAX_SIZE: 102400
              MAX_FILES: 300
            server:
              DISABLE_SSH: true
              DOMAIN: gitea.domain.com
              ROOT_URL: https://gitea.domain.com
              ENABLE_GZIP: true
              LFS_START_SERVER: true
            database:
              DB_TYPE: postgres
              HOST: main-cluster-rw.databases.svc.newcluster.local
            session:
              PROVIDER: db
            ui:
              THEMES: gitea,arc-green,github-dark,codeberg,anamecool
              DEFAULT_THEME: arc-green
            attachment:
              MAX_SIZE: 102400
              MAX_FILES: 300
            git.timeout:
              GC: 500
            actions:
              ENABLED: true
            service:
              REGISTER_EMAIL_CONFIRM: false
              REGISTER_MANUAL_CONFIRM: true
              DISABLE_REGISTRATION: false
              REQUIRE_SIGNIN_VIEW: true
              ENABLE_NOTIFY_MAIL: true
              ALLOW_ONLY_EXTERNAL_REGISTRATION: false
              ENABLE_CAPTCHA: false
              DEFAULT_KEEP_EMAIL_PRIVATE: true
              DEFAULT_ALLOW_CREATE_ORGANIZATION: true
              DEFAULT_ENABLE_TIMETRACKING: true
              NO_REPLY_ADDRESS: "thing@gmail.com"
            cache:
              ENABLED: true
              ADAPTER: redis
              HOST: redis://:gitea@gitea-redis.gitea.svc.newcluster.local:6379/0?pool_size=100&idle_timeout=180s&
            queue:
              TYPE: redis
              CONN_STR: redis://:gitea@gitea-redis.gitea.svc.newcluster.local:6379/0?pool_size=100&idle_timeout=180s&
            cron:
              ENABLED: true
            cron.update_mirrors:
              SCHEDULE: 0 */12 * * *
            security:
              LOGIN_REMEMBER_DAYS: 180
            log:
              LEVEL: Debug
            mailer:
              ENABLED: true
          metrics:
            enabled: true

Database

PostgreSQL

[wxiaoguang]

Maybe related to Fix Matrix and MSTeams nil dereference #28089

[perfectra1n]

Yep, looking at that now - I do have a Matrix webhook execute upon my repo, org, user, etc., so I bet that it is.