Storage in S3 buckets using minio not working in Gitea 1.20.0 with working config from Gitea 1.19.4

[tobiasbp]

Description

Storage in s3 buckets (using minio) no longer works when using Gitea 1.20.0 with chart v9.0.0. Changing the Gitea version to 1.19.4 with chart v9.0.0 (same configuration) makes storage in s3 buckets work again as described in the issue here:
https://gitea.com/gitea/helm-chart/issues/469

The storage configuration is:

    storage:
      MINIO_ACCESS_KEY_ID: ***
      MINIO_BUCKET: my-existing-bucket
      MINIO_ENDPOINT: storage.googleapis.com
      MINIO_INSECURE_SKIP_VERIFY: false
      MINIO_SECRET_ACCESS_KEY: ***
      MINIO_USE_SSL: true
      SERVE_DIRECT: false
      STORAGE_TYPE: minio

With the above storage config, Gitea 1.20.0 says:

2023/07/18 16:33:54 ...s/setting/session.go:74:loadSessionFrom() [I] Session Service Enabled
2023/07/18 16:33:54 ...s/storage/storage.go:176:initAttachments() [I] Initialising Attachment storage with type: minio
2023/07/18 16:33:54 ...les/storage/minio.go:81:NewMinioStorage() [I] Creating Minio storage at storage.googleapis.com:my-existing-bucket with base path attachments/
2023/07/18 16:33:55 routers/init.go:60:mustInit() [F] code.gitea.io/gitea/modules/storage.Init failed: The requested bucket name is not available. The bucket namespace is shared by all users of the system. Please select a different name and try again

Is Gitea/minio trying to create the existing bucket?

Gitea Version

1.20.0

Can you reproduce the bug on the Gitea demo site?

No

Log Gist

No response

Screenshots

No response

Git Version

No response

Operating System

No response

How are you running Gitea?

In Kubernetes with chart v9.0.0.

Database

PostgreSQL

[tobiasbp]

I have updated to Gitea 1.20.1 (Now using Gitea chart 9.0.4). If I configure MINIO_LOCATION to the location of my existong bucket, it sounds like minio tries to create the bucket??

Defaulted container "gitea" out of: gitea, init-directories (init), init-app-ini (init), configure-gitea (init)
2023/07/25 15:51:00 cmd/web.go:223:runWeb() [I] Starting Gitea on PID: 7
2023/07/25 15:51:00 cmd/web.go:148:serveInstalled() [I] Gitea version: 1.20.1 built with GNU Make 4.4.1, go1.20.6 : bindata, timetzdata, sqlite, sqlite_unlock_notify
2023/07/25 15:51:00 cmd/web.go:149:serveInstalled() [I] App path: /usr/local/bin/gitea
2023/07/25 15:51:00 cmd/web.go:150:serveInstalled() [I] Work path: /data
2023/07/25 15:51:00 cmd/web.go:151:serveInstalled() [I] Custom path: /data/gitea
2023/07/25 15:51:00 cmd/web.go:152:serveInstalled() [I] Config file: /data/gitea/conf/app.ini
2023/07/25 15:51:00 cmd/web.go:153:serveInstalled() [I] Run mode: dev
2023/07/25 15:51:00 cmd/web.go:154:serveInstalled() [I] Prepare to run web server
2023/07/25 15:51:01 routers/init.go:112:InitWebInstalled() [I] Git version: 2.40.1, Wire Protocol Version 2 Enabled (home: /data/home)
2023/07/25 15:51:02 ...les/setting/cache.go:75:loadCacheFrom() [I] Cache Service Enabled
2023/07/25 15:51:02 ...les/setting/cache.go:90:loadCacheFrom() [I] Last Commit Cache Service Enabled
2023/07/25 15:51:02 ...s/setting/session.go:74:loadSessionFrom() [I] Session Service Enabled
2023/07/25 15:51:02 ...s/storage/storage.go:176:initAttachments() [I] Initialising Attachment storage with type: minio
2023/07/25 15:51:02 ...les/storage/minio.go:81:NewMinioStorage() [I] Creating Minio storage at storage.googleapis.com:gitea-stg-aqrk with base path attachments/
2023/07/25 15:51:02 routers/init.go:60:mustInit() [F] code.gitea.io/gitea/modules/storage.Init failed: permission denied

[lunny]

I have updated to Gitea 1.20.1 (Now using Gitea chart 9.0.4). If I configure MINIO_LOCATION to the location of my existong bucket, it sounds like minio tries to create the bucket??

Defaulted container "gitea" out of: gitea, init-directories (init), init-app-ini (init), configure-gitea (init)
2023/07/25 15:51:00 cmd/web.go:223:runWeb() [I] Starting Gitea on PID: 7
2023/07/25 15:51:00 cmd/web.go:148:serveInstalled() [I] Gitea version: 1.20.1 built with GNU Make 4.4.1, go1.20.6 : bindata, timetzdata, sqlite, sqlite_unlock_notify
2023/07/25 15:51:00 cmd/web.go:149:serveInstalled() [I] App path: /usr/local/bin/gitea
2023/07/25 15:51:00 cmd/web.go:150:serveInstalled() [I] Work path: /data
2023/07/25 15:51:00 cmd/web.go:151:serveInstalled() [I] Custom path: /data/gitea
2023/07/25 15:51:00 cmd/web.go:152:serveInstalled() [I] Config file: /data/gitea/conf/app.ini
2023/07/25 15:51:00 cmd/web.go:153:serveInstalled() [I] Run mode: dev
2023/07/25 15:51:00 cmd/web.go:154:serveInstalled() [I] Prepare to run web server
2023/07/25 15:51:01 routers/init.go:112:InitWebInstalled() [I] Git version: 2.40.1, Wire Protocol Version 2 Enabled (home: /data/home)
2023/07/25 15:51:02 ...les/setting/cache.go:75:loadCacheFrom() [I] Cache Service Enabled
2023/07/25 15:51:02 ...les/setting/cache.go:90:loadCacheFrom() [I] Last Commit Cache Service Enabled
2023/07/25 15:51:02 ...s/setting/session.go:74:loadSessionFrom() [I] Session Service Enabled
2023/07/25 15:51:02 ...s/storage/storage.go:176:initAttachments() [I] Initialising Attachment storage with type: minio
2023/07/25 15:51:02 ...les/storage/minio.go:81:NewMinioStorage() [I] Creating Minio storage at storage.googleapis.com:gitea-stg-aqrk with base path attachments/
2023/07/25 15:51:02 routers/init.go:60:mustInit() [F] code.gitea.io/gitea/modules/storage.Init failed: permission denied

What's your workround? Maybe I can get some clues to fix it.

[tobiasbp]

What's your workround? Maybe I can get some clues to fix it.

I have no workaround? I can no longer use my S3 storage. It's just that the error i the log changes if I set MINIO_LOCATION.

I assume my problem is related to Rework storage settings -> #23911

But since I only have one section with storage config, I think my config should still work. I have a hard time following the changes being described in #23911

[lunny]

What's your workround? Maybe I can get some clues to fix it.

I have no workaround? I can no longer use my S3 storage.

I assume my problem is related to Rework storage settings -> #23911

Yes, I think so. Can you post your generated storage configuration in app.ini and remove the seurity informations. And I will figure it out tomorrow.

[tobiasbp]

Here is my app.ini. I have had to set storage.STORAGE_TYPE = local to get the pod to start up. The rest of the config in [storage] is untouched. I don't know if I should configure storage.MINIO_LOCATION or not? The error changes when I do (As described in earlier posts):

/var/lib/gitea $ cat /data/gitea/conf/app.ini


APP_NAME = Gitea (stg)
RUN_MODE = dev
WORK_PATH = /data

[storage]
MINIO_LOCATION = europe-north1
MINIO_USE_SSL = true
MINIO_ENDPOINT = storage.googleapis.com
STORAGE_TYPE = local
SERVE_DIRECT = false
MINIO_BUCKET = gitea-stg-aqrk
MINIO_SECRET_ACCESS_KEY = **REDACTED**
MINIO_ACCESS_KEY_ID = **REDACTED**
MINIO_INSECURE_SKIP_VERIFY = false

[indexer]
ISSUE_INDEXER_TYPE = db

[server]
LFS_JWT_SECRET = **REDACTED**
ROOT_URL = **REDACTED**
LFS_START_SERVER = true
ENABLE_PPROF = false
LANDING_PAGE = login
SSH_PORT = 22
APP_DATA_PATH = /data
SSH_LISTEN_PORT = 2222
PROTOCOL = http
HTTP_PORT = 3000
START_SSH_SERVER = true
DISABLE_SSH = true
SSH_DOMAIN = **REDACTED**
DOMAIN = **REDACTED**

[database]
NAME = gitea-db
SSL_MODE = disable
DB_TYPE = postgres
HOST = sql-proxy-gcloud-sqlproxy:5432
USER = gitea-user
PASSWD = **REDACTED**

[service]
SHOW_MILESTONES_DASHBOARD_PAGE = false
REQUIRE_SIGNIN_VIEW = true
DISABLE_REGISTRATION = true
EXPLORE = map[DISABLE_USERS_PAGE:true REQUIRE_SIGNIN_VIEW:true]
DEFAULT_ORG_VISIBILITY = private
DEFAULT_USER_VISIBILITY = private
DEFAULT_ALLOW_CREATE_ORGANIZATION = false

[repository]
DISABLE_STARS = true
DISABLE_MIGRATIONS = true
FORCE_PRIVATE = true
ROOT = /data/git/gitea-repositories
DISABLED_REPO_UNITS = repo.wiki, repo.ext_wiki, repo.packages, repo.projects, repo.issues, repo.ext_issues, repo.pulls
DEFAULT_PRIVATE = true

[oauth2_client]
REGISTER_EMAIL_CONFIRM = false
USERNAME = email
UPDATE_AVATAR = true
ENABLE_AUTO_REGISTRATION = true
OPENID_CONNECT_SCOPES = openid profile email

[security]
DISABLE_WEBHOOKS = true
INTERNAL_TOKEN = **REDACTED**
SECRET_KEY = **REDACTED**
INSTALL_LOCK = true

[session]
PROVIDER_CONFIG = redis://redis-1.source-access.private:6379/0?pool_size=100&idle_timeout=180s
PROVIDER = redis

[cache]
HOST = redis://redis-1.source-access.private:6379/0?pool_size=100&idle_timeout=180s
ENABLED = true
ADAPTER = redis

[admin]
DISABLE_REGULAR_ORG_CREATION = true
DEFAULT_EMAIL_NOTIFICATIONS = false

[metrics]
ENABLED = true

[oauth2]
JWT_SECRET = **REDACTED**

[evanreichard]

I'm assuming this is related, but since the update my [storage.packages] override no longer works. I don't think I violate the incompatibility detailed in #23911

For the time being I just set up a symbolic link.

My redacted configuration:

APP_NAME = Gitea: Git with a cup of tea
RUN_MODE = prod
RUN_USER = git
WORK_PATH = /data/gitea

[repository]
ROOT = /data/git/repositories

[repository.local]
LOCAL_COPY_PATH = /data/gitea/tmp/local-repo

[repository.upload]
TEMP_PATH = /data/gitea/uploads

[server]
APP_DATA_PATH = /data/gitea
SSH_DOMAIN = <REDACTED>
HTTP_PORT = 3000
ROOT_URL = <REDACTED>
DISABLE_SSH = false
SSH_PORT = 22
SSH_LISTEN_PORT = 22
LFS_START_SERVER = true
DOMAIN = <REDACTED>
LFS_JWT_SECRET = <REDACTED>
OFFLINE_MODE = false

[database]
PATH = /data/gitea/gitea.db
DB_TYPE = sqlite3
HOST = localhost:3306
NAME = gitea
USER = <REDACTED>
PASSWD = <REDACTED>
SCHEMA = 
SSL_MODE = disable
CHARSET = utf8

[indexer]
ISSUE_INDEXER_PATH = /data/gitea/indexers/issues.bleve

[session]
PROVIDER_CONFIG = /data/gitea/sessions
PROVIDER = file

[picture]
AVATAR_UPLOAD_PATH = /data/gitea/avatars
REPOSITORY_AVATAR_UPLOAD_PATH = /data/gitea/repo-avatars
DISABLE_GRAVATAR = false
ENABLE_FEDERATED_AVATAR = true

[log]
ROOT_PATH = /data/gitea/log
MODE = file
LEVEL = info

[security]
INSTALL_LOCK = true
SECRET_KEY = <REDACTED>
INTERNAL_TOKEN = <REDACTED>

[service]
DISABLE_REGISTRATION = true
REQUIRE_SIGNIN_VIEW = false
REGISTER_EMAIL_CONFIRM = false
ENABLE_NOTIFY_MAIL = false
ALLOW_ONLY_EXTERNAL_REGISTRATION = false
ENABLE_CAPTCHA = false
DEFAULT_KEEP_EMAIL_PRIVATE = false
DEFAULT_ALLOW_CREATE_ORGANIZATION = true
DEFAULT_ENABLE_TIMETRACKING = true
NO_REPLY_ADDRESS = noreply.localhost
SHOW_REGISTRATION_BUTTON = false

[oauth2]
JWT_SECRET = <REDACTED>

[mailer]
ENABLED = false

[openid]
ENABLE_OPENID_SIGNIN = false
ENABLE_OPENID_SIGNUP = false

[ui]
DEFAULT_THEME = arc-green
THEMES = gitea,arc-green

[lfs]
PATH = /data/git/lfs

[attachment]
PATH = /data/gitea/attachments
MAX_SIZE = 50

[webhook]
ALLOWED_HOST_LIST = <REDACTED>

[storage.packages]
PATH = /packages

[tobiasbp]

Any idea as to what could be causing this issue? Is it a problem in Minio?

[Lapotor]

On v1.20.2 the error still appears.
I think that is a problem on the implementation because it tries to create the bucket on every storage.