Plugin system for obscure/non-universal features

[ethantkoenig]

I propose adding a plugin system to gitea, which would allow gitea users and developers to add new features while keeping the core gitea codebase small and maintainable. Please read my proposal, and let me know what you think.

Motivation

As gitea grows, there will be requests for more and more features. Many features will be relatively peripheral; while necessary/useful to some users, they will not be used by other users. An example of such a feature is #2211. I'm afraid that gitea will eventually become a bloated collection of incoherent features. Even if these features can be enabled/disabled in a config file, they still make the codebase more complicated and harder to maintain.

Meanwhile, many users of gitea may want to perform small-scale changes and customization to their personal forks. These could range from logo changes to small features that are for their specific use-case (and thus wouldn't make sense in upstream gitea). It would be nice if such users could make these changes without facing merge conflicts when they pull from upstream.

I believe plugins could solve both of these problems.

Specifics (I'm open to suggestions and discussion)

Plugins would be in-process (included during compilation). Plugins would interact with "core" gitea in the following ways:

• Plugins can update templates (this would happen during start-up). This allows plugins to add new UI elements
• Plugins can register new routes
• Plugins can install listeners/callbacks that are triggered when various events happened (e.g. new content is pushed to a repo, which could be useful to a plugin for full-text repo search)
• Plugins can add new tables to the database. Unfornuately, there doesn't seem to be a clean way for plugins to alter existing tables, since they corresponding struct in "core" gitea would need to be updated.

Unanswered Questions

1. What if two plugins rely on different version of the same dependency?

I don't have an answer right now, but hope to investigate possible solutions soon. The answer also depends on whether we end up switching to golang/dep (#2218)

[sapk]

We could use the plugin method introduce in Go1.8 : https://golang.org/pkg/plugin/

[sapk]

Futhermore, I propose :

• Plugins can add command line command (like: import github/bitbucket/gogs)

This would permit to add script from community wihout bloating main prog.

[lafriks]

@sapk Go plugins are supported only on Linux

[sapk]

@lafriks I have to correct what I have said on compiled plugin and vendored deps : golang/go#20481

This should be fix but not yet ready.

[sapk]

Windows and macos plugin build are an objectif but not yet planned and no one on it. 😞
I would still suggest it and explain that it is only possible for linux host in the wait of golang improvement.
Maybe there are better solutions ?

[andreynering]

Gogs cross-reference: gogs/gogs#2438

First, I completely agree the we should not bloat Gitea with more and more small features.

About the plugin system, it's also hard to implement. Go has interpreters for other languages we could look:

• JavaScript: https://github.com/robertkrimen/otto
• Lua: https://github.com/yuin/gopher-lua
• Lua: https://github.com/Shopify/go-lua
• Bash: https://github.com/mvdan/sh (this one may not be suitable for the plugin system, but for allowing running bash commands that will run on Windows, too)

We should look for an approach that is either easy to maintain in the Gitea core and also for plugin developers.

[sapk]

https://github.com/hashicorp/go-plugin suggested on gogs issue is also a good solution. It support some few good feature like negotiation for compatibility between main and plugins (and is planning "plugin fetching"). And plugin could be develop in various languages just need gRPC interface.

[sapk]

gRPC permit support for plugin triggerring event and not wait to be called.

[jonasfranz]

I think that we only need a better filtering of features and we have to know what we want to implement and what not. But I disagree with you statement that #2211 is only used be a "small minority of users" because it makes gitea interesting for small businesses that need such features. And my apprehension is that the features will not be such good integrated or have a good quality if they come via plugin.
I agree that there should be a simple possibility to add small additions like #2221 but I do not think that it would be a good option for (in relation) big features like #2211.

[lunny]

I agree with @JonasFranzDEV enhancement of issues will let small company remove their redmine and such others.

[ethantkoenig]

@JonasFranzDEV @lunny Apologies if I came across as unfairly criticizing the timetracking feature. I didn't mean to say that the feature is useless, but only that it is peripheral to the main purpose of gitea, and that large subset of gitea users do not need it. I've updated the wording in my description accordingly.

I'm not sure "we just need to do a better job deciding what to implement" is a viable solution. Who decides what is worth implementing? What if user X needs feature Y, but it is decided that feature Y will not be implemented? User X's only option is to implement feature Y on his/her own fork, which is problematic as I described in the description.

Finally, I disagree with the "too big for a plugin" argument. The fact that the timetracking feature is large means it will add even more overhead and complexity to the codebase, which I believe further underscores the need for moving it to a plugin.

[lafriks]

@ethantkoenig plugins are good thing to have but also come with additional problems like it is harder to change core functionality/design/structure as such changes can and most probably will break plugins and is harder to test

[sapk]

@lafriks It also give us room for change because with too much function integrate in core we lock the structure of code and giving a standard interface (with versioning) for plugin will allow function to evolve at there own rythm. If a user want to keep a func implemented by a plugin that does not support the last version of core, it become the choice of the user to stay with an older version in the wait of an update of the plugin (or contribute to the update).

I think we all agree to had plugin, we just need to decide what type and how to limit bruden for gitea and plugins developer by starting a roadmap and defining interface and what a plugin can do.