Manager for proxied apps and ability to configure proxied apps CLI args via JupyterLab UI interface

[mahendrapaipuri]

Proposed change

We use jupyter-server-proxy to give access to applications like tensorboard, mlflow, etc to our users via our JupyterHub deployment. These applications (certainly many more) needs CLI args (for instance logdir for tensorboard) to be able to work well. This can be tackled by using custom env vars or config files that users need to configure before launching the app. But it would be better if users can set these CLI args directly from the JupyterLab UI before launching the app.

When user clicks the proxy app on the JupyterLab launcher, it shows an input dialog as below

and then user can simply setup CLI args and/or env vars needed to launch that app. We will send these args to the backend, sanitize them and append to the proxy app command.

Another feature is the ability to launch multiple instances of a given app. This can be achieved by adding one more config option, say num_instances, and then registering the handlers with suffix app_{instance_num}. This way user can have multiple instances of a given app and it is admins who decide how many instances to allow.

With current implementation of jupyter server proxy, there is no way to terminate the app via UI (user can always send a SIGTERM to process pid "manually"). The idea is to add a manager to the proxied apps that can be managed as kernels, terminals in the JupyterLab sidebar as below

The users can terminate apps here, which will send a SIGTERM signal to the PID that we got from simpervisor.

Who would use this feature?

All the users that use jupyter server proxy to get UI access to the applications running on a platform. This will be very useful particularly for HPC platforms where JupyterHub/JupyterLab is most common way to provide web access on the compute nodes.

(Optional): Suggest a solution

We have already implemented these for our platform and it has running well for almost a year. If the developers of jupyter server proxy are interested, we would be happy to open a PR.

[ryanlovett]

@mahendrapaipuri Those seem like nice features to me! I would suggest creating separate PRs for each of the new features for discussion.

Would the parameterisation UI eventually account for all server process options?

The manager UI could be a good place to show available apps (i.e. all entry points) and the attributes of currently running servers, like the assigned port numbers.

[mahendrapaipuri]

@ryanlovett Yes, the idea is to split it in different PRs to have cleaner discussion and review.

Would the parameterisation UI eventually account for all server process options?

So, basically in my proposal we extend the callable arguments with CLI args and environment variables. That means the besides port and base_url in callable function, we have now args and env as well. To parameterise options like map_path, I think we will have to modify what we did for our deployment. Nevertheless I think it is a good idea to be able to parameterise other options as well and we already had few cases where this would be useful. I will look into it. Thanks!!

[bollwyvl]

Rather than a bespoke form UI I'd recommend considering react-json-schema-form, as is used for the JupyterLab settings editor. While it would perhaps take a bit more up-front work, if the schema is derived directly from the traitlets-at-rest, it means future changes to the configuration would require far less client fiddling... indeed, if /api/schema returns the values of interest, the client wouldn't even need to be aware of what it's actually drawing, at the field level, but would be all-but-guaranteed to only submit valid values, likely with better error messages up-front.

Further, instead of putting the form in a block-the-world modal, I'd recommend thinking about it in terms of either a JupyterLab sidebar or draggable main area widget, either of which would play to the strengths of JupyterLab in looking at multiple documents/activities while building up a description of a new proxied command.

Conversely, and looking a few weeks in the future: this will also be possible in Notebook 7, and some thought should be put into how this will play there.

[bollwyvl]

#425 is in, which extends the labextension to work in the /tree view provided by Notebook 7.

The first pass tried to emulate the Classic behavior as closely as possible, with just a menu that always opens a new fat browser tab.

Revisiting this design: the block-the-world-modal would still work in Notebook 7 (and has precedent for e.g. keyboard shortcuts) but some other extensions like Running or the Settings Editor are in tabs in the sidebar-less dockpanel in the 970px main area.

Some treatment ideas:

• keep the menu
  • add an icon indicator for configurable proxies, e.g. ⚙️ or 🔧 which opened a new panel.
    • pop a new tab, labeled with the Title of just the form
    • dispose of the tab when the proxy is launched
• always open a new tab, e.g. Proxies
  • add some other treatment

[mahendrapaipuri]

@bollwyvl Cheers for the ideas. Before we can implement the configuration of proxy apps at runtime via form, we need to finish working on #395 and merge it to main. I have worked on the remaining little issues and please have a look at it. I have added both unit tests and acceptance tests.

If agree with the changes made for all your previous comments, I will rebase onto the main and add acceptance tests for Notebook 7 as well.