Create a network of containers to run interfaces

[kaczmarj]

Nipype could be installed in a clean environment without other software (like FSL, FreeSurfer, SPM, etc.), and nipype would orchestrate other containers that include necessary software to run nodes/workflows. One can imagine one or several versioned containers per interface and potentially minified containers.

[chrisgorgo]

Each Interface would have a dedicated container (but many Interfaces could share one container). Container name would bet a property of the Interface.

This would allow running workflows without installing any software other than nipype and Docker or Singularity.

[kaczmarj]

Kubernetes might be useful for this.

[kaczmarj]

Related to #2071.

[anibalsolon]

A container spawning jobs on Kubernetes would beautifully do the trick.

The minikube could be used to run Kubernetes locally, however, these are the installation requirements. It might be cumbersome for initial users, but managed Kubernetes clusters from Google Cloud, Azure, IBM and AWS (on preview) could provide usage traction for this feature.

After this initial minikube setup, a Nipype container with some cluster capatibilites could use the Kubernetes python client to spawn jobs using BIDS apps images, sharing a persistent volume to provide the initial data and retrieve the results.

I believe this job scheduler could be a nipype's execution plugin to run seamlessly the workflows. 👌

[unikzforce]

I suggest you take a look at Argo workflow. It's a kubernetes workflow engine.. it could be used as execution engine behind the scene. It's api is based on openapi so generating python client is seamless for it.