Summary

Support for FUSE filesystem for workdir

Actual behavior

Workdir reads and writes many intermediate files, with higher than normal latency. This leads to workflows failing due to files not present. (FUSE file systems keep a copy of the file in RAM and writes once the handle is closed. But the actual time it takes the files to be present for a new process can vary).

Expected behavior

Nipype, should have a delay mechanism on file read fails possibly with exponential backoff.

Platform details:

I tested this in GCP, with gcsfuse. There were random failures of workflow files not being present.

Execution environment

GCP machine, with gcsfuse mounted workdir. Singularity container installed with required tools with nipype.