TestSignalExtraction.test_signal_extr_traits_valid failing

[djarecka]

Summary

I just noticed that TestSignalExtraction.test_signal_extr_traits_valid is failing on my OSX with the message:

E           traits.trait_errors.TraitError: The trait 'out_file' of a SignalExtractionOutputSpec instance is an existing file name, but the path  '/private/var/folders/5s/hk2j7ylx54b2clm9bx1mpvj00000gq/T/tmp_2bektow/SignalExtraction/signals.tsv' does not exist.

I can see that the file signals.tsv is being created but is being deleted here.

@oesteban I'll try to figure out why this happens on my OSX and not on CircleCI (or other Linux we tried today), but if you have any suggestions, please let me know.

[oesteban]

Looks a lot like nipreps/fmriprep#936

[oesteban]

@djarecka could you try on you OSX the PR referenced above? Thanks!

[djarecka]

it could be related to the temporary directory paths IMO.

i'm debugging it right now, and I can see that needed file here contains signals.tsv as /private/var/folders/5s/hk2j7ylx54b2clm9bx1mpvj00000gq/T/tmpuuucmm6n/SignalExtraction/signals.tsv
but file from walks_files(cwd) has /var/folders/5s/hk2j7ylx54b2clm9bx1mpvj00000gq/T/tmpuuucmm6n/SignalExtraction/signals.tsv.

[oesteban]

Okay, it definitely looks like nipreps/fmriprep#936. I bet this is only happening with interfaces based off of SimpleInterface.
Could you give #2383 a quick try?

[djarecka]

just run with your oesteban-maint/node-hash and the test is still failing

[djarecka]

@oesteban not sure if this helps, but only signals.tsv in needed_files has /private at the beginning, i.e.:

(Pdb) needed_files
['/private/var/folders/5s/hk2j7ylx54b2clm9bx1mpvj00000gq/T/tmp1fmidn70/SignalExtraction/signals.tsv', '/var/folders/5s/hk2j7ylx54b2clm9bx1mpvj00000gq/T/tmp1fmidn70/SignalExtraction/_0x074086d35e0945269f2581ef95cdb8c1_unfinished.json', '/var/folders/5s/hk2j7ylx54b2clm9bx1mpvj00000gq/T/tmp1fmidn70/SignalExtraction/_inputs.pklz', '/var/folders/5s/hk2j7ylx54b2clm9bx1mpvj00000gq/T/tmp1fmidn70/SignalExtraction/_node.pklz']

[oesteban]

Ok, this is the problem

nipype/nipype/interfaces/nilearn.py

Line 109 in 045b28e

self._results['out_file'] = os.path.abspath(self.inputs.out_file)

which aligns well with this discussion and the issue in fmriprep.

The problem, basically, is that interfaces shouldn't use os.getcwd() and stick with runtime.cwd. In general this shouldn't be a problem, but apparently SimpleInterface circumvents the change of directory in Node.

Since you are running step-by-step, would you mind checking why this line:

nipype/nipype/pipeline/engine/nodes.py

Line 462 in f3b0912

os.chdir(outdir)

seems not to be run with SimpleInterfaces (like SignalExtraction)?

EDIT: the problem is not restricted to SimpleInterfaces :(

[oesteban]

Got it. Working on a fix. Sorry for the issue.

[djarecka]

@oesteban sorry for late reply, had a lunch break. Let me know if there is something I can test for you

[djarecka]

#2384 solves the problem on my OSX!