New SpecifyEvents interface

[tyarkoni]

This is a first pass at a new interface for flexible transformation/processing of events, as discussed in #1359. It's a WIP and there are no tests yet--plus it depends on a JSON transformation spec that will need to be refined--but I'm opening a PR for discussion.

The basic idea is to introduce a new SpecifyEvents interface (meant to parallel SpecifyModel) that takes a set of event files (or a Bunch of onset/duration/amplitude) as input, performs an arbitrary set of transformations on the data (more on that below), and then outputs the new events as a Bunch compatible with the subject_info input of SpecifyModel. This is fully compatible with SpecifyModel, so users who don't need to manipulate event information in any way can simply ignore the new interface (though at some point it might be worth discussing whether SpecifyEvents should become the mandatory way to go from event files to a design matrix).

Here's a partial example of what SpecifyEvents usage looks like in context:

eventspec = pe.Node(interface=events.SpecifyEvents(), name="eventspec")
eventspec.inputs.input_units = 'secs'
eventspec.inputs.time_repetition = 3
eventspec.inputs.transformations = 'transformations.json'

# assume a standard fMRI workflow exists--pipe lists of event files to the
# event_files input, which can be either flat event files, or Bunch.
wf.connect(datasource, ('behav', check_behav_list), eventspec, 'event_files')
# Connect the output to the input of an instance of SpecifyModel
wf.connect(eventspec, 'subject_info', modelspec, 'subject_info')

Now suppose we have a list of conditions ["cond001", "cond002", "cond003", "cond004"]. Then we can pass in a json transformation file that looks like this:

{
  "steps": [
    {
      "name": "orthogonalize",
      "input": [
        "cond001"
      ],
      "X_cols": [
        "cond002",
        "cond003",
        "cond004"
      ],
      "output": [
        "cond001_orth"
      ]
    },
    {
      "name": "standardize",
      "input": [
        "cond001_orth",
        "cond002",
        "cond003",
        "cond004"
      ]
    },
    {
      "name": "select",
      "input": [
        "cond001_orth",
        "cond003",
        "cond004"
      ]
    }
  ]
}

Here we're performing 3 different operations. First, we orthogonalize cond001 with respect to the other 3 conditions. We rename the output cond001_orth. Next, we standardize all 4 columns (i.e., subtract mean and divide by SD). Finally, we decide to keep only 3 of the conditions for inclusion in the model. These will be passed as a Bunch to SpecifyModel, and any existing workflow should in principle work just fine (though this remains to be tested).

Note that there are many other implemented features not illustrated in the above example. For instance, one can multiply conditions or sets of conditions (e.g., to code interaction terms); dummy-code variable; apply grouping operations (e.g., standardize a given condition separately for each level of another condition); etc.

[satra]

perhaps simply create a flag here: no_pandas = True and raise it in the run_interface function below

[qmac]

Should this be have_pandas?

[djarecka]

@tyarkoni - are you still working on this pr?

[tyarkoni]

Not directly, no. But indirectly, yes. We're working on a pybids module that does all of this work, and once it's done, the plan is to wrap that with a much lighter Nipype interface that mostly just passes a BIDS-Model spec through. But I'll close the PR, because the next iteration will probably be a radical departure from this code. Thanks for the reminder!