Steps 1-4 create preprocessing information (e.g., from the ICA, segment labeling), that later is applied to the raw data starting in step 5.
All steps use FieldTrip and were executed within MATLAB 2020a.
Deviations from the standard pipeline:
Many scripts were run on the high performance computing cluster (HPC) at the Max Planck Institute for Human Development (Berlin, Germany). One of two different deployment approaches was used: either (a) the relevant script was compiled or (b) the script was directly called with multiple MATLAB instances. In the case of (a), the respective code folder will contain a "_prepare" file that was used to compile the code, as well as a '_START' bash script, that deployed the compiled code; in the case of (b) bo compiling was done, and the '_START' script immediately calls the relevant MATLAB code. To re-run scripts outside of the HPC environment, a script needs to be written that loops the script across all subjects. Note that for debugging purposes, code usually checks whether it is run on a mac, and if so, performs all computations on an example subject. Depending on the deployment situation, this section may need to be adapted.
This step loads the raw data again and segments them to the desired time window, prior filters will be applied to these data in the next step.
Identify additional artifacts after removing ICA components. This step does NOT yet remove anything. We only calculate the data to be removed in the next step.
This repository is a DataLad dataset. It provides fine-grained data access down to the level of individual files, and allows for tracking future updates. In order to use this repository for data retrieval, DataLad is required. It is a free and open source command line tool, available for all major operating systems, and builds up on Git and git-annex to allow sharing, synchronizing, and version controlling collections of large files. You can find information on how to install DataLad at handbook.datalad.org/en/latest/intro/installation.html.
A DataLad dataset can be cloned by running
datalad clone <url>
Once a dataset is cloned, it is a light-weight directory on your local machine. At this point, it contains only small metadata and information on the identity of the files in the dataset, but not actual content of the (sometimes large) data files.
After cloning a dataset, you can retrieve file contents by running
datalad get <path/to/directory/or/file>`
This command will trigger a download of the files, directories, or subdatasets you have specified.
DataLad datasets can contain other datasets, so called subdatasets. If you clone the top-level dataset, subdatasets do not yet contain metadata and information on the identity of files, but appear to be empty directories. In order to retrieve file availability metadata in subdatasets, run
datalad get -n <path/to/subdataset>
Afterwards, you can browse the retrieved metadata to find out about
subdataset contents, and retrieve individual files with datalad get.
If you use datalad get <path/to/subdataset>, all contents of the
subdataset will be downloaded at once.
DataLad datasets can be updated. The command datalad update will
fetch updates and store them on a different branch (by default
remotes/origin/master). Running
datalad update --merge
will pull available updates and integrate them in one go.
DataLad datasets contain their history in the git log.
By running git log (or a tool that displays Git history) in the dataset or on
specific files, you can find out what has been done to the dataset or to individual files
by whom, and when.
More information on DataLad and how to use it can be found in the DataLad Handbook at handbook.datalad.org. The chapter "DataLad datasets" can help you to familiarize yourself with the concept of a dataset.