# Spelling Interface using Introcortical Signals in a Completely Locked-In Patient enabled via Auditory Neurofeedback Training The dataset in this repository accompanies an article published in Nature Communications. In case you use the data, please cite the paper: Ujwal Chaudhary, Ioannis Vlachos, Jonas B. Zimmermann, Arnau Espinosa, Alessandro Tonin, Andres Jaramillo-Gonzalez, Majid Khalili-Ardali, Helge Topka, Jens Lehmberg, Gerhard M. Friehs, Alain Woodtli, John P. Donoghue, and Niels Birbaumer: Spelling Interface using Introcortical Signals in a Completely Locked-In Patient enabled via Auditory Neurofeedback Training. Nature Communications, **ToDo: update**. ## License This dataset is shared under the [Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International Public License](https://creativecommons.org/licenses/by-nc-sa/4.0/). See the `LICENSE` document. The Wyss Center for Bio and Neuroengineering, Geneva, Switzerland, is the owner of the dataset. ## Description of the dataset This dataset contains intracranial data (threshold crossings converted to multi-unit spike rate estimates) as well as behavioral data (events from the BCI software) recorded from one participant in a clinical case over a year. The dataset contains data from two conditions: - auditory neurofeedback - speller (training and free) In addition, electrode array impedance files as well as neural event files from time points presented in the paper are shared. ### Neurofeedback blocks Neurofeedback trial structure is described in the paper. A list of blocks is given in the Excel file `DataOverview.xlsx`, sheet 'neurofeedback'. The name of the datafile corresponds to the files stored in `KIAP_BCI_neurofeedback`. Each block is associated with a spike rate binary data file as well as two log files, an `events_*.txt` and an `info_*.log` file. The `info` file only contains decoder decision events, while `events` contains other events in the trial as well. #### Format of `info` files A line in the `info` log file has the following form: 2019-07-20 16:34:16.255668 feedback - Decoder decision: unclassified - ('feedback', 'down') 1. `2019-07-20 16:34:16.255668` Timestamp in local time 2. `feedback - Decoder decision:` task name 3. `unclassified`: result of classification. One of {`yes`, `no`, `unclassified`}. `yes`: normalized firing rate crossed upper threshold. `no`: normalized firing rate crossed lower threshold. `unclassified`: trial timed out without crossing the threshold. 4. `('feedback', 'down')`: task name and target; one of {`up`, `down`}. `up` maps to `yes`, `down` maps to `no`. Until 2019-07-19 (inclusive), the result logged was slightly different: Instead of logging the classification result, it was logged whether the patient reached the target or not. When the target was reached the result is logged as `yes`, both for `up` and `down` targets. It was logged as `unclassified` in other cases, i.e. when the patient did not acquire the target within the timeout period. #### Format of `events` files Rows in the `events` files have the following format: 95862, b'feedback, Training, up, Block, start' 501039, b'feedback, Training, up, baseline, start' 558534, b'feedback, Training, up, baseline, stop' 558534, b'feedback, Training, up, stimulus, start' 585989, b'feedback, Training, up, stimulus, stop' 592204, b'feedback, Training, up, response, start' 659908, b'feedback, Training, up, response, stop' 659908, b'feedback, Training, up, Decoder decision is: no' 3231064, b'feedback, Training, down, Block, stop' 1. First, there is a timestamp in sample clock ticks (1 tick = 1/30 millisecond) 2. The message is enclosed by `b'` and `'` 3. The message consists of task, mode, target cue, trial period, {start|stop} for trial phase messages 4. At the end of the trial, there is a 'Decoder decision' message, which can be 'yes' (corresponding to 'up' target), 'no' ('down' target), and 'unclassified' (no class detected). 5. The log messages should be enclosed in a 'Block, start' and 'Block, stop' message pair. For these block messages, the target is irrelevant. Until 2019-07-19 (inclusive), the decoder decision logged was slightly different: Instead of logging the classification result, it was logged whether the patient reached the target or not. When the target was reached the result is logged as `yes`, both for `up` and `down` targets. It was logged as `unclassified` in other cases, i.e. when the patient did not acquire the target within the timeout period. #### Format of `data` files Raw neural signals sampled at 30kS/s were bandpass filtered between 0.3 and 7500 Hz, then digitally high-pass filtered at 250 Hz. Thresholds for spike detection were estimated at -4.5×RMS. Events sxceeding the threshold were assumed to be unsorted spikes and sent to a separate computer for online processing. Events were converted online to spike rates in by counting events in 1 s long windows, advanced in 50 ms increments, for each channel separately. The resulting 20Hz firing rate data were saved in spike rate data binary files. Spike rate data files are binary files, consisting of a variable number of data records. Each data record has a header of constant size and a variable number of samples. All multi-byte data are stored in little-endian order. Each record consists of the following fields: | Field name | Description | Data Type | Length (bytes) | |:-------------------|:----------------------------------------------------------------------------------------------------------------------------------------------|:-------------------------------------|:-----------------------------| | `Datetime` | Computer's wall clock timestamp | `datetime64[us]` | 8 | | `NSP time` | Timestamp of the data acquisition system in sample clock ticks | `int64` | 8 | | `NBytes` | Number of bytes in following sample block | `uint64` | 8 | | `NSamples` | Number of samples in following sample block | `int64` | 8 | | `NChannels` | Number of channels in following sample block | `int64` | 8 | | `ChListLen` | Number of bytes in following channel list | `int16` | 2 | | `ChList` | List of channels that were recorded. Each channel ID is saved as `uint16`. | `ChListLen` × `uint16` | 2 × `ChListLen` | | `Samples` | Matrix of samples `NSamples` × `NChannels` (samples are stored by sample, i.e. for each sample time, channels are stored one after the other) | `NSamples` × `NChannels` × `float32` | 4 × `NSamples` × `NChannels` | | `SampleTimestamps` | List of timestamps in sample clock ticks, associated with each sample | `NSamples` × `uint64` | 8 × `NSamples` | ### Speller blocks Speller trial structure is described in the paper. A list of blocks is given in the Excel file `DataOverview.xlsx`, sheet 'speller'. The name of the datafile corresponds to the files stored in `KIAP_BCI_speller`. Each block is associated with a spike rate binary data file as well as two log files, an `events_*.txt` and an `info_*.log` file. The `info` file only contains decoder decision events, while `events` contains other events in the trial as well. #### Format of `info` files Lines in the `info` files contain the state of the speller after each decoding: 2019-08-14T14:26:01.132830 color - Decoder decision: no - ('', 'S') 2019-08-14T14:26:05.201561 color - Decoder decision: yes - ('', 'R') 2019-08-14T14:26:13.483555 color - Decoder decision: yes - ('R', 'gelb') 2019-08-14T14:26:17.974635 color - Decoder decision: yes - ('R', 'E') 2019-08-14T14:26:26.235286 color - Decoder decision: no - ('RE', 'gelb') 2019-08-14T14:26:31.320232 color - Decoder decision: no - ('RE', 'gruen') First item in each line is the computer's wall clock timestamp. The decoder decision 'yes', 'no', or 'unclassified' refers to the presented option, which is the second item in the tuple at the end of each line. The first item refers to the speller output so far. For example, in the first line, the speller status was 'empty'. Letter `S` was rejected, letter `R` was selected. The speller state then changes to 'R'. The speller after letter selection proceeds with presentation of letter groups, in this case `gelb` (yellow). Upon its selection, letter `E` was selected, and so on. #### Format of `events` files The format of `events` files is essentially the same as in the neurofeedback case: 30973163, b'color, Free, baseline, start' 31012029, b'color, Free, baseline, stop' 31012029, b'color, Free, stimulus, start' 31057116, b'color, Free, stimulus, stop' 31063329, b'color, Free, response, start' 31079659, b'color, Free, response, stop' 31079659, b'color, Free, Decoder decision is: no' timestamp in sample clock ticks, message. Message consists of `color` (for color speller), `Free` or `Validation`. Blocks marked `Validation` were copy speller blocks, i.e. with a given target to spell (see Excel table). The letter group and letter options were presented during the 'stimulus' period. #### Format of `data` files The format is identical to the one described above. ### Impedances The folder `Impedances` contains six files covering the electrode array impedances over time. The files list channel label and impedance in kΩ. The order of the channel follows the logical order 1–128. ### Spike data Raw Blackrock `nev` files containing unsorted multiunit threshold crossings. One file was acquired before the time covered in the paper, the rest covers the paper period. See Blackrock Neuroport manual for a description of the file format. ## Reproducing of paper figures See the README file in `scripts_for_figures`.