first commit

Data Pre-processing.ipynb

@@ -0,0 +1,240 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Import libraries\n",
+    "\n",
+    "Make sure that R and the following libraries (recommended version ID) are installed on your computer before running this cell:\n",
+    "\n",
+    "R.matlab (3.6.2),\n",
+    "xlsx (0.6.1)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "library(R.matlab)\n",
+    "library(xlsx)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Read data for processing"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Enter the name of the file to be processed\n",
+    "dataFull <- read.xlsx('SubjectID_ExpType_MM_DD_20YY_protocol.xlsx',1)\n",
+    "\n",
+    "# Preview of the data frame\n",
+    "head(dataFull)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Add some information about the experiment\n",
+    "This information will later be included in the output files and their names"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Manually enter some information \n",
+    "\n",
+    "dateClock    = \"MM-DD-20YY-HH-\"       # Date and hour\n",
+    "stayTime     = 1                      # Target duration\n",
+    "subject      = \"e.g. subject's name\"  # Full ID of the animal\n",
+    "subjectID    = \"ML001\"                # Short ID of the animal (here: Mouse Lemur 001)\n",
+    "timeout      = 60                     # Trial duration\n",
+    "xArena       = 400                    # X-coordinate of the arena\n",
+    "yArena       = 300                    # Y-coordinate of the arena\n",
+    "radArena     = 300                    # Arena radius\n",
+    "trialCounter = 1                      # Should start at 1              "
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Detect indivudal trials in your data frame"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Creates empty vectors for trial information\n",
+    "trialStart    = c()        # Time [s]\n",
+    "trialStop     = c()        # Time [s]\n",
+    "trialStartT   = c()        # Time [Frame ID]\n",
+    "trialStopT    = c()        # Time [Frame ID]\n",
+    "trialFinished = c()        # 1 = TRUE; 0 = FALSE\n",
+    "\n",
+    "# Scans the data frame for trial\n",
+    "for (t in 2:nrow(dataFull)) {\n",
+    "  \n",
+    "  # Changes from phase 1 to 2 = trial initiated\n",
+    "  if (dataFull$Phase[t] == 2 && dataFull$Phase[t-1] == 1)  {\n",
+    "    trialStart   <- c(trialStart, dataFull$Time..s.[t])\n",
+    "    trialStartT  <- c(trialStartT, t)\n",
+    "  }\n",
+    "  # Changes from phase 3 to 0 = trial finished successfully\n",
+    "  if (dataFull$Phase[t] == 0 && dataFull$Phase[t-1] == 3)  {\n",
+    "    trialStop <- c(trialStop, dataFull$Time..s.[t-1])\n",
+    "    trialStopT  <- c(trialStopT, t-1)\n",
+    "    trialFinished <- c(trialFinished, 1)\n",
+    "  }\n",
+    "  # Changes from phase 2 to 0 = trial finished without reward\n",
+    "  if (dataFull$Phase[t] == 0 && dataFull$Phase[t-1] == 2)  {\n",
+    "    trialStop <- c(trialStop, dataFull$Time..s.[t-1])\n",
+    "    trialStopT  <- c(trialStopT, t-1)\n",
+    "    trialFinished <- c(trialFinished, 0)\n",
+    "  }\n",
+    "}\n",
+    "\n",
+    "# Removes trials that were unfinished at session end\n",
+    "if (length(trialStart)>length(trialStop)) {\n",
+    "  trialStart   <- trialStart[1:length(trialStop)]\n",
+    "  trialStartT  <- trialStartT[1:length(trialStop)]\n",
+    "}\n",
+    "\n",
+    "# Shows the lengths of all vectors (should all be equal)\n",
+    "length(trialStart)\n",
+    "length(trialStop)\n",
+    "length(trialStartT)\n",
+    "length(trialStopT)\n",
+    "length(trialFinished)\n",
+    "\n",
+    "# Determines the number of finished trials\n",
+    "no_trials = length(trialFinished)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Generate output files\n",
+    "\n",
+    "This last cell generates an individual MATLAB file for each trial, using the formatting and nomenclature of the original MATLAB script from XYZ et al. 2020 (doi:XYZ). The output files can be fed to MATLAB for final analysis as described in XYZ et al. 2020 (doi:XYZ)."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Separates the trials and generates an individual output file (= Matlab input) for every trial\n",
+    "for(i in 1:no_trials) { \n",
+    "  nam <- paste(\"Trial\", i, sep = \"\")\n",
+    "  assign(nam, dataFull[trialStartT[i]:(trialStopT[i]+1),]) \n",
+    "  trialData <- eval(parse(text = nam))\n",
+    "  g = nrow(trialData) \n",
+    "  \n",
+    "  # Adds success marker\n",
+    "  if (trialFinished[i] == 1) {\n",
+    "    trialData$Animal_x[g] = 5000\n",
+    "    trialData$Animal_y[g] = 5000\n",
+    "  }\n",
+    "  \n",
+    "  # Adds fail marker\n",
+    "  else if (trialFinished[i] == 0) {\n",
+    "    trialData$Animal_x[g] = NaN\n",
+    "    trialData$Animal_y[g] = NaN\n",
+    "  }\n",
+    "  \n",
+    "  # Defines values \n",
+    "  xTarget        = trialData$Target_x[1]\n",
+    "  yTarget        = trialData$Target_y[1]\n",
+    "  radTarget      = trialData$Target_rad[1]\n",
+    "  xSubject       = trialData$Animal_x\n",
+    "  ySubject       = trialData$Animal_y\n",
+    "  xStart         = trialData$Start_x[1]\n",
+    "  yStart         = trialData$Start_y[1]\n",
+    "  radStart       = trialData$Start_rad[1]\n",
+    "  timeStamps     = trialData$Time..s.\n",
+    "  timeStampsNorm = timeStamps-timeStamps[1]\n",
+    "  majAxis     <- rep(1000, length(ySubject))\n",
+    "  minAxis     <- rep(1000, length(ySubject))\n",
+    "  detected    <- rep(1000, length(ySubject))\n",
+    "  orient      <- rep(1000, length(ySubject))\n",
+    "  soundPlayed <- rep(1000, length(ySubject))\n",
+    "  \n",
+    "  # Generates a standardized file name for the current output file\n",
+    "  if (trialCounter < 10) {\n",
+    "    fileName = paste(\"Pos_\",subjectID,\"-\",dateClock,\"0\",trialCounter,\"-00.mat\", sep = \"\", collapse = NULL)\n",
+    "  }\n",
+    "  else if (trialCounter >= 10) {\n",
+    "    fileName = paste(\"Pos_\",subjectID,\"-\",dateClock,trialCounter,\"-00.mat\", sep = \"\", collapse = NULL)\n",
+    "  }\n",
+    "  trialCounter = trialCounter+1\n",
+    "  \n",
+    "  # Writes a MATLAB file for the current trial\n",
+    "  writeMat(fileName, \n",
+    "           \"wid\" = array(c(radTarget), dim = c(1,1)),\n",
+    "           \"staytime\" = array(c(stayTime), dim = c(1,1)),\n",
+    "           \"Cx\" = array(c(xArena), dim = c(1,1)),\n",
+    "           \"Cy\" = array(c(yArena), dim = c(1,1)),\n",
+    "           \"Cwid\" = array(c(radArena), dim = c(1,1)), \n",
+    "           \"X.plat\" = array(c(xStart), dim = c(1,1)),\n",
+    "           \"Y.plat\" = array(c(yStart), dim = c(1,1)),\n",
+    "           \"wid.plat\" = array(c(radStart), dim = c(1,1)),\n",
+    "           \"timeout\" = array(c(timeout), dim = c(1,1)),\n",
+    "           \"freq\" = array(c(145), dim = c(1,1)), \n",
+    "           \"Xt\" = array(c(xTarget), dim = c(1,1)),\n",
+    "           \"Yt\" = array(c(yTarget), dim = c(1,1)),\n",
+    "           \"XYs\" = array(c(xSubject,ySubject), dim = c(length(ySubject),2)), \n",
+    "           \"timestamps\" = array(c(timeStampsNorm), dim = c(length(ySubject),1)),\n",
+    "           \"maj.ax\" = array(c(majAxis), dim = c(length(ySubject),1)),\n",
+    "           \"min.ax\" = array(c(minAxis), dim = c(length(ySubject),1)),\n",
+    "           \"detected\" = array(c(detected), dim = c(length(ySubject),1)),\n",
+    "           \"orient\" = array(c(majAxis), dim = c(length(ySubject),1)),\n",
+    "           \"sound.played\" = array(c(soundPlayed), dim = c(length(ySubject),1)))\n",
+    "}"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "R",
+   "language": "R",
+   "name": "ir"
+  },
+  "language_info": {
+   "codemirror_mode": "r",
+   "file_extension": ".r",
+   "mimetype": "text/x-r-source",
+   "name": "R",
+   "pygments_lexer": "r",
+   "version": "3.6.1"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}

README.md

@@ -0,0 +1,57 @@
+# Sensory Island Task (SIT)
+
+The project provides several Jupyter notebooks for the behavioral testing of animals in open-field-based sensory perception  experiments, the Sensory Island Task, as described in Ferreiro et al. 2020 (doi:XYZ **TODO**) and the pre-processing of the output data for subsequent analysis in MATLAB.
+
+### Installation
+
+This project was developed and tested on different Windows (Win7 and Win10) machines. We recommend using Anaconda to run the project notebooks.
+
+To install Anaconda, download the Anaconda installer with Python 3 (tested with Anaconda3-2019.10, bundled with Python 3.7) and follow the instructions on the download page.
+
+After installation, we recommend creating an Anaconda environment from which Jupyter Notebook will be started to run the provided notebooks. You can create new environments using Anaconda's navigator or from an Anaconda prompt using:
+
+    conda create --name myenv
+
+Once created, activate the environment and make sure it has both, Python 3 and R installed. Usually, you will also have to install an R-Kernel for Jupyter Notebook. To do so, open an Anaconda promt and go to the environment you created:
+
+    activate myenv
+
+Once the environment has been activated, install R, its essentials, and the kernel via:
+
+    conda install r-base
+    conda install r-essentials 
+    conda install -c r r-irkernel
+
+Finally, for the notebooks to run, you will have to install additional python modules and R libraries and their dependencies to your environment. Notebooks may not run smoothly, if the wrong versions of the modules are installed. Especially having the wrong versions of openCV can cause trouble, which is why we provide the Jupyter notebooks that run on Python 3 for two different versions of openCV (3.4.2 & 4.0.1). The code below will install the latest version of the modules/libraries to your environment. Version numbers in brackets are the latest module/library versions the notebooks were tested with (5th of Febuary, 2020).
+
+**Python modules:** numpy (1.18.1), openCV (3.4.2 or 4.0.1), tkinter (8.6.8), xlxswriter (1.2.7), scipy (1.3.2), sounddevice (0.3.14), pyfirmata (1.1.0)
+
+    conda install numpy
+    conda install opencv
+    conda install tk
+    conda install xlsxwriter
+    conda install scipy
+    conda install -c conda-forge python-sounddevice
+    conda install -c conda-forge pyfirmata
+
+**R libraries:** R.matlab (3.6.2), xlsx (0.6.1)
+
+    conda install -c r r-R.matlab
+    conda install -c r r-xlsx
+
+Once you have installed the modules and libraries, run Juypter Notebook from your environment
+
+    jupyter notebook
+
+To work with the different notebooks, select the folder to which you downloaded the files of this repository in the Jupyter browser and follow the instruction provided in the respective notebook.
+
+## Authors
+
+* **Daniel Schmidtke** 
+
+## License
+
+This project is licensed under the MIT License (see the license file **TODO** for details). 
+## Acknowledgments
+
+* The implemented tracking algorithm was inspired by **colinlaney**'s animal-tracking at: https://github.com/colinlaney/animal-tracking