# Speaker confusion model

The Voice Type Classifier (VTC) is a speech detection algorithm that classify each detected speech segment into one of four classes: Female adults, Male adults, the Key child, and Other children. This allows to measure the input children are exposed to as they are acquiring language, as well as their own speech production.

However, the VTC is not perfectly accurate, and the speaker may be misidentified (or, speech may be left undetected).
For instance, the key child is occasionally confused with other children. Not only such confusions have consequences on the amount of speech measured within each of these four categories, but they also tend to generate spurious correlations
between the amount of speech produced by each kind of speaker. For instance, more actual "Key child" (CHI) speech will systematically and spuriously lead to more "Other children" (OCH) detected speech due to speaker misidentification, which in turn will appear as if there was a true correlation between CHI and OCH amounts' of speech.

The present bayesian model captures the errors of the VTC (false negatives and confusion between speakers) in order to assess the impact of such errors on correlation analyses. Such a model, in principle, also allows to recover the unbiased values of the parameters of interest (e.g., the true correlation between two speakers), by translating bias into larger posteriors on the parameters.

## Installation

1. The installation of this repository requires DataLad (``apt install datalad`` on Linux; ``brew install datalad`` on MacOS; read more, including instructions for windows, [here](https://handbook.datalad.org/en/latest/intro/installation.html))

2. Once DataLad is installed, the repository can be installed:

```bash
datalad install -r git@gin.g-node.org:/LAAC-LSCP/speaker-confusion-model.git
```

This requires access to the corpora that are used to train the model.

3. Install necessary packages

```bash
pip install -r requirements.txt
```

## Run the model

### Command-Line Arguments

The model can be found in ``code/models/main.py``. Running the model simultaneously performs the two following tasks:

1. Deriving the confusion rates and their posterior distribution
2. Generating null-hypothesis samples (i.e. assuming no true correlation between speakers' amounts of speech) simulating a 40 children corpora with 8 recorded hours each.

```bash
$ python code/models/main.py  --help
usage: main.py [-h] [--group {corpus,child}] [--chains CHAINS] [--samples SAMPLES] [--validation VALIDATION] [--output OUTPUT]

main model described throughout the notes.

optional arguments:
  -h, --help            show this help message and exit
  --group {corpus,child}
  --chains CHAINS
  --samples SAMPLES
  --validation VALIDATION
  --output OUTPUT
  ```

  The ``--group`` parameter controls the primary level of the hierarchical model. The model indeed assumes that confusion rates (i.e. confusion probabilities) vary across corpora (``corpus``) or children (``child``).

  The ``--chains`` parameter sets the amount of MCMC chains, and ``--samples`` controls the amount of MCMC samples, warmup excluded.

  The ``--validation`` parameter sets the amount of annotation clips used for validation rather than training. Set it to 0 in order to use as much data for training as possible.

  The ``--output`` parameter controls the output destination. Training data will be saved to ``output/samples/data_{output}.pickle`` and the MCMC samples are saved as ``output/samples/fit_{output}.parquet``

  ### Confusion probabilities

  The marginal posterior distribution of the confusion matrix is shown below:

  ![](output/fit_vanuatu.png)


  ### Speech distribution

  Speech distributions used to generate the simulated "null-hypothesis" corpora are fitted against the training data using Gamma distributions. The code used to fit these distributions is found in ``code/models/speech_distribution``. 

  The match between the training data and the Gamma parametrization can be observed in various plots in ``output``. See below for the Key child:

  ![](output/dist_CHI.png)