Browse Source

started making changes, but I think this is not the latest version

alecristia 5 months ago
parent
commit
e7ed54161e
1 changed files with 9 additions and 7 deletions
  1. 9 7
      CODE/SM.Rmd

+ 9 - 7
CODE/SM.Rmd

@@ -267,13 +267,13 @@ df.icc.mixed[df.icc.mixed$formula=="no_exp",c("data_set","metric")]
 
 ```
 
-Out of the `r dim(df.icc.mixed)[1]` fitted models, `r table(df.icc.mixed$formula)["full"]` could be fit with the full model, yielding a measure of Corpus ICC. Of these, `r round(sum(df.icc.mixed$icc_corpus<.2,na.rm=T)/sum(!is.na(df.icc.mixed$icc_corpus))*100)`% had Corpus ICCs smaller than .2, consistent with the idea that LENA and ACLEW metrics are robust to corpus differences. For the `r table(df.icc.mixed$formula)["no_exp"]` for which the full model was singular, we fit the data with the No Corpus model, and none was singular then, allowing us to have Child ICC for all `r dim(df.icc.mixed)[1]` metrics. 
+Out of the `r dim(df.icc.mixed)[1]` fitted models, we could fit `r table(df.icc.mixed$formula)["full"]` with the full model, yielding a measure of Corpus ICC. Of these, `r round(sum(df.icc.mixed$icc_corpus<.2,na.rm=T)/sum(!is.na(df.icc.mixed$icc_corpus))*100)`% had Corpus ICCs smaller than .2, consistent with the idea that LENA and ACLEW metrics are robust to corpus differences. For the `r table(df.icc.mixed$formula)["no_exp"]` for which the full model was singular, we fit the data with the No Corpus model, and none was singular then, yielding Child ICCs for all `r dim(df.icc.mixed)[1]` metrics. 
 
 
 
-## Code to reproduce Fig. 3
+## Code to reproduce Fig. 4
 
-```{r icc-allexp-fig3, echo=F,fig.width=4, fig.height=3,fig.cap="Distribution of ICC attributed to corpus (a) and children (b), when combining data from all corpora."}
+```{r icc-allexp-fig4, echo=F,fig.width=4, fig.height=3,fig.cap="Distribution of ICC attributed to corpus (a) and children (b), when combining data from all corpora."}
 
 df.icc.mixed$Type<-get_type(df.icc.mixed)
 
@@ -288,7 +288,7 @@ ggplot(df.icc.mixed, aes(y = icc_child_id, x = toupper(data_set))) +
 
 
 
-## Code to reproduce text under Figure 3
+## Code to reproduce text under Figure 4
 
 ```{r reg model icc}
 
@@ -343,6 +343,8 @@ read.csv("../input/quechua_md.csv")->x
 
 ```
 
+Figure 5 shows the distribution of Child ICC across all 69 metrics, separately for each pipeline. The majority of metrics had Child ICCs between .3 and .5. Seven metrics had Child ICCs higher or equal to .5. Surprisingly, the top 6 metrics in terms of Child ICC corresponded to the "other child" category, reported to have the worst accuracy according to previous analyses (Cristia et al., 2020). In an analysis fully reported in supplementary materials (SM J), we find some evidence that this may be due to the presence versus absence of siblings. The next metric with the highest Child ICC corresponded to an output measure, namely the total vocalization duration per hour extracted from ACLEW annotations, with a Child ICC of .5. Among adult metrics, the average vocalization duration for female vocalizations for ACLEW and the ACLEW equivalent of CTC had the highest Child ICC (.45 and .46, respectively). 
+
 
 Six measures had higher ICCs, and surprisingly, they corresponded to the "other child" category, known to have the worst accuracy according to previous analyses (Cristia et al., 2020). We reasoned this may be because children in our corpora vary in terms of the number of siblings they have, and that siblings' presence may be stable across recordings. To address this possibility, we fit the full model again to predict number of vocalizations from other children, but this time including sibling number as a fixed effect $lmer(metric~ age + sibling_number + (1|corpus/child))$, so that individual variation that was actually due to sibling number was captured by that fixed effect instead of the random effect for child. We did this for the metric with the highest Child ICC, ACLEW's total vocalization duration by other children. Results indicated the full model was singular, a first sign that included variables explained shared variance. When we fitted the No Corpus model, Child ICC was indeed reduced from `r round(df.icc.mixed[df.icc.mixed$metric=="voc_dur_och_ph" & df.icc.mixed$data_set=="aclew","icc_child_id"],2)` to `r round(icc.result.split["icc_child_id"],2)`.
 
@@ -450,9 +452,9 @@ mytab=gsub("0.",".",mytab,fixed=T)
 print(mytab)
 ```
 
-## Code to reproduce Fig. 4
+## Code to reproduce Fig. 5
 
-```{r r-fig4, echo=F,fig.width=4, fig.height=3,fig.cap="Distribution of correlation coefficients."}
+```{r r-fig5, echo=F,fig.width=4, fig.height=3,fig.cap="Distribution of correlation coefficients."}
 rval_tab$Type<-get_type(rval_tab)
 
 
@@ -479,7 +481,7 @@ reg_anova_cor=Anova(lr_icc_chi)
 
 ```
 
-To see whether correlations in this analysis differed by talker types and pipelines, we fit a linear model with the formula lm(cor ~ type * pipeline), where type indicates whether the measure pertained to the key child, (female/male) adults, other children; and pipeline LENA or ACLEW. We found an adjusted R-squared of `r round(reg_sum_cor$adj.r.squared*100)`%, suggesting this model did not explain a great deal of variance in correlation coefficients. Moreover, a Type 3 ANOVA on this model revealed no significant effects or interactions (all p's > .1). See SMXX for fuller results.
+To see whether correlations in this analysis differed by talker types and pipelines, we fit a linear model with the formula lm(cor ~ type * pipeline), where type indicates whether the measure pertained to the key child, (female/male) adults, other children; and pipeline LENA or ACLEW. We found an adjusted R-squared of `r round(reg_sum_cor$adj.r.squared*100)`%, suggesting this model did not explain a great deal of variance in correlation coefficients. Moreover, a Type 3 ANOVA on this model revealed no significant effects or interactions (all p's > .1). 
 
 ```{r print out anova results rec on cor}
 kable(reg_anova_cor)