-
Notifications
You must be signed in to change notification settings - Fork 0
/
README.Rmd
175 lines (126 loc) · 8.44 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
```
# sunflower: A Package to Assess and Categorize Language Production Errors
<!-- badges start -->
 [](https://www.gnu.org/licenses/gpl-3.0) 
<!-- badges end -->
<div align="justify">
*sunflower* is a package designed to assist clinicians and researchers in the fields of Speech Therapy and Neuropsychology of Language. Its primary goal is to facilitate the management of multiple response data and compute formal similarity indices to assess the quality of oral and written productions in patients with aphasia and related disorders, such as apraxia of speech, in Spanish. Additionally, the package allows for the classification of these productions according to classical typologies in the field, prior to computing formal and semantic similarity measures. For the computation of the latter, *sunflower* partially relies on natural language processing models such as word2vec. The outputs provided by this package facilitate statistical analyses in R, a widely-used tool in the field for data wrangling, visualization, and analysis.
## Installation
*sunflower* can be installed as an R package with:
```r
install.packages("devtools")
devtools::install_github("ismaelgutier/sunflower")
```
The *sunflower* package works using the pipe operator (`%>%`) from the [*tidyverse* package](https://www.tidyverse.org/), allowing it to work seamlessly with functions from other packages in the *tidyverse*, such as *dplyr* for data wrangling, *readr* for data reading, and *ggplot2* for data visualization. This can significantly enhance our workflow.
## How to use
```{r include=FALSE}
require("sunflower")
require("tidyverse")
require("htmltools")
```
### Loading the packages
Once installed, we only need to load the *sunflower* package. However, as previously mentioned, the *tidyverse* package can also be valuable for other complementary tasks.
```r
require("sunflower")
require("tidyverse")
```
### Compute Formal Quality Indexes
```{r}
df_to_formal_metrics = sunflower::IGC_long_phon %>%
dplyr::select(-c(modality, task_modality,task_type, test, task)) # keep some relevant columns
formal_metrics_computed = df_to_formal_metrics %>%
get_formal_similarity(item_col = "item",
response_col = "response",
attempt_col = "Attempt",
group_cols = c("ID", "item_ID"))
formal_metrics_computed %>% head(8) %>% knitr::kable()
```
`Note`: Move the dataframe to the right to see all the columns and metrics.
### Obtain Positional Accuracy Data
```{r message=FALSE}
positions_accuracy = formal_metrics_computed %>%
positional_accuracy(match_col = "itemL_adj_strict_match_pos", last_ID_col = "targetL")
positions_accuracy %>% head(8) %>% knitr::kable()
```
```{r include=FALSE}
# Convertir targetL a character para evitar problemas al combinar dataframes
positions_accuracy <- positions_accuracy %>%
mutate(targetL = as.character(targetL))
# Duplicar y modificar el dataframe para crear 'positions_general'
positions_general <- positions_accuracy %>%
mutate(targetL = "General")
# Combinar ambos dataframes
positions <- bind_rows(positions_accuracy, positions_general)
# Especificar manualmente los niveles en el orden deseado
desired_levels <- c("3", "4", "5", "6", "7", "8", "9", "10", "11", "12",
"13", "14", "15", "17", "21", "22", "24", "48", "General")
# Convertir correct_pos a numérico y ordenar targetL como factor según desired_levels
positions <- positions %>%
mutate(correct_pos = as.numeric(correct_pos),
targetL = factor(targetL, levels = desired_levels)) %>%
arrange(correct_pos, targetL)
# Definir un conjunto de linetypes que se pueda repetir
custom_linetypes <- rep(c("solid", "dashed", "dotted", "longdash", "dotdash"),
length.out = nlevels(positions$targetL))
# Calcular la precisión y contar el número de observaciones por grupo
plot_positions <- positions %>%
group_by(Position, targetL) %>%
summarize(acc = mean(correct_pos, na.rm = TRUE),
n = n()) %>%
ggplot(aes(x = as.numeric(Position), y = acc, group = targetL,
fill = targetL, color = targetL, lty = targetL)) +
geom_line(size = 0.70, alpha = 0.6) +
geom_point(aes(size = n), shape = 21, color = "black", alpha = 0.6) +
scale_linetype_manual(values = custom_linetypes) +
theme(panel.border = element_rect(colour = "black", fill = NA),
panel.background = element_blank(),
panel.grid.major = element_blank(),
panel.grid.minor = element_blank(),
axis.line = element_blank()) +
ylab("Proportion (%) of correct phonemes") +
xlab("Phoneme position") +
guides(fill = guide_legend(title = "Word Length"),
lty = guide_legend(title = "Word Length"),
color = guide_legend(title = "Word Length"),
size = guide_legend(title = "Datapoints")) +
theme_gray() +
theme(legend.position = "right",
legend.key.size = unit(0.6, "lines"))
```
***Note.*** A plot depicting the positions' accuracy of 14,418 datapoints.
```{r plot_positions, out.width = "75%", fig.align="center", echo=FALSE}
plot_positions
```
### Classify Errors
```{r include=FALSE}
df_to_classify = sunflower::IGC_long %>% select(-c(modality, task_modality,task_type, test, task))
m_w2v = word2vec::read.word2vec(file = file.choose(), normalize = TRUE)
```
```{r}
errors_classified = df_to_classify %>%
check_lexicality(item_col = "item", response_col = "Response", criterion = "database") %>%
get_formal_similarity(item_col = "item", response_col = "Response",
attempt_col = "Attempt", group_cols = c("ID", "item_ID")) %>%
get_semantic_similarity(item_col = "item", response_col = "Response", model = m_w2v) %>%
classify_errors(response_col = "Response", item_col = "item",
access_col = "accessed", RA_col = "RA", also_classify_RAs = T)
errors_classified %>% head(8) %>% knitr::kable()
```
***Notes.*** Move the dataframe to the right to see all the columns and errors.
*sunflower* allows for the classification of production errors once some indexes related to responses to a stimulus have been obtained and contextualized based on whether they come from repeated attempts or single productions. This process involves three steps. First, a lexicality check of the response is performed using the `lexicality_check()` function, which involves determining whether the response is a real word. To do this, the package searches for the response in a database such as *BuscaPalabras* ([BPal](https://www.uv.es/~mperea/Davis_Perea_in_press.pdf)) and compares its frequency with the target word to determine if it is a real word based on whether it has a higher frequency or not when the parameter `criterion = "database"` is set. Alternatively, the response can be checked against a dictionary (*sunflower* searches for responses among entries from the *Real Academia Española*, [RAE](https://www.rae.es/)) when the parameter `criterion = "dictionary"` is used.
Next, similarity measures between the targets and the responses are obtained using various algorithms within the `get_formal_similarity()` function. Finally, the cosine similarity between the two productions is computed if possible using the `get_semantic_similarity()` function, based on an NLP model. In our case, the parameter `model = m_w2v` refers to a binary file containing a Spanish Billion Words embeddings corpus created using the word2vec algorithm. This file is included in the zip file (for more information, see the markdown in the vignettes) located within the dependency-bundle zip, which can be found in our supplementary [OSF repository mirror](https://osf.io/akuxv/).
----
Thanks to Cristian Cardellino for making his work on the [Spanish Billion Word Corpus and Embeddings](https://crscardellino.github.io/SBWCE/) publicly available.
----
Any suggestions, comments, or questions about the package's functionality are warmly welcomed. If you’d like to contribute to the project, please feel free to get in touch. 🌻