-
Notifications
You must be signed in to change notification settings - Fork 10
/
Copy pathREADME.Rmd
215 lines (148 loc) · 11.1 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
---
output:
rmarkdown::github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r, echo = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "README-"
)
```
# rdhs <img src="https://raw.githubusercontent.com/ropensci/rdhs/main/tools/logo.png" align="right" style="padding-left:10px;background-color:white;" />
[](https://www.repostatus.org/#active)
[](https://github.com/ropensci/rdhs/actions)
[](https://app.codecov.io/github/ropensci/rdhs?branch=main)
[](https://docs.ropensci.org/rdhs/)
[](https://cran.r-project.org/package=rdhs)
[](https://www.r-pkg.org:443/pkg/rdhs)
[](https://cran.r-project.org/package=rdhs)
[](https://github.com/ropensci/software-review/issues/238)
[](https://doi.org/10.5281/zenodo.2423635)
## Motivation
The Demographic and Health Surveys (DHS) Program has collected population survey data from over 90 countries for over 30 years. In many countries, DHS provide the key data that mark progress towards targets such as the Sustainable Development Goals (SDGs) and inform health policy. Though standard health indicators are routinely published in survey final reports, much of the value of DHS is derived from the ability to download and analyse standardized microdata datasets for subgroup analysis, pooled multi-country analysis, and extended research studies. The suite of tools within `rdhs` improves the accessibility of these datasets for statistical analysis with R, with aim to support reproducible global health research and simplify common analytical pipelines.
> For questions regarding how to analyse DHS survey data, please read the DHS website's data section first. If you have any questions after this then please create an [issue](https://github.com/ropensci/rdhs/issues) with your question. It is really likely that your question will help other people and so posting them publically as an issue may help others with similar questions.
---
`rdhs` is a package for management and analysis of [Demographic and Health Survey (DHS)](https://www.dhsprogram.com) data. This includes functionality to:
1. Access standard indicator data (i.e. [DHS STATcompiler](https://www.statcompiler.com/)) in R via the [DHS API](https://api.dhsprogram.com/).
1. Identify surveys and datasets relevant to a particular analysis.
1. Download survey datasets from the [DHS website](https://dhsprogram.com/data/available-datasets.cfm).
1. Load datasets and associated metadata into R.
1. Extract variables and combining datasets for pooled multi-survey analyses.
## Installation
You can install the latest version from [`CRAN`](https://cran.r-project.org/package=rdhs) using:
```{r cran-installation, message=FALSE, eval = FALSE}
install.packages("rdhs")
```
You can also install the development version of `rdhs` with the latest patches from github with:
```{r gh_installation, message=FALSE, eval = FALSE}
#install.packages("devtools")
devtools::install_github("ropensci/rdhs")
```
```{r}
# Load the package
library(rdhs)
```
## Getting started
To be able to **download survey datasets from the DHS website**, you will need to **set up an account with the DHS website**, which will enable you to request access to the datasets. Instructions on how to do this can be found [here](https://dhsprogram.com/data/Access-Instructions.cfm). The email, password, and project name that were used to create the account will then need to be provided to `rdhs` when attempting to download datasets.
---
* Request dataset access from the DHS website [here](https://dhsprogram.com/data/Access-Instructions.cfm).
* Full functionality is described in the tutorial [here](https://docs.ropensci.org/rdhs/articles/introduction.html).
* An example workflow using `rdhs` to calculate trends in anemia prevalence is available [here](https://docs.ropensci.org/rdhs/articles/anemia.html).
## Basic Functionality
### Query the [DHS API](https://api.dhsprogram.com/).
Obtain survey estimates for Malaria prevalence among children from the Democratic Republic of Congo and Tanzania in the last 5 years (since 2013) that included rapid diagnostic tests (RDTs).
```{r api, message = FALSE}
dhs_indicators(indicatorIds = "ML_PMAL_C_RDT", returnFields=c("IndicatorId", "ShortName"))
dhs_data(countryIds = c("CD","TZ"), indicatorIds = "ML_PMAL_C_RDT", surveyYearStart = 2013,
returnFields=c("Indicator", "SurveyId", "Value", "SurveyYearLabel", "CountryName"))
```
### Identify survey datasets
Now, obtain survey microdatasets to analyze these same indicators. Query the *surveyCharacteristics* endpoint to identify the survey characteristic ID for malaria RDT testing.
```{r sc}
## call with no arguments to return all characterstics
sc <- dhs_survey_characteristics()
sc[grepl("Malaria", sc$SurveyCharacteristicName), ]
```
Use `dhs_surveys()` identify surveys for the countries and years of interest.
```{r surv}
## what are the countryIds - we can find that using this API request
ids <- dhs_countries(returnFields=c("CountryName", "DHS_CountryCode"))
## find all the surveys that match the search criteria
survs <- dhs_surveys(surveyCharacteristicIds = 89, countryIds = c("CD","TZ"), surveyYearStart = 2013)
```
Lastly, identify the datasets required for download. By default, the recommended option is to download either the spss (.sav), `fileFormat = "SV"`, or the flat file (.dat), `fileFormat = "FL"` datasets. The flat is quicker, but there are still one or two very old datasets that don't read correctly, whereas the .sav files are slower to read in but so far no datasets have been found that don't read in correctly. The household member recode (`PR`) reports the RDT status for children under five.
```{r }
datasets <- dhs_datasets(surveyIds = survs$SurveyId, fileFormat = "FL", fileType = "PR")
str(datasets)
```
### Download datasets
We can now go ahead and download our datasets. To be able to download survey datasets from the DHS website, you will need to set up an account with them to enable you to request access to the datasets. Instructions on how to do this can be found [here](https://dhsprogram.com/data/Access-Instructions.cfm). The email, password, and project name that were used to create the account will then need to be provided to `rdhs` when attempting to download datasets.
Once we have created an account, we need to set up our credentials using the function `set_rdhs_config()`. This will require providing as arguments your `email` and `project` for which you want to download datasets from. You will then be prompted for your password.
You can also specify a directory for datasets and API calls to be cached to using `cache_path`. In order to comply with CRAN, this function will also ask you for your permission to write to files outside your temporary directory, and you must type out the filename for the `config_path` - "rdhs.json". (See [introduction vignette](https://docs.ropensci.org/rdhs/articles/introduction.html) for specific format for config, or `?set_rdhs_config`).
```{r client , R.options = list("rappdir_permission" = TRUE)}
## login
set_rdhs_config(email = "rdhs.tester@gmail.com",
project = "rdhs R package development",
config_path = "rdhs.json",
global = FALSE)
```
The path to your config is saved between sessions so you only have to set this once. With your credentials set, all API requests will be cached within the `cache_path` directory provided so that these can be returned when working remotely or with a poor internet connection.
```{r client_api_cache}
# the first time this will take a few seconds
microbenchmark::microbenchmark(dhs_datasets(surveyYearStart = 1986),times = 1)
# after caching, results will be available instantly
microbenchmark::microbenchmark(dhs_datasets(surveyYearStart = 1986),times = 1)
```
Now download datasets by providing a list of desired dataset filenames.
```{r download, message=FALSE}
# download datasets
downloads <- get_datasets(datasets$FileName)
str(downloads)
```
### Load datasets into R
The `get_datasets()` function returns a vector with a file path to the saved location of the downloaded datasets. These are read using `readRDS()`:
```{r read a dataset}
# read in first dataset
cdpr <- readRDS(downloads$CDPR61FL)
```
Value labels are stored as attributes to each of the columns of the data frame using the `labelled` class (see `haven::labelled` or our introduction vignette for more details). Variable labels are stored in the `label` attribute.
### Extract variables and pool datasets
The client also caches all variable labels to quickly query variables in each survey *without* loading the datasets.
```{r questions}
# rapid diagnostic test search
vars <- search_variable_labels(datasets$FileName, search_terms = "malaria rapid test")
```
Then extract these variables from the datasets. Optionally, geographic data may be added.
```{r extract_questions}
# and now extract the data
extract <- extract_dhs(vars, add_geo = FALSE)
```
The returned object is a list of extracted datasets.
Dataset extracts can alternate be specified by providing a vector of surveys and vector of variable names:
```{r extract_variables}
# and grab the questions from this now utilising the survey variables
vars <- search_variables(datasets$FileName, variables = c("hv024","hml35"))
# and now extract the data
extract <- extract_dhs(vars, add_geo = FALSE)
```
Finally, the two datasets are pooled using the function `rbind_labelled()`. This function works specifically with our lists of labelled `data.frame`s. Labels are specified for each variable: for `hv024` all labels are retained (concatenate) but for `hml35` labels across both datasets to be "Neg" and "Pos".
```{r rbind_labelled}
# now let's try our second extraction
extract <- rbind_labelled(extract,
labels = list("hv024" = "concatenate",
"hml35" = c("Neg"=0, "Pos"=1)))
```
There is also an option to process downloaded datasets with labelled variables coded as strings, rather than labelled variables. This is specified by the argument `reformat=TRUE`.
```{r reformat}
# identify questions but specifying the reformat argument
questions <- search_variables(datasets$FileName, variables = c("hv024", "hml35"),
reformat=TRUE)
# and now extract the data
extract <- extract_dhs(questions, add_geo = FALSE)
# group our results
extract <- rbind_labelled(extract)
# our hv024 variable is now just character strings, so you can decide when/how to factor/label it later
str(extract)
```