-
Notifications
You must be signed in to change notification settings - Fork 1
/
README.Rmd
164 lines (103 loc) · 6.96 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
---
output: github_document
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
```{r, include = FALSE}
knitr::opts_chunk$set(
collapse = TRUE,
comment = "#>",
fig.path = "man/figures/README-",
out.width = "100%"
)
```
# czso <img src='man/figures/logo.png' align="right" height="138" />
<!-- badges: start -->
[![CRAN status](https://www.r-pkg.org/badges/version/czso)](https://CRAN.R-project.org/package=czso)
[![CRAN downloads](https://cranlogs.r-pkg.org/badges/grand-total/czso)](https://CRAN.R-project.org/package=czso)
[![CRAN monthly downloads](https://cranlogs.r-pkg.org/badges/last-month/czso)](https://CRAN.R-project.org/package=czso)
[![Lifecycle: maturing](https://img.shields.io/badge/lifecycle-maturing-blue.svg)](https://lifecycle.r-lib.org/articles/stages.html)
[![Mentioned in Awesome Official Statistics ](https://awesome.re/mentioned-badge.svg)](https://github.com/SNStatComp/awesome-official-statistics-software)
[![R-CMD-check](https://github.com/petrbouchal/czso/actions/workflows/R-CMD-check.yaml/badge.svg)](https://github.com/petrbouchal/czso/actions/workflows/R-CMD-check.yaml)
<!-- badges: end -->
The goal of czso is to provide direct, programmatic, hassle-free access from R to open data provided by the Czech Statistical Office (CZSO).
This is done by
1. **providing direct access from R to the catalogue of open CZSO datasets**, eliminating the hassle from data discovery. Normally this is done done through [the CZSO's product catalogue](https://www.czso.cz/csu/czso/otevrena-data-v-katalogu-produktu-csu) which is unfortunately a bit clunky, or [data.gov.cz](https://data.gov.cz), which is not a natural starting point for many.
2. **providing a function to load a specific dataset to R** directly from the CZSO's datastore, eliminating the friction of copying a URL, downloading, unzipping etc.
Additionally, the package provides access to metadata on datasets and to codelists (číselníky) as a special case of datasets listed in the catalogue.
## Installation
You can install the package from CRAN:
``` r
install.packages("czso")
```
You can install the latest in-development release from [github](https://github.com/petrbouchal/czso) with:
``` r
remotes::install_github("petrbouchal/czso", ref = github_release())
```
or the latest version with:
``` r
remotes::install_github("petrbouchal/czso")
```
I also keep binaries in a `drat` repo, which you can access by
```
install.packages("czso", repos = "https://petrbouchal.xyz/drat")
```
## Example
Say you are looking for a dataset whose title refers to wages (mzda/mzdy):
First, retrieve the list of available CZSO datasets:
```{r example-catalogue}
library(czso)
suppressPackageStartupMessages(library(dplyr))
suppressPackageStartupMessages(library(stringr))
catalogue <- czso_get_catalogue()
```
Now search for your terms of interest in the dataset titles:
```{r example-filter}
catalogue %>%
filter(str_detect(title, "[Mm]zd[ay]")) %>%
select(dataset_id, title, description)
```
You could also search in descriptions or keywords which are also retrieved into the catalogue.
We can see the `dataset_id` for the required dataset - now use it to get the dataset:
```{r example-cont}
czso_get_table("110080")
```
You can retrieve the schema for the dataset:
```{r example-schema}
czso_get_table_schema("110080")
```
and download the documentation in PDF:
```{r example-doc}
czso_get_dataset_doc("110080", action = "download", format = "pdf")
```
If you are interested in linking this data to different data, you might need the NUTS codes for regions. Seeing that the lines with regional breakdown list `uzemi_cis` as `"100"`, you can get that codelist (číselník):
```{r example-codelist}
czso_get_codelist(100)
```
You would then need to do a bit of manual work to join this codelist onto the data.
### A note about "tables" and "datasets"
In the parlance of the official open data catalogue, a `dataset` can have multiple distributions (typically multiple formats of the same data). These are called resources in the internals, and manifest as tables in this package. Some metainformation is the property of a dataset (the documentation), while other - the schema - is the property of a table. Hence the function names in this package. This is to keep things organised even if the CZSO almost always provides only one table per dataset and appends new data to it over time.
## Data sources
The catalogue is drawn from https://data.gov.cz through the [SPARQL endpoint](https://data.gov.cz/sparql).
The data and specific metadata is then accessed via the `package_show` endpoint of the CZSO API at (example) https://vdb.czso.cz/pll/eweb/package_show?id=290038r19.
## Credit and notes
- not created or endorsed by the Czech Statistical Office, though they, as well as [the open data team at the Ministry of Interior](https://data.gov.cz/) deserve credit for getting the data out there.
- the package relies on the data.gov.cz catalogue of open data and on the CZSO's local catalogue
- NB: The robots.txt at the domain hosting the CZSO's catalogue prohibits robots from accessing it; while this may be an inappropriate/erroneous setting for what is in essence a data API, this package tries to honor the spirit of that setting by only accessing the API once per `czso_get_table()` call, relying on a different system for `czso_get_catalogue()`. Hence, *do not use this package for harvesting large numbers of datasets from the CZSO.*
### Acknowledgments
Thanks to @jakubklimek and @martinnecasky for [helping me figure out](https://github.com/datagov-cz/nkod/issues/19) the [SPARQL endpoint](https://data.gov.cz/sparql) on the Czech National Open Data Catalogue.
### The logo
An homage to the CZSO's work in releasing its data in an open format, something that is not necessarily in its DNA.
It alludes to the shades of the country reflected in the tabular data provided, By interspersing the comma symbol into the name of the package, it refers to both integration between statistics and open data and the slight disruption that the world of statistics undergoes when that integration happens.
## See also
This package takes inspiration from the packages
- [eurostat](https://github.com/rOpenGov/eurostat/)
- [OECD](https://github.com/expersso/OECD)
which are very useful in their own right - much recommended.
For Czech geospatial data, see [CzechData](https://github.com/JanCaha/CzechData/) by [JanCaha](https://github.com/JanCaha/).
For Czech fiscal data, see [statnipokladna](https://github.com/petrbouchal/statnipokladna).
For various transparency disclosures, see [Hlídač státu](https://www.hlidacstatu.cz/) and the [{hlidacr}](https://cran.r-project.org/package=hlidacr) package.
For access to some of Prague's open geospatial data in R, see [pragr](https://github.com/petrbouchal/pragr).
## Contributing / code of conduct
Please note that the 'czso' project is released with a
[Contributor Code of Conduct](https://petrbouchal.xyz/czso/CODE_OF_CONDUCT.html).
By contributing to this project, you agree to abide by its terms.