diff --git a/README.md b/README.md index 4aadba15..f9715ddb 100644 --- a/README.md +++ b/README.md @@ -39,10 +39,10 @@ The package is documented in the [online manual](https://jgcri.github.io/gcamfao ### Quick Start in R (> 4.0) & Rstudio -#### 1. Download and install: +#### 1. Download and install (size < 1 GB): -* Using `devtools` to download and install (The size < 1 GB): -* `devtools::install_github("jgcri/gcamfaostat")` +* On the command line: navigate to your desired folder location and then enter +* `git clone https://github.com/JGCRI/gcamfaostat.git` #### 2. Load and run the gcamdata package diff --git a/man/FF_download_FAOSTAT.Rd b/man/FF_download_FAOSTAT.Rd index 7a3372fb..48ba1898 100644 --- a/man/FF_download_FAOSTAT.Rd +++ b/man/FF_download_FAOSTAT.Rd @@ -11,7 +11,7 @@ FF_download_FAOSTAT( ) } \arguments{ -\item{DATASETCODE}{Dataset code in FAO metadata, e.g., QCL is the production dataset.} +\item{DATASETCODE}{Single dataset code in FAO metadata, e.g., QCL is the production dataset.} \item{DATA_FOLDER}{Destination folder for storing raw data} diff --git a/man/FF_download_RemoteArchive.Rd b/man/FF_download_RemoteArchive.Rd index 60a2c30b..077434f2 100644 --- a/man/FF_download_RemoteArchive.Rd +++ b/man/FF_download_RemoteArchive.Rd @@ -12,7 +12,7 @@ FF_download_RemoteArchive( ) } \arguments{ -\item{DATASETCODE}{Dataset code in FAO metadata, e.g., QCL is the production dataset.} +\item{DATASETCODE}{Single dataset code in FAO metadata, e.g., QCL is the production dataset.} \item{RemoteArchiveURL}{Zenodo URL, default (GCAM v7) = "https://zenodo.org/record/8260225/files/"} diff --git a/vignettes/vignette_getting_started.Rmd b/vignettes/vignette_getting_started.Rmd index 4ee3269b..28a84f4c 100644 --- a/vignettes/vignette_getting_started.Rmd +++ b/vignettes/vignette_getting_started.Rmd @@ -60,21 +60,24 @@ Nonetheless, the FAOSTAT data employed within gcamdata has traditionally involve * R version 4.0 or higher and RStudio are recommended. -#### Clone this repository +#### Clone this repository (size < 1 GB) -* On the command line: navigate to your desired folder location and then enter `git clone https://github.com/JGCRI/gcamfaostat.git` +* On the command line: navigate to your desired folder location and then enter +```{r eval = FALSE} +git clone https://github.com/JGCRI/gcamfaostat.git +``` + +* Or using `devtools` to download and install in r environment (into `.libPaths()`) +```{r eval = FALSE} +install.packages("devtools") # install devtools package if not already +devtools::install_github("jgcri/gcamfaostat") +``` -* If using GitHub Desktop: Go to https://github.com/jgcri/gcamfaostat and click the green "Code" button. Then click "Open with GitHub Desktop". +* Note that `gcamfaostat` is not dependent on GCAM, so it can be installed to folders as desired by the user. #### Loading the `gcamfaostat` package * Open the `gcamfaostat` folder you just cloned and double-click the `gcamfaostat.Rproj` file. RStudio should open the project. - -* If you have not installed `devtools` before, first enter the following into the R console -```{r eval = FALSE} -install.packages("devtools") -``` - * Then to load the `gcamfaostat` package: ```{r eval = FALSE} @@ -82,14 +85,10 @@ devtools::load_all() ``` #### Package dependencies -* You may also need to install package dependencies, for example using: -```R -devtools::install_deps() -``` -* Note that could be issues with some recent package dependencies. Users can consider using `renv`. +Users can consider using `renv`. - * `renv` is an R package for dependency management. - * Using it will save a private R library with the correct versions of any package dependencies. + * `renv` has been initialized and a `renv.lock` file was included. + * Using `renv` will save a private R library with the correct versions of any package dependencies. * Please find more details [here](https://jgcri.github.io/gcamdata/articles/getting-started/getting-started.html#optional-renv-currently-included-in-gcam-6-0-and-gcamdata-release) in `gcamdata` manual. diff --git a/vignettes/vignette_preparing_data.Rmd b/vignettes/vignette_preparing_data.Rmd index 5b338368..7e1d0dc0 100644 --- a/vignettes/vignette_preparing_data.Rmd +++ b/vignettes/vignette_preparing_data.Rmd @@ -73,15 +73,38 @@ Note that if Exist_Prebuilt is `TRUE` for a dataset, it suggests the raw data wa Two functions above are created for downloading the raw data from a remote archive or the FAOSTAT API (latest data). The dataset code variable in the two functions, if including all, can be generated using `gcamfaostat_metadata(OnlyReturnDatasetCodeRequired = T)`. +Example: +``` {r, eval =FALSE} +# Dataset PP, producer prices, is downloaded from RemoteArchiveURL to DATA_FOLDER +# RemoteArchiveURL = "https://zenodo.org/record/8260225/files/" +# DATA_FOLDER = "inst/extdata/aglu/FAO/FAOSTAT" +FF_download_RemoteArchive( + DATASETCODE = "PP", + OverWrite = TRUE # overwrite existing PP dataset + ) + +# Dataset OA, population, is downloaded from FAOSTAT to DATA_FOLDER +FF_download_FAOSTAT(DATASETCODE = "OA", OverWrite = TRUE) + +# Note that single DATASETCODE is allowed in both functions. +``` ### 4. Check local raw data info [`FF_rawdata_info()`](https://jgcri.github.io/gcamfaostat/reference/FF_rawdata_info.html) -* Providing more detailed metadata information, similar to `gcamfaostat_metadata`(which calls `FF_rawdata_info`). +* Providing more detailed metadata information, similar to `gcamfaostat_metadata` (which calls `FF_rawdata_info`). * Indicate whether an update from FAOSTAT is potentially needed. * Can be used to download data if not exist. +Example: +``` {r, eval =FALSE} +# Provide detailed metadata of "PP" and "OA" in DATA_FOLDER ("inst/extdata/aglu/FAO/FAOSTAT") +FF_rawdata_info(DATASETCODE = c("PP", "OA"), DOWNLOAD_NONEXIST = FALSE) + +# If "PP" or "OA" does not exist, download from remote archive +FF_rawdata_info(DATASETCODE = c("PP", "OA"), DOWNLOAD_NONEXIST = TRUE, FAOSTAT_or_Archive = "Archive") +``` ### 5. Load raw data into package @@ -90,6 +113,12 @@ Two functions above are created for downloading the raw data from a remote archi * Loading FAOSTAT raw data, e.g., used in `xfaostat_L191_RawDataPreProc*` modules. * Note that `.` in column name is substituted with `_`. +Example: +``` {r, eval =FALSE} +# Read raw data of "PP" and "OA" from DATA_FOLDER ("inst/extdata/aglu/FAO/FAOSTAT") to .GlobalEnv +FAOSTAT_load_raw_data(DATASETCODE = c("PP", "OA"), .Envir = .GlobalEnv) +``` + ### 6. Update the Prebulit data If users updated the FAOSTAT raw data, and run `driver_drake` is run with `Process_Raw_FAO_Data <- TRUE` in `constants.R`, the updated preprecessed data are stored in drake cache. And they can be used to update `Prebulit data` by sourcing `data-raw/generate_package_data.R`. diff --git a/vignettes/vignette_use_cases.Rmd b/vignettes/vignette_use_cases.Rmd index b52781ea..bd51df21 100644 --- a/vignettes/vignette_use_cases.Rmd +++ b/vignettes/vignette_use_cases.Rmd @@ -26,9 +26,10 @@ knitr::opts_chunk$set( To update the output data by including new data years, e.g., for model base year updates, the user needs to implement the following steps: 1. Download the latest FAOSTAT data using the `FF_download_FAOSTAT` function. -2. In the configuration file (`constants.R`), adjust the year variables and set `Process_Raw_FAO_Data` to `TRUE`. -3. Verify and update the output formats and names in the data exporting modules. -4. Execute the `driver_drake` function to initiate the data processing. +2. In the configuration file (`R/constants.R`), adjust the year variables and set `Process_Raw_FAO_Data` to `TRUE`. +3. If more recent data is available the maximum year in `FAOSTAT_Hist_Year_*` can be updated accordingly. +4. Verify and update the output formats and names in the data exporting modules. +5. Execute the `driver_drake` function to initiate the data processing. ## Adding new data processing modules @@ -51,7 +52,9 @@ If all the necessary FAOSTAT raw data is already incorporated into gcamfaostat, ## Country aggregation and disaggregation -Since the 1970s, the number of countries in the world has increased due to the dissolution of regions. In other words, when a region dissolves, the country associated with it ceases to exist, and new countries emerge in its place, see [**Figure 1**](FigCountryChange). We included functions to deal with changes in the country classifications. +Since the 1970s, the number of countries in the world has increased due to the dissolution of regions. In other words, when a region dissolves, the country associated with it ceases to exist, and new countries emerge in its place, see [**Figure 1**](FigCountryChange). We included functions to deal with changes in the country classifications. Note the functions rely on the `area_code` ID included in FAO dataset. They are also consistently defined in the function. + + [`FAOSTAT_AREA_RM_NONEXIST`](https://jgcri.github.io/gcamfaostat/reference/FAOSTAT_AREA_RM_NONEXIST.html)