Skip to content

Latest commit

 

History

History
125 lines (113 loc) · 4.55 KB

README.md

File metadata and controls

125 lines (113 loc) · 4.55 KB
Raw

Context

Description

This project contains some examples for HDSR to get WIWB (neerslag, etc.) and vdSat (bodemvocht till 2024-04-30) data. Moreover, it can resample the data (NetCDF) to .csv, to point(s), but also to polygon (aggregation in space).

  • WIWB API is a public API (behind a login) so we request data from the cloud.
  • vdSat data is already stored on HDSR's network drive. This data is not updated anymore. We use wiwb to get the data and optionally resample it to timeseries per point or polygon.

Usage

  1. build conda environment from file if you don't have environment already:
> conda env create --name wiwb_vdsat_examples --file <path_to_project>/environment.yml
  1. Edit main.py to enable/disable wiwb and/or vdsat (end of main.py just comment out stuff)
  2. Optionally edit examples/wiwb.py if you want to get wiwb data
  3. Optionally edit examples/vdsat.py if you want to get vdsat data
  4. Run code:
> conda activate wiwb_vdsat_examples
> python <path_to_project>/main.py

License

MIT

Releases

None

Contributions

All contributions, bug reports, bug fixes, documentation improvements, enhancements and ideas are welcome on https://github.com/hdsr-mid/wiwb_vdsat_examples/issues

Test coverage

None

Conda general tips

Build conda environment (on Windows) from any directory using environment.yml:

Note1: prefix is not set in the environment.yml as then conda does not handle it very well Note2: env_directory can be anywhere, it does not have to be in your code project

> conda env create --prefix <env_directory><env_name> --file <path_to_project>/environment.yml
# example: conda env create --prefix C:/Users/xxx/.conda/envs/project_xx --file C:/Users/code_projects/xx/environment.yml
> conda info --envs  # verify that <env_name> (project_xx) is in this list

Start the application from any directory:

> conda activate <env_name>
At any location:
> (<env_name>) python <path_to_project>/main.py

Test the application:

> conda activate <env_name>
> cd <path_to_project>
> pytest  # make sure pytest is installed (conda install pytest)

List all conda environments on your machine:

At any location:
> conda info --envs

Delete a conda environment:

Get directory where environment is located 
> conda info --envs
Remove the enviroment
> conda env remove --name <env_name>
Finally, remove the left-over directory by hand

Write dependencies to environment.yml:

The goal is to keep the .yml as short as possible (not include sub-dependencies), yet make the environment reproducible. Why? If you do 'conda install matplotlib' you also install sub-dependencies like pyqt, qt icu, and sip. You should not include these sub-dependencies in your .yml as:

  • including sub-dependencies result in an unnecessary strict environment (difficult to solve when conflicting)
  • sub-dependencies will be installed when dependencies are being installed
> conda activate <conda_env_name>

Recommended:
> conda env export --from-history --no-builds | findstr -v "prefix" > --file <path_to_project>/environment_new.yml   

Alternative:
> conda env export --no-builds | findstr -v "prefix" > --file <path_to_project>/environment_new.yml 

--from-history: 
    Only include packages that you have explicitly asked for, as opposed to including every package in the 
    environment. This flag works regardless how you created the environment (through CMD or Anaconda Navigator).
--no-builds:
    By default, the YAML includes platform-specific build constraints. If you transfer across platforms (e.g. 
    win32 to 64) omit the build info with '--no-builds'.

Pip and Conda:

If a package is not available on all conda channels, but available as pip package, one can install pip as a dependency. Note that mixing packages from conda and pip is always a potential problem: conda calls pip, but pip does not know how to satisfy missing dependencies with packages from Anaconda repositories.

> conda activate <env_name>
> conda install pip
> pip install <pip_package>

The environment.yml might look like:

channels:
  - defaults
dependencies:
  - <a conda package>=<version>
  - pip
  - pip:
    - <a pip package>==<version>

You can also write a requirements.txt file:

> pip list --format=freeze > <path_to_project>/requirements.txt