Herbie 2024.8.0

Thanks for all the contributions!

What's Changed

Model template changes/updates

• Add gdas_wave model by @alcoat in #342
• Add HREF by @karlwx in #349
• Additional NAM Products by @karlwx in #351
• Add HIRESW Model by @karlwx in #352
• Update GFS and GEFS template by @blaylockbk in #358
• Added NCEI as GFS source older than Jan 1, 2021 by @blaylockbk in #361
• Handle naming convention change for RRFS files by @timdonohue-aerology in #360
• Minor fix in RRFS template by @blaylockbk in #362

Other enhancements

• xarray accessor: added 80 and 100 m wind by @williamhobbs in #344
• Add type hints by @blaylockbk in #354
• Fix type hint by @blaylockbk in #356

New Contributors

• @timdonohue-aerology made their first contribution in #360

Full Changelog: 2024.7.1...2024.8.0

Herbie 2024.7.1

This release allows Numpy 2.0 to be used, since pygrib recently published support in pygrib v2.6.1.

What's Changed

• Numpy 2.0 support is ready by @blaylockbk in #340
• Also fixed some links in the docs

Full Changelog: 2024.7.0...2024.7.1

Herbie 2024.7.0

I'm excited about this release because I brought in some tools for plotting data that I've developed over the years from Carpenter_Workshop into Herbie. I hope people will test it, tell me if they are helpful for you, and tell me if I've broken anything.

Optional Dependencies

The "core" function of Herbie is to download data and read it with xarray. Thus, extra features, like plotting and extracting data at a point, now require installing "extra" dependencies that are not automatically installed. These optional dependencies include cartopy, metpy, matplotlib, and scikit-learn.

1. Installing Herbie from conda-forge will always install extra dependencies.
2. Install from pip pip install herbie-data will only install core dependencies.
3. Install from pip pip install herbie-data[extras] will install dependencies for extra features.

I anticipate this might cause some breaking changes for people. If this change didn't get it right, please open an issue (and offer some help).

Numpy <2.0

Related to dependencies, it seems pygib isn't working with Numpy 2.0 jswhit/pygrib#251, so I've tagged Herbie to require numpy<2.0. Please let me know if this changes or you have an idea to fix this. Actually, pygrib is only used by Herbie to parse the coordinate reference system from the grib files. It would be nice if Herbie didn't need to depend on pygrib (PR anyone??). Maybe pygrib should be another optional dependency.

Mature functions from Carpenter_Workshop moved into Herbie

I use my Carpenter Workshop repository as a workshop for building new stuff and testing ideas. Some of the tools I made there I use nearly every day (my EasyMap class for making Cartopy more simple), so I've migrated some of the mature features into Herbie.

1. from herbie.toolbox import EasyMap EasyMap lets you make a cartopy axes without a lot of boilerplate code. I find it useful. Read more in the docs.

   ax = EasyMay('50m', crs=..., figsize=[10,8]).STATES().LAND().OCEAN().BORDERS().ax

2. from herbie.toolbox import ... Other stuff in the toolbox, like unit conversions
3. from herbie import paint Paint is a bunch of additional colormaps I've collected over the years that I think look nice. For example, here are the National Weather Service standard color curves:
   
   I'm not completely satisfied with how these are implemented and organized, so this is considered an unstable feature. The documentation is fresh and incomplete.

---

What's Changed

• Add GFS GraphCast AI Model by @karlwx in #313
• Include uvRelativeToGrid as GRIB key included in DataArray attrs by @blaylockbk in #316
• Improving HerbieWait by @karlwx in #322
• Optional dependencies by @rafa-guedes in #319
• Migrate mature capabilities from Carpenter_Workshop into Herbie by @blaylockbk in #312
• Add xarray accessor with_wind() to compute wind speed/direction by @blaylockbk in #314
• add accessors to_180 and to_360 by @blaylockbk in #335

New Contributors

• @karlwx made their first contribution in #313
• @rafa-guedes made their first contribution in #319

Full Changelog: 2024.5.0...2024.7.0

Herbie 2024.5.0

This release of Herbie has some new features and changes I've been developing and experimenting with, so I hope to hear from you if something isn't working as expected or if you have other ideas.

💣 Deprecation / Changes

I renamed the argument searchString to search. This change only affects you if you are using the keyword argument when calling the inventory or xarray methods. Herbie will now give a warning if you use searchString, so please update your scripts.

 from herbie import Herbie
 H = Herbie('2024-01-01')

- H.inventory(searchString="TMP:2 m")
- H.xarray(searchString="TMP:2 m")

+ H.inventory(r"TMP:2 m")
+ H.xarray(r"TMP:2 m")

The reason for the name change is 1) simplicity and 2) the search string isn't an exact match search but is a regex search. (I would have named it filter, but that is a python built-in and couldn't use that). I should also mention that it is recommended to use the r or raw string representation (i.e. search=r":TMP:\d+ mb") since the string is used as a regular expression.

✨ New Feature: ds.herbie.pick_points xarray accessor

I'm very excited to share this new feature. I created a new xarray accessor to get nearest neighbor point data from a model grid. I've had this on my mind for a long time and am happy with what I have made so far. This uses the Ball Tree algorithm, and thus adds scikit-learn as a new required dependency.

Here is a very simple example getting two values from the HRRR grid:

from herbie import Herbie
import pandas as pd

# Get HRRR 2-m temperature analysis
ds = Herbie('2024-1-1', model='hrrr').xarray("TMP:2 m")


# DataFrame of points to get from model grid
points = pd.DataFrame(
    {
        "longitude": [-100.25, -99.4],
        "latitude": [44.25, 45.4],
    }
)

# Pick points from HRRR grid
ds_points = ds.herbie.pick_points(points)

See more in-depth description and examples in the docs. There are options to get values from $k$ nearest neighbors from the requested point and compute the distance-weighted average of nearby points.

If you give this new feature a try, please let me know if you see any unexpected results.

📄 Documentation

I overhauled the organization of the documentation pages. Hopefully you find them organized a little better. Feel free to contribute to the docs if you see some details missing/misleading/confusing 😄

What's Changed

• Added a __bool__ method to Herbie by @blaylockbk in #298
• Adding Ability to Download NBM QMD Data by @fleegs79 in #301
• Add model templates for GDPS and RDPS. Update HRDPS model template. by @blaylockbk and @bryanguarente in #304
• Use BallTree for getting values at nearest neighbor points by @blaylockbk in #296
• Reorganize docs: spring cleaning 🌷🧹 by @blaylockbk in #308
• Rename searchString argument to search by @blaylockbk in #305

New Contributors

• @fleegs79 made their first contribution in #301
• @bryanguarente made their first contribution in #304

Full Changelog: 2024.3.1...2024.5.0

Future Changes

I am planning to rename the argument fxx for the forecast lead time to step to match the terminology used by cfgrib.

- Herbie('2023-01-01', model='hrrr', fxx=6)
+ Herbie('2023-01-01', model='hrrr', step=6)

Again, this isn't implemented yet, but I plan to implement it eventually and wanted to let you know. Provide any comments if you have any in #160

Herbie 2024.3.1

Just two minor changes; wanted to get these out there for the few it may impact...

What's Changed

• Add Cyrilex to the 'Tip' section in the user guide searchString by @cyrilbois in #295
• change common_features into EasyMap to fix the accessors.plot() by @Davidxswang in #270

Note: the ds.herbie.plot() accessor is not implemented well. It could use a lot of work to make some standard, quick-look plots, which would be really cool, if anyone has some big ideas of what that should look like. I'm open to suggestions.

New Contributors

• @cyrilbois made their first contribution in #295
• @Davidxswang made their first contribution in #270

Full Changelog: 2024.3.0...2024.3.1

Herbie 2024.3.0

ECMWF shared this exciting news a few days ago about their open data products for the Integrated Forecast System (IFS)

ECMWF now provides a much larger open dataset to the public, representing weather forecasts at a higher resolution and a reduction in some release times.
- ECMWF: Media Centre

This Herbie release updates the ECMWF model templates to access the new IFS and AIFS 0.25 degree datasets. Historical access to the 0.4 degree datasets is still available for dates before February 1, 2024. Thanks @alcoat!

Deprecation

Now that ECMWF provides open access for two different models, the argument model='ecmwf' is deprecated. Instead, you should access the Integrated Forecast System (IFS) with model='ifs' or model='aifs for the Artificial Intelligence IFS.

- Herbie('2024-3-1', model='ecmwf')  # Deprecated

+ Herbie('2024-3-1', model='ifs')

  Herbie('2024-3-1', model='aifs')  # New Model

New Feature

Herbie has two new experimental functions that may be helpful to some people.

• HerbieLatest Find the most recent available model run.
• HerbieWait Wait for a particular model run to become available.

You can see examples in the docs: Herbie Latest Data.
If these are helpful to you but you want them to be improved, I'm looking for people to help make these better, (along with FastHerbie which could use some work too).

Herbie at the American Meteorological Society

I gave a talk on Herbie at the American Meteorology Society in January. You can see my slides here. Soon, you should be able to go to the conference website and watch the recorded presentation.

It was a lot of fun talking to some of you at AMS this year. Its always a surprise to me when I'm wandering around the poster hall and someone says, "Hey Brian, Herbie was used to make this poster!" Thanks everyone for your kindness.

Please feel free to open issues when you see problems, share how you use Herbie in the discussions, and feel free to contribute your ideas and PRs.

What's Changed

• Update GEFS model template by @blaylockbk in #268
• New helper functions to find the latest model data, HerbieLatest and HerbieWait by @blaylockbk in #274
• update RRFS template by @blaylockbk in #269
• Check that curl is in system path; warn if it is not by @blaylockbk in #276
• ECMWF IFS change by @alcoat in #283

New Contributors

• @alcoat made their first contribution in #283

Full Changelog: 2023.12.4...2024.3.0

Herbie 2023.12.4

What's Changed

• Hide curl messages unless verbose by @alexander0042 in #258
• Bug Fix: Skip downloading invalid cURL range, such as in the RAP model wind messages. by @blaylockbk in #260

New Contributors

• @alexander0042 made their first contribution in #258

Full Changelog: 2023.12.3...2023.12.4

Herbie 2023.12.3

What's Changed

• BUG FIX: Fix curl range for last grib message by @blaylockbk in #257

Full Changelog: 2023.12.2...2023.12.3

Herbie 2023.12.2

What's Changed

Enhancements

• Refine config file creation by @blaylockbk in #253
• Add option to use environment variables for some settings by @blaylockbk in #254

Documentation

• DOCS: update and fix Sphinx config by @blaylockbk in #255

Full Changelog: 2023.12.1...2023.12.2

Herbie 2023.12.1

It's been a while since the last release, and I wanted to get out all the changes that have built up.

Please open an issue if you see problems. I'm trying a new "auto release on new tag" workflow action, so I may have some micro releases coming.

What's Changed

New model templates

• NEW MODEL: Add template for URMA and more RTMA by @blaylockbk in #190
• NEW MODEL: Add Canada's HRDPS model by @blaylockbk in #208
• NEW MODEL: Add HAFS model as a Herbie template by @blaylockbk in #227
• NEW MODEL: Update gefs.py to add Azure and Google as source options by @williamhobbs in #248

Enhancements

• ENHANCEMENT: Added fallback to ecmwf to also look in short cutoff by @blaylockbk in #224
• ENHANCEMENT: Only print xarray note if self.verbose=True by @joshuaeh in #228
• ENHANCEMENT: Workflow to automatically upload releases to PyPI by @blaylockbk in #211

Bug fixes

• BUG FIX: Added correct URL for 0.25 deg GEFS datasets by @swnesbitt in #176
• BUG FIX: Add quotes around filepath in curl statement by @blaylockbk in #194
• BUG FIX: Add missing requirements for hrrr zarr datasets by @haim0n in #209
• BUG FIX: Fix GEFS member check by @davidlandry93 in #221
• BUG FIX: Fix the UnicodeDecodeError: 'utf-8' error that occurred when Windows usernames contained UTF-8 characters. by @IncubatorShokuhou in #230
• BUG FIX: Fix #232 cURL off-by-one error, add tests, etc. by @GabrielKS in #233

Documentation

• DOCS: Converted searchString docs page to Markdown by @blaylockbk in #238
• DOCS: Add version switcher by @blaylockbk in #191

New Contributors

• @swnesbitt made their first contribution in #176
• @davidlandry93 made their first contribution in #221
• @joshuaeh made their first contribution in #228
• @IncubatorShokuhou made their first contribution in #230
• @GabrielKS made their first contribution in #233
• @williamhobbs made their first contribution in #248

Full Changelog: 2023.3.0...2023.12.0