Hindcasts
Author: Jason Fleming (GitHub ID: jasonfleming)
Retrospective analyses (i.e., "hindcasts") of historical tropical cyclone events are an important part of model development as well as risk assessment. The ASGS is an ideal system for producing hindcasts because it already contains features for working with tropical cyclone data including BEST track files as well as the Forecast/Advisories that are issued by the National Hurricane Center.
Creating hindcasts of historical hurricanes with the parametric Generalized Asymmetric Holland Model (GAHM) is a three step process:
- Select a storm to hindcast and create the associated input data
- Configure the ASGS properly to run the hindcast
- Execute the ASGS and possibly manipulate the input data feed to make the hindcast progress
These steps are detailed in the sections below using Hurricane Ike of 2008 as a case study.
September 13, 2008 - Hurricane Ike makes landfall at Galveston as a large Category 2 hurricane. Its large size contributes to a storm surge that is as high as 20ft (6.1m), which inundates many of the barrier islands off the Texas coast. Many structures on the Bolivar Peninsula are destroyed. Most notably, in Gilchrist all but one house is destroyed by the storm surge. The storm also causes heavy rains where it makes landfall, peaking at 18.9in (480mm). At one point, the storm knocks out power to as many as an estimated 2.6 million people.[ Ike is one of the most destructive hurricanes ever to hit Texas and one of the deadliest, causing $19.3 billion in damages and killing 84 people.
Ike advisory 01 was issued almost two weeks before landfall when the storm system was still in the central Atlantic (see below). Since this time period will not be relevant to the target area (the Texas Gulf coast), the Operator will need to make a decision on when the retrospective period should start. This decision will also need to be informed by initialization concerns, including the development of a fully developed wave field, as well as any potential coastal water level set up.
I normally set the start date by examining the graphics issued with each forecast/advisory, and take the associated physics into account, along with looking at the location and forward speed of the storm. For Ike, I selected the point of emergence of the storm off the coast of Cuba into the Gulf of Mexico in advisory 36 as a reasonable starting point, as shown below.
Choosing the end time of the retrospective period should take several factors into account, including the "drain down" time period after the storm has passed, as well as the total length of the retrospective. Some storm events don't have much drain down time in the BEST track (i.e., the storm dissipates shortly after landfall) and some have so much that it would make the hindcast too long to be practical. Looking at Ike in particular, the final advisory was issued at 4amCDT on Sunday 14 September 2008, which gives us about one day of drain down time and a total hindcast length of 5 days (see below).
Qualitatively, our hindcast plan will be to
-
run a tides-only initialization to the point that we want to start including meteorological forcing from Ike
-
set up a BEST track file and forecast/advisory file that represents the time that we want our hindcast to end
Conceptually, the timeline looks like the following for our Ike hindcast:
BEST track BEST track
START END
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Advisory 36 |
|**********************| |
START END | Advisory 53
| |********************|
| START END
| |
|---------------------v|^=======================v|
COLDSTARTDATE HINDCAST HINDCAST
START END
Where the ...---v indicates that a hotstart file was written at the end of a model run,
and the ^===... indicates that a hotstart file was read at the start of a model run.
In order to accomplish this, we will configure ASGS to start its initialization phase on the COLDSTARTDATE; it will perform a tides-only
run for a number of days equal to the HINDCASTLENGTH parameter, write a hotstart file, and then start looking for a forecast/advisory. We
will feed it Ike advisory 53 and the full BEST track file. It will see that it needs to perform a catch-up
nowcast from the end of the tidal initialization until the start time for advisory 53. We will also set SCENARIOPACKAGESIZE=0 so that it does not actually try to run a forecast for advisory 53.
In order to start the retrospective modelling at a time corresponding to advisory 36, we need to take a look at the BEST track file for the storm, and the data that corresponds to the start date we've selected. Forecast/advisory 36 was issued at 11pmEDT on Tuesday 9 September 2008, which corresponds to 03Z on Wednesday 10 September 2008. Therefore, the last available BEST track data at the time that this advisory was issued was at 00Z on 10 September 2008. We now need to find the full BEST track file for this storm and look to this time as our starting point.
The official archive of BEST track files contains links to BEST track files by year. Ike occurred in 2008, in the Atlantic basin (prefix al) and was the 9th tropical cyclone of the season, so the file to look for is bal092008.dat.gz.
Inside the file, we find this time on line 100 (i.e., the first line below that starts with AL, 09, 2008091000,...):
AL, 09, 2008090918, , BEST, 0, 227N, 833W, 65, 966, HU, 34, NEQ, 155, 150, 105, 170, 1008, 275, 15, 80, 0, L, 0, , 0, 0, IKE, D, 12, NEQ, 180, 180, 90, 180
AL, 09, 2008090918, , BEST, 0, 227N, 833W, 65, 966, HU, 50, NEQ, 100, 75, 30, 90, 1008, 275, 15, 80, 0, L, 0, , 0, 0, IKE, D, 12, NEQ, 180, 180, 90, 180
AL, 09, 2008090918, , BEST, 0, 227N, 833W, 65, 966, HU, 64, NEQ, 20, 0, 0, 30, 1008, 275, 15, 80, 0, L, 0, , 0, 0, IKE, D, 12, NEQ, 180, 180, 90, 180
AL, 09, 2008091000, , BEST, 0, 231N, 840W, 65, 968, HU, 34, NEQ, 160, 150, 110, 180, 1006, 200, 15, 80, 0, L, 0, , 0, 0, IKE, D, 12, NEQ, 180, 210, 150, 240
AL, 09, 2008091000, , BEST, 0, 231N, 840W, 65, 968, HU, 50, NEQ, 100, 75, 30, 90, 1006, 200, 15, 80, 0, L, 0, , 0, 0, IKE, D, 12, NEQ, 180, 210, 150, 240
AL, 09, 2008091000, , BEST, 0, 231N, 840W, 65, 968, HU, 64, NEQ, 30, 0, 0, 20, 1006, 200, 15, 80, 0, L, 0, , 0, 0, IKE, D, 12, NEQ, 180, 210, 150, 240
AL, 09, 2008091006, , BEST, 0, 234N, 846W, 70, 964, HU, 34, NEQ, 170, 150, 115, 180, 1006, 200, 15, 85, 0, L, 0, , 0, 0, IKE, D, 12, NEQ, 180, 210, 150, 240
Based on the above, and considering that we want our initialization period, a.k.a. "tidal spinup", to last 30 days (HINDCASTLENGTH=30.0) and to end at 00Z on 10 September 2008, we see that we should set COLDSTARTDATE=2008081100 (00Z on 11 August 2008). When in doubt doing date math, you can always used the little dateCalculator.pl command line utility:
asgs (default)> dateCalculator.pl --input-date 20080910000000 --ddays -30.0
The resulting date is 2008 08 11 00 00 00.
The first line in the BEST track file, indicating the time the system spent as an "INVEST" is as follows:
AL, 09, 2008090106, , BEST, 0, 172N, 370W, 30, 1006, TD, 0, , 0, 0, 0, 0, 1011, 250, 90, 0, 0, L, 0, , 0, 0, INVEST, D,
The last line in the Ike BEST track file is as follows:
AL, 09, 2008091512, , BEST, 0, 472N, 711W, 35, 986, EX, 34, NEQ, 0, 180, 150, 0,
So the data we can use with the GAHM model ends at 12Z September 15 2008.
Now that we have the BEST track file, we will also need file to represent Ike advisory 53. This file must exist and be valid to prevent parse errors from the ASGS, but the actual contents will not be used, since it represents a forecast and we are not planning to run a forecast for advisory 53. This is an arbitrary limitation in the ASGS that I think we need to relax in the future, but for now it gives us an excuse to illustrate how mock RSS entries are created for forecast/advisories.
The text advisories from the National Hurricane Center forecast advisory archive. Just click on the year to see all the storms for that year. In real time, these advisories are posted to the NHC's RSS feed embedded in XML files. The XML files for many historical storms and advisories are already available in the StormSurgeLive storm archive repository.
The HTML files for the forecast/advisories from target storm can be used
to create a mock RSS feed that mimics those issued via RSS by the NHC during the live event in
a file callsed index-at.xml (-at means Atlantic basin). An
example index-at.xml file from advisory 52 of Ike can be found in the
storm archive repository.
When looking at Ike advisory 53, I noticed that the mock RSS entry for this final advisory was
not present in the storm-archive repository. I cloned the repository, changed to the ike subdirectory,
and entered the following commands (in ASGS Shell, which populates the SCRIPTDIR environmental variables and
provides the Perl environment, among other things):
asgs (default)> curl -O https://data.coastalrisk.live/track_archive/2008/09_IKE/al092008.fstadv.053.shtml
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 20762 100 20762 0 0 56986 0 --:--:-- --:--:-- --:--:-- 57038
asgs (default)> perl $SCRIPTDIR/input/sample_advisories/advisory_xml_filler.pl --input al092008.fstadv.053.shtml --template $SCRIPTDIR/input/sample_advisories/template.index.xml --best bal092008.dat
[2021-Dec-15-T11:37:28] INFO: advisory_xml_filler.pl: BASIN: AL
[2021-Dec-15-T11:37:28] INFO: advisory_xml_filler.pl: STORM NUMBER: 09
[2021-Dec-15-T11:37:28] INFO: advisory_xml_filler.pl: STORM YEAR: 2008
[2021-Dec-15-T11:37:28] INFO: advisory_xml_filler.pl: Issued at: 0900 UTC SUN SEP 14 2008
After completing this procedure, I committed the mock RSS entry and associated BEST track file for advisory 53 back to the storm-archive repository.
Another thing I noted about the timing of the final advisory and the final BEST track file was that the final update to the BEST track file was actually after the final forecast/advisory was issued. The ASGS normally uses BEST track (or any retrospective and/or concurrent data) greedily, that is, if the the BEST track and forecast/advisory time periods overlap, it uses the BEST track data for the overlapping time period, since that is considered definitive rather than speculative. We will see what effect this overlapping has on the way the ASGS finishes the hindcast.
We can now update our conceptual timeline for the retrospective model run with actual dates and times.
BEST track BEST track
START END
2008090112 2008091512
|~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~|
Advisory 36 |
|**********************| |
START END |
2008091000 2008091500
| | Advisory 53
| |********************|
| START END
| 2008091509 2008091512
| |
|---------------------v|^=======================v|
2008081100 2008091000 2008091512
COLDSTARTDATE HINDCAST HINDCAST
START END
In normal operation, the ASGS contacts remote web and ftp sites to collect the real time data that it requires. During a retrospective analysis, however, the Operator will be supplying these files. If the ASGS Operator has administrative access to a web server and ftp server, these can be set up to mimic the web and ftp sites of the NHC, and the ASGS can be configured to go to the mock sites instead of the real ones.
However, it is simpler to reconfigure the ASGS to look to its local filesystem for the mock RSS and BEST track files, thus eliminating the requirement for having web and ftp servers handy. A sample set of test configuration parameters for external data sources is as follows (using Ike as an example):
GRIDNAME=TX2008
source $SCRIPTDIR/config/mesh_defaults.sh
STORM=09 # storm number, e.g. 05=ernesto in 2006
YEAR=2008 # year of the storm
TRIGGER=rssembedded
RSSSITE=filesystem
FTPSITE=$RSSSITE
FDIR=$WORK/storm-archive/ike
HDIR=$FDIR
PSEUDOSTORM=yes
SCENARIOPACKAGESIZE=0 # <-<< will not run any forecasts
where the FDIR parameter points to the location where the storm-archive repository has been cloned to the local filesystem.
The resulting ASGS configuration file can be found in the StormSurgeLive asgs-configs repository.
Once the mock advisories have been developed and the ASGS has been configured, it is up to the Operator to play the role of the NHC and "issue" new hindcast/nowcast data and new forecast advisories. Step-by-step instructions for doing this are as follows:
-
Go to the directory where the mock RSS feed for the forecast/advisories are stored.
-
Make a symbolic link to the first advisory in the test (because the ASGS is hardwired to look for the
index-at.xmlfile) -
Make a symbolic link to the BEST track file that contains all the data in the hindcast. In this case, for Ike, we will use the final BEST track file rather than the one that was current at the time that the last advisory was issued. This is because the last entries to the BEST track file for Ike were later than the final advisory. The ASGS is hardwired to look for a BEST track file that is named
bal<STORM><YEAR>.dat.
These steps are illustrated as follows:
asgs (default)> cd $WORK/storm-archive/ike
asgs (default)> ln -s 00.bal092008.dat bal092008.dat
asgs (default)> ln -s 53.092008.index-at.xml index-at.xml
After performing these steps, the Operator can start the ASGS as usual. It will perform its initialization phase with no meteorological forcing, then look to the local filesystem for a forecast/advisory. It will pick up the advisory and BEST track file that are symbolically linked there, and construct a nowcast run with the BEST track file that we can use for our retrospective analysis. Once that is complete, it will see that there are no forecasts to run, and will go looking for a new advisory. The Operator can then shut this ASGS instance down, and begin working on constructing a corresponding dataset for the winds at 10m (i.e., without the surface roughness or canopy coefficient nodal attributes).