-
Notifications
You must be signed in to change notification settings - Fork 14
Finding Data
At present each district has there own way of naming a location. Some spell out the site name others use short acryonyms. If the names are generally spelled it is easier to find things but it's not impossible to track down others.
Additionally, we don't have a location/timeseries browsing GUI at this time, this services is primarily intended for various websites to be able to pull data directly and make few assumptions about how one would want to use it.
The catalog end point https://cwms-data.usace.army.mil/cwms-data/catalog facilitates searching for data available. Currently there are two options "LOCATIONS" and "TIMESERIES". Locations will include additional public and long names that will help tracking down the name to use for a timeseries.
To just get all of either would be:
https://cwms-data.usace.army.mil/cwms-data/catalog/locations
or
https://cwms-data.usace.army.mil/cwms-data/catalog/timeseries
NOTE: at this time you can only search by a limit set of fields; the long and public names unfortunately are not among them, however in writing this document that was a clear short coming the will address as soon as we can.
"Joe Pool Lake" in the Fort Worth District (SWF) is a good example. The Location name used to access the data is JPLT2. But unless you know that you likely wouldn't think of it while querying data.
To search use the location catalog end point. Use the following to retrieve all locations starting with J for the SWF district. (NOTE: office 3 letter codes can be retrieved from https://cwms-data.usace.army.mil/cwms-data/offices)
https://cwms-data.usace.army.mil/cwms-data/catalog/LOCATIONS?office=SWF&like=J.*
While it is possible the abbreviation for a given location may not actually start with the letter of the common name it makes sense to start that way. Requests will generally be faster the more you can narrow things down.
This will return data that looks like the following:
<catalog>
<page-size>500</page-size>
<total>125</total>
<entries>
<entry xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="locationCatalogEntry" office="SWF">
<name>MSDT2-Project</name>
<nearestCity>Presque Isle</nearestCity>
<kind>SITE</kind>
<timeZone>US/Central</timeZone>
<latitude>0.0</latitude>
<longitude>0.0</longitude>
<publishedLatitude>0.0</publishedLatitude>
<publishedLongitude>0.0</publishedLongitude>
<elevation>0.0</elevation>
<unit>m</unit>
<verticalDatum>NGVD29</verticalDatum>
<nation>UNITED STATES</nation>
<state>TX</state>
<county>Travis</county>
<boundingOffice>SWF</boundingOffice>
<mapLabel>MSDT2</mapLabel>
<active>true</active>
<aliases/>
</entry>
<entry xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:type="locationCatalogEntry" office="SWF">
<name>JSPT2-Project</name>
<nearestCity>Presque Isle</nearestCity>
<kind>SITE</kind>
<timeZone>US/Central</timeZone>
<latitude>0.0</latitude>
<longitude>0.0</longitude>
<publishedLatitude>0.0</publishedLatitude>
<publishedLongitude>0.0</publishedLongitude>
<elevation>0.0</elevation>
<unit>m</unit>
<verticalDatum>NGVD29</verticalDatum>
<nation>UNITED STATES</nation>
<state>TX</state>
<county>Jasper</county>
<boundingOffice>SWF</boundingOffice>
<mapLabel>JSPT2</mapLabel>
<active>true</active>
<aliases/>
</entry>
</entries>
You will have to scan through the list to find it. Here is the relevant section for Joe Pool Lake
<entry>
<name>JPLT2</name>
<nearestCity>Florence Hill, TX</nearestCity>
<publicName>Joe Pool Lake</publicName>
<longName>Joe Pool Lake</longName>
<description>8049800 - Joe Pool Lk nr Duncanville, TX</description>
<kind>PROJECT</kind>
</entry>
As you can see the name we want to use for Joe Pool Lake is "JPLT2".
Now let's search for the required data. For this exercise we will look for outflow.
Let's get all time series that start with JPLT2 that include "Flow". I've intentionally left out the office parameter to show it works; however, including it if known is recommended for performance.
https://cwms-data.usace.army.mil/cwms-data/catalog/timeseries?like=JPLT2.*Flow.*
In this case we get a lot of timeseries with "Flow", but we can see this district is using "Flow-Out" for the outflow timeseries (some districts use Flow-Res Out) so let's narrow it down more.
https://cwms-data.usace.army.mil/cwms-data/catalog/timeseries?like=JPLT2.*Flow-Out.*
There are still several options. However most of them are because of variations in interval or whether data is averaged.
For this excercise we will assume 1 Hour instantaneous outflow is desired.
We will use JPLT2.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI for our timeseries retrieval.
will provide the following:
<?xml version="1.0" encoding="windows-1252"?><time-series><query-info><time-of-query>2022-12-05T22:41:56Z</time-of-query><process-query>PT1.000S</process-query><format-output>PT0.001S</format-output><requested-start-time>2022-12-05T16:41:56Z</requested-start-time><requested-end-time>2022-12-05T22:41:56Z</requested-end-time><requested-format>XML</requested-format><requested-office>SWF</requested-office><requested-item><name>JPLT2.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI</name><unit>EN</unit><datum>NATIVE</datum></requested-item><total-time-series-retrieved>1</total-time-series-retrieved><unique-time-series-retrieved>1</unique-time-series-retrieved><total-values-retrieved>5</total-values-retrieved><unique-values-retrieved>5</unique-values-retrieved></query-info><quality-codes><!-- The following quality codes are used in this dataset--><code value="0" meaning="Unscreened"/></quality-codes><time-series><office>SWF</office><name>JPLT2.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI</name><alternate-names><name>1Hour-Flow-Out</name><name>JOE POOL LAKE.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI</name><name>JPLT2.Flow-Out.Inst.1Hour.0.GagingData</name><name>JPLT2.Flow-Out.Inst.1Hour.0.Reporting</name><name>TX08007.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI</name></alternate-names><regular-interval-values interval="PT1H" unit="cfs" segment-count="1">
<segment position="1" first-time="2022-12-05T17:00:00Z" last-time="2022-12-05T21:00:00Z" value-count="5"><!-- value quality-code -->
16 0
16 0
16 0
16 0
16 0
</segment>
</regular-interval-values></time-series></time-series>
If you are using curl, the swagger-ui, or a library of some sort the preferred way would be to use the accept header instead of the format query parameter:
curl -X 'GET' \ 'https://cwms-data.usace.army.mil/cwms-data/timeseries?name=JPLT2.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI&office=SWF&unit=cfs&begin=PT-6H' \ -H 'accept: application/json;version=2'
will provide the following:
{
"begin": "2022-12-05T17:20:53+0000[UTC]",
"end": "2022-12-05T23:20:53+0000[UTC]",
"interval": "PT0S",
"interval-offset": 0,
"name": "JPLT2.Flow-Out.Inst.1Hour.0.Rev-SWF-REGI",
"office-id": "SWF",
"page": "MTY3MDI2MzIwMDAwMHx8Nnx8NTAw",
"page-size": 500,
"time-zone": "US/Central",
"total": 6,
"units": "cfs",
"value-columns": [
{
"name": "date-time",
"ordinal": 1,
"datatype": "java.sql.Timestamp"
},
{
"name": "value",
"ordinal": 2,
"datatype": "java.lang.Double"
},
{
"name": "quality-code",
"ordinal": 3,
"datatype": "int"
}
],
"values": [
[ 1670263200000, 16, 0 ],
[ 1670266800000, 16, 0 ],
[ 1670270400000, 16, 0 ],
[ 1670274000000, 16, 0 ],
[ 1670277600000, 16, 0 ],
[ 1670281200000,null,0 ]
]
}
The times are suitable for loading into a javascript Date or most other languages Date/Time facilities. The values sets are:
[time,value,quality]
NOTE: the qualities are numeric values, and are not currently documented while there are more for now:
The basics are 0 - not screened (Note with data like outflow this will be the case because it's a calculated value anything else - screened in some way.
It recommended to call the district directly. You can open issues and discussions here but it may take some time to track down who to get answer you. If you do ask here about specific data please include at least the rough geographic area so assist in that search.
There are 3 options:
- The local measuring is having issues and not transmitting data
- The district has chosen not to provide that data to the system (the timeseries is still listed though)
- The data flow has been interrupted.
However, in either case, please contact the owning districts Water Management office to get more detail.