Technical Board 2025_10_29

[ChrisJohnNOAA]

Attendees:
Seth Champagne
Chris John
Roy Mendelssohn
Jonathan Sherman
Shane St Savage
Giulio Verazzo

1. Review Technical Board 2025_07_30 #335

Chris: We had the first ERRDAP™ showcase, I think it was a success.

Seth: S3 updates are in, there was a change in behavior. We now make a new s3 client and then close it for each request. if this causes a performance degradation we can look into caching clients, but this is also much less likely to allow memory leaks to happen.

Some discussion on how to handle a server that needs multiple s3 connections, but determined that is best handled when there is an admin/server with that need.

2. Go over the Priority Projects #158 and discuss any changes we'd like to make.

#328 was added, there's demand for improved graphing capabilities.

Seth: NAVO has a WMS implementation using GeoTools. He'll check to see if they are willing to open source it.
Roy mentioned contour support is a frequent request. Seth said this is in the NAVO implementation.

3. Review of 2.29.0

Update to JDK25. Just doing the jdk update had noticeable performance improvements.

Big Items: MQTT (Ayush) and S3 improvements (Seth)

Also has:

• Configurable ISO.xml version
• Calendar -> java.time update/optimization
• Other minor improvements and bug fixes (various)

Chris in progress

• Working on some other optimizations
• DAP4 support (this is the big thing I'm really hoping to have done soon and in)- this is using the netcdf-java opendap implementation

4. Plan what work we are targeting for the next release (2.30.0)

JTE (Harish) good progress during GSoC but not all pages are covered. This slipped from 2.29.0, but Shane and Chris plan to co-ordinate to get it ready for 2.30.0.

5. ERDDAP™ Showcase

Tentatively planned for January, currently have 3 planned demos:
ERDDAP CMS (Giulio Verazzo)
Copy Datasets (Matthew Biddle)
Interesting search/dataset setup- (Madison Richardson)

Some discussion on if we need another. The group things there may be enough questions (particularly about CMS) to just go with these 3.

6. Datasets.xml improvements/changes

Managing datasets is one of the biggest pain points for admins. Interest in improving it.

ERDDAP CMS relies on it and other scripts (called from python). It then does some additional processing to the xml to fix it. These additional improvements could maybe get ported to the java.

Shane speculated that a better format like JSON would make programmatic interaction easier (in addition to the question/answer flow). Maybe also generating an xml file for the dataset, which with the SAXParser support for XInclude could easily then be imported into a datasets.xml file.

Seth is working on supporting XML editing from the ERDDAP™ pages directly. To this end would like to see Related to this, a more centralized xml output, with each dataset providing an intermediate data object.

Proposals:

• Improve the xml that is generated (port improvements from ERDDAP CMS)
• JSON input to GenerateDatasetsXML to provide better programmatic interaction
• (optionally) Output dataset as xml file
• Separate xml generation from creating a dataobject to represent the dataset

7. Grid Axis, how to identify special axis- destinationName, ioosCategory?

Ran out of time to discuss: #373 (comment)
Chris to open a GitHub discussion about the topic.

Action Items

• (Chris) Co-ordinate with presenters to schedule January showcase
• (@SethChampagneNRL ) Check with NAVO about open sourcing WMS implementation
• (Chris) Make sure there is proper tracking of items left for JTE
• (Chris) Make sure GenerateDatasetsXml updates are covered in priority projects/issues
• (Chris) Open a discussion about the Grid Axis identification

[ChrisJohnNOAA]

While writing up the notes I was thinking about the GenerateDatasetsXml discussion. I think we have a lot of great ideas, but we don't have unlimited man power, so we may need to prioritize.

There were 2 major directions discussed:
a) Support for managing datasets.xml through external tools
b) Integrate datasets.xml management into ERDDAP™

I like 'b' for the simplicity of admins not needing to run an additional service/tool. However because allowing users of the site to essentially change the datasets (or any config really) is new functionality, that does come with needing to be extremely careful about security.

If 'a' is a service exposed to the internet that would have the same security concerns.

That said there's items that would benefit both approaches and the current CLI usage (json inputs, improving the generated xml). So lots todo without needing to commit to an end goal.

I'd be interested in hearing thoughts from others on their ideal end goal of how to manage a datasets.xml file.