Using Nebari as a backend for xpublish-backed web apps

[rsignell-usgs]

There is a lot of ongoing activity using xpublish to deliver APIs like OPeNDAP, WMS and OGC EDR that use xarray to access data (often from Zarr datasets or kerchunked NetCDF or GRIB data on object storage).

Notes from a meet-and-greet meeting last week: xpublish-community/xpublish#138

Would it be possible to deploy these services using Nebari as a backend?

[costrouc]

@rsignell-usgs if I'm understanding what you are saying is that you'd like to deploy additional services/APIs within Nebari? Currently there are two good methods for this (which I checked the Nebari docs and they are not documented).

• You can deploy additional helm charts via https://github.com/nebari-dev/nebari/blob/develop/nebari/schema.py#L73 and https://github.com/nebari-dev/nebari/blob/develop/nebari/schema.py#LL556C5-L556C20. So you can apply any helm chart at the end via:

helm_extensions:
   - name: ...
     repository: ...
     chart: ...
     version: ...
     overrides:
         # override values.yaml
  ....
  - name: ...
     repository: ...
     chart: ...
     version: ...
     overrides:
         # override values.yaml

I will admit this does require some devops knowledge but this is extremely flexible and the recommended way to extend Nebari at the moment. I would like to enhance this feature since we know several people using this.

The other approach works well if you have one off docker images/REST APIs you would like to deploy with a url and optional enforced authentication see https://github.com/nebari-dev/nebari/blob/develop/nebari/template/stages/08-nebari-tf-extensions/tf-extensions.tf. This is even less documented and not what i would recommend.

I'm seeing both of these features used more and more and I think that we need to extend this more and document it.

I've created an issue in the documentation nebari-dev/nebari-docs#246

[abkfenris]

Is there a way to define additional Kubernetes manifests directly, without wrapping them up as Helm charts or in Terraform?

Once Xpublish is further along, there may be some distributions that are ready to distribute Helm charts, but we're still a ways away from that.

[costrouc]

Is there a way to define additional Kubernetes manifests directly, without wrapping them up as Helm charts or in Terraform?

If you are deploying to Kubernetes I strongly recommend helm charts just because it is the way the Kubernetes community primarily packages things. That said tf-extensions mentioned above might do what you need. All it needs is a docker image to use. We have talked about supporting kustomize as well which I feel is simpler to use but has its own difficulties.

You can have the helm charts be within the nebari deployment repo that you have since repository: ... supports filesystem paths. I do agree that helm charts are intimidating with all the templating .

[rsignell-usgs]

@costrouc , awesome. @jonmjoyce, hoping you guys can test this out on your Nebari deployment to serve your xpublish-based APIs asascience-open/nextgen-dmac#23
cc: @abkfenris, @mpiannucci, @cheryldmorse