Skip to content

Harmony 1929 - Ensure OPeNDAP downloads use POST #97

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Nov 22, 2024
Merged

Conversation

vinnyinverso
Copy link
Contributor

@vinnyinverso vinnyinverso commented Nov 21, 2024

Jira Issue ID

HARMONY-1929

Description

Since opendap can have potentially very long data download URLs we need to ensure that we convert the query string to POST body parameters in those cases and submit a POST request.

NOTE: I did see OPeNDAP timing out for very long URLs.

The snyk check is failing due to an internal snyk error.

Local Test Steps

This is how I tested:

  1. With a local harmony instance running, submit a request.
  2. Once it finishes, change one of the job link data URLs in the job links table to:
    https://opendap.uat.earthdata.nasa.gov/collections/C1261703111-EEDTEST/granules/SC:ATL03.006:264549068.dap.nc4?dap4.ce=/ds_surf_type[0:1:4]
  3. Copy the job ID for this job as you will use it to make a download request with harmony-py.
  4. Create a virtual environment with this version of harmony-py installed (pip install -e ./path/to/harmony_py).
  5. In a notebook (see basic.ipynb), create a client with env=Environment.LOCAL and make a download request (e.g. harmony_client.download_all('88aebf3b-bb47-475d-8d54-05fa9812ae77', overwrite=True)).
  6. Verify that the opendap data is successfully retrieved.

PR Acceptance Checklist

  • Acceptance criteria met
  • Tests added/updated (if needed) and passing
  • Documentation updated (if needed)

@vinnyinverso vinnyinverso marked this pull request as ready for review November 22, 2024 13:47
Copy link
Contributor

@indiejames indiejames left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally

@vinnyinverso vinnyinverso merged commit 26f014e into main Nov 22, 2024
11 of 13 checks passed
@vinnyinverso vinnyinverso deleted the HARMONY-1929 branch November 22, 2024 19:08
@vinnyinverso vinnyinverso restored the HARMONY-1929 branch November 26, 2024 20:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants