Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove couchdb env vars #1149

Merged
merged 3 commits into from
Nov 4, 2024
Merged

Remove couchdb env vars #1149

merged 3 commits into from
Nov 4, 2024

Conversation

barbarahui
Copy link
Collaborator

@barbarahui barbarahui commented Nov 4, 2024

This PR removes use of UCLDC_COUCH_URL and UCLDC_COUCH_TIMEOUT env vars from the codebase, as CouchDB is no longer running.

When running the harvest_collection dag, the validation report generation was getting skipped. We decided to reinstate it.

When running the validator, the reports will be created without validating isShownAt and isShownBy, since those validations rely on CouchDB data.

I did not actually do the work of stripping out all of the logic relating to couchdb. I figure it makes sense to do that when we overhaul the validator to compare against OpenSearch data.

There is also a 2nd commit in this PR that I think fixes a logic error: 64277b5

@@ -266,7 +265,7 @@ def validate_endpoint_task(url, mapped_versions, params=None, **context):
print(f"please validate manually: {list(errored_collections.keys())}")
print("*" * 60)

if not len(errored_collections) == len(validations):
if len(errored_collections) == len(validations):
Copy link
Collaborator Author

@barbarahui barbarahui Nov 4, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@amywieliczka It seems like this was an error in logic, but I might not be understanding what exactly this is trying to achieve? Right now, if we are validating one collection, and the validation succeeds, then this causes a ValueError to be raised, and the task fails.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, yeah, that looks like an error.

@barbarahui barbarahui marked this pull request as ready for review November 4, 2024 21:20
@barbarahui barbarahui force-pushed the remove-couchdb-env-vars branch from 1aab144 to cf11a4d Compare November 4, 2024 23:00
@@ -266,7 +265,7 @@ def validate_endpoint_task(url, mapped_versions, params=None, **context):
print(f"please validate manually: {list(errored_collections.keys())}")
print("*" * 60)

if not len(errored_collections) == len(validations):
if len(errored_collections) == len(validations):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, yeah, that looks like an error.

@barbarahui barbarahui merged commit cb14126 into main Nov 4, 2024
2 checks passed
@barbarahui barbarahui deleted the remove-couchdb-env-vars branch November 4, 2024 23:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Rikolti validate_by_mapper_type dag is in a success state, but doesn't produce any validation reports
2 participants