@@ -26,10 +26,9 @@ The following are DAGs grouped by their primary tag:
26
26
27
27
### Data Normalization
28
28
29
- | DAG ID | Schedule Interval |
30
- | ----------------------------------------------------------- | ----------------- |
31
- | [ ` add_license_url ` ] ( #add_license_url ) | ` None ` |
32
- | [ ` update_science_museum_urls ` ] ( #update_science_museum_urls ) | ` None ` |
29
+ | DAG ID | Schedule Interval |
30
+ | ------------------------------------- | ----------------- |
31
+ | [ ` add_license_url ` ] ( #add_license_url ) | ` None ` |
33
32
34
33
### Data Refresh
35
34
@@ -171,7 +170,6 @@ The following is documentation associated with each DAG (where available):
171
170
1 . [ ` smk_workflow ` ] ( #smk_workflow )
172
171
1 . [ ` staging_database_restore ` ] ( #staging_database_restore )
173
172
1 . [ ` stocksnap_workflow ` ] ( #stocksnap_workflow )
174
- 1 . [ ` update_science_museum_urls ` ] ( #update_science_museum_urls )
175
173
1 . [ ` wikimedia_commons_workflow ` ] ( #wikimedia_commons_workflow )
176
174
1 . [ ` wikimedia_reingestion_workflow ` ] ( #wikimedia_commons_workflow )
177
175
1 . [ ` wordpress_workflow ` ] ( #wordpress_workflow )
@@ -1057,26 +1055,6 @@ authorization required. API is undocumented.
1057
1055
1058
1056
----
1059
1057
1060
- ### ` update_science_museum_urls `
1061
-
1062
- #### Update Science Museum URLs
1063
-
1064
- One-time maintenance DAG to update Science Museum records to have valid URLs.
1065
- See https://github.com/WordPress/openverse/issues/4261 .
1066
-
1067
- For each Science Museum record, this DAG:
1068
-
1069
- - updates the url to the new format, excluding ` /images/ ` in the path if it
1070
- exists
1071
- - validates whether the url is reachable. If not, the record ID is added to an
1072
- ` invalid_science_musem_ids ` table.
1073
-
1074
- Once complete, we can use the ` science_museum_invalid_ids ` to identify records
1075
- to delete. They are not automatically deleted by this DAG, in order to give us
1076
- an opportunity to first see how many there are.
1077
-
1078
- ----
1079
-
1080
1058
### ` wikimedia_commons_workflow `
1081
1059
1082
1060
** Content Provider:** Wikimedia Commons
0 commit comments