Support new OEIL procedure file URLs

These have changed from `https://oeil.secure.europarl.europa.eu/oeil/popups/ficheprocedure.do?lang=en&procedure=...` to `https://oeil.secure.europarl.europa.eu/oeil/en/procedure-file?reference=...`. Old press releases still use the old-style links, so we have to support both.
HowTheyVote · Feb 3, 2025 · f071539 · f071539
1 parent 04a08e2
commit f071539
Show file tree

Hide file tree

Showing 3 changed files with 2,340 additions and 1 deletion.
diff --git a/backend/howtheyvote/scrapers/press_releases.py b/backend/howtheyvote/scrapers/press_releases.py
@@ -236,7 +236,10 @@ def _procedure_references(self, doc: BeautifulSoup) -> list[str] | None:
 
         for link in self._links(doc):
             if not link.startswith(
-                "https://oeil.secure.europarl.europa.eu/oeil/popups/ficheprocedure.do"
+                (
+                    "https://oeil.secure.europarl.europa.eu/oeil/popups/ficheprocedure.do",
+                    "https://oeil.secure.europarl.europa.eu/oeil/en/procedure-file",
+                )
             ):
                 continue