Add script to initialize the rikolti-stg index alias

record_indexer/README.md

@@ -32,24 +32,38 @@ export OPENSEARCH_PASS=Rikolti_05
 export OPENSEARCH_IGNORE_TLS=True
 ```
 
-**To use this OpenSearch docker container with mwaa-local-runner, set the previous values and the below endpoint in dags/startup.sh:**
+**To connect to this OpenSearch docker container from mwaa-local-runner, set the previous values and the below endpoint in dags/startup.sh:**
 
 ```
 export OPENSEARCH_ENDPOINT=https://host.docker.internal:9200/
 ```
 
-## Initializing an OpenSearch instance to work with Rikolti
+**To use this OpenSearch docker container from your host machine, set the previous values and the below endpoint in env.local:**
+
+```
+export OPENSEARCH_ENDPOINT=https://localhost:9200/
+```
 
-Create an index template for rikolti:
+## Initializing an OpenSearch instance to work with Rikolti
 
 Make sure that OPENSEARCH_ENDPOINT and the relevant authentication is set in your environment.
 
+1. Create an index template for rikolti:
+
 ```
 python -m record_indexer.index_templates.rikolti_template
 ```
 
 This creates a record template that will be used for adding documents to any index with name matching `rikolti*` is added to the cluster.
 
+2. Create an index and add it to the alias `rikolti-stg`
+
+```
+python -m record_indexer.initialize.add_rikolti-stg_index
+```
+
+This creates an empty index named `rikolti-<current timestamp>` (enforcing the use of the rikolti_template for all records indexed into it) and assigns it to the alias `rikolti-stg`. 
+
 ## Running the Record Indexer
 
 TODO: We don't currently support running the indexer from the command line
@@ -60,6 +74,8 @@ See the Rikolti README page section on [Airflow Development](https://github.com/
 
 ## Index lifecycle
 
+TODO: this section is all pretty defunct now; update documentation
+
 The lifecycle of an index is as follows:
 
 #### Create new index

record_indexer/initialize/add_rikolti-stg_index.py

@@ -0,0 +1,83 @@
+import argparse
+import json
+import sys
+from datetime import datetime
+
+import requests
+
+from .. import settings
+from ..utils import print_opensearch_error
+
+"""
+    Create a rikolti index and add the rikolti-stg alias to it
+    https://opensearch.org/docs/2.3/opensearch/index-templates/
+"""
+
+
+def main(name=None):
+    # check if index with rikolti-stg alias exists
+    url = f"{settings.ENDPOINT}/_alias/rikolti-stg"
+    r = requests.get(
+        url,
+        auth=settings.get_auth(),
+        verify=settings.verify_certs()
+    )
+    if r.status_code == 200:
+        print("Index with alias rikolti-stg already exists; OpenSearch "
+              "instance ready for use")
+        return
+
+    # create index
+    if not name:
+        name = datetime.now().strftime("%Y%m%d%H%M%S")
+
+    index_name = f"rikolti-{name}"
+    url = f"{settings.ENDPOINT}/{index_name}"
+    r = requests.put(
+        url,
+        headers={"Content-Type": "application/json"},
+        auth=settings.get_auth(),
+        verify=settings.verify_certs()
+    )
+    if not (200 <= r.status_code <= 299):
+        print_opensearch_error(r, url)
+        r.raise_for_status()
+    print(r.text)
+
+    # add alias
+    url = f"{settings.ENDPOINT}/_aliases"
+    data = {
+        "actions": [
+            {"add": {"index": index_name, "alias": "rikolti-stg"}}
+        ]
+    }
+    r = requests.post(
+        url,
+        headers={"Content-Type": "application/json"},
+        data=json.dumps(data),
+        auth=settings.get_auth(),
+        verify=settings.verify_certs()
+    )
+    if not (200 <= r.status_code <= 299):
+        print_opensearch_error(r, url)
+        r.raise_for_status()
+    print(r.text)
+
+
+if __name__ == "__main__":
+    parser = argparse.ArgumentParser(
+        description=(
+            "Creates an empty index at rikolti-<name>; if no name "
+            "provided, uses the current timestamp. Adds the index to "
+            "the rikolti-stg alias."
+        )
+    )
+    parser.add_argument(
+        "-n", "--name", 
+        help=(
+            "Optional name for the index; created index will be "
+            "rikolti-<name>, if none provided, current timestamp will be used"
+        )
+    )
+    args = parser.parse_args()
+    sys.exit(main(args.name))