Commits on Sep 19, 2023

1. [core] Moved namespaces and computes to database …

   The existing implementation was connecting to external Altus and DWX
   api's to fetch the available computes. This commit changes that by
   moving the cluster config to database. Two new tables have been added
   (a) `beeswax_namespace` to hold config for a namespace/dialect
   (b) `beeswax_compute` to hold configs for individual compute clusters
   linked to the namespaces.
   This change currently support hive and impala clusters.
   
   There is also a service-discovery component that keeps the list of
   namespaces and computes updated in the corresponding tables.
   `sync_warehouses.py` performs the service discovery in the CDW environ
   by talking to kubernetes api's. It creates one Hive and one Impala
   namespace and one compute for each virtual warehouse. The command is
   supposed to run every minute and keep the list of warehouses updated.
   
   There are related changes and fixes to the rest of the code to support
   the query execution on different computes.
   
   Change-Id: Ifd8dbc8d716dfe2000fbfa8121e39f2610051fa1

   

   amitsrivastava committed Sep 19, 2023
   Configuration menu

   • View commit details
   • Copy full SHA for 3594a73
   • Browse repository at this point

   Copy the full SHA

   3594a73 View commit details

   Browse the repository at this point in the history