Skip to content

Commit

Permalink
editing spark deployment in "kube-roll-all" to roll Spark if the ETL …
Browse files Browse the repository at this point in the history
…job is not running (#2543)
  • Loading branch information
EliseCastle23 authored Feb 13, 2025
1 parent 6bd624d commit db2d2d2
Showing 1 changed file with 6 additions and 2 deletions.
8 changes: 6 additions & 2 deletions gen3/bin/kube-roll-all.sh
Original file line number Diff line number Diff line change
Expand Up @@ -129,9 +129,13 @@ fi

if g3k_manifest_lookup .versions.spark 2> /dev/null; then
#
# Only if not already deployed - otherwise it may interrupt a running ETL
# Only if an ETL job is not running
#
if ! g3kubectl get deployment spark-deployment > /dev/null 2>&1; then
# Save etl pods to a var
PODS=$(kubectl get pods -o jsonpath='{range .items[*]}{.metadata.name}{" "}{.status.phase}{"\n"}{end}' | grep "etl")
# Get any running ETL pods and save them to a var
RUNNING_POD=$(echo "$PODS" | grep " Running")
if [ -z "$RUNNING_POD" ]; then
gen3 kube-setup-spark &
fi
else
Expand Down

0 comments on commit db2d2d2

Please sign in to comment.