User flow for blue-green % traffic split (initial attempt)

[sriumcp]

0. Decide if you want to allow ingress traffic to go to model mesh. If yes, do the following.

Note: chart names / other conventions to be revisited

cat << EOF >> values.yaml
targetEnv: kserve-modelmesh
templateName: externalize
# optional parameter; by default, this is called mm-external
serviceName: mm-external
# optional parameter; by default, this is called mm-external
gatewayName: mm-external-gateway
EOF
helm template <hub repo> traffic-templates -f values.yaml | kubectl apply -f -

Note:

• We assume Istio for traffic engineering above
• The above creates a service of type "ExternalName" and a gateway.

2. Create inference service

cat <<EOF | kubectl apply -f -
# Model component 1 - inferenceservice
apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: wisdom-primary
  namespace: modelmesh-serving
  labels:
    app.kubernetes.io/name: wisdom
    app.kubernetes.io/version: v1
    iter8.tools/watch: "true"
  annotations:
    serving.kserve.io/deploymentMode: ModelMesh
    serving.kserve.io/secretKey: localMinIO
spec:
  predictor:
    model:
      modelFormat:
        name: sklearn
      storageUri: s3://modelmesh-example-models/sklearn/mnist-svm.joblib
EOF

3. Initialize routing

Note: chart names / other conventions to be revisited

cat << EOF >> values.yaml
targetEnv: kserve-modelmesh
templateName: blue-green
modelName: wisdom
# optional isvc names for versions
# versions:
# - name: wisdom-primary
#   weight: 100
# - name: wisdom-candidate
#   weight: 100
EOF
helm template <hub repo> traffic-templates -f values.yaml | kubectl apply -f -

Assumptions:

1. We assume Istio for traffic engineering above
2. No need to touch the primary weight ever. Candidate weights are relative to primary. So, no need to create a separate weight cm for primary.