Skip to content

Latest commit

 

History

History
220 lines (170 loc) · 7.26 KB

developer-guide.md

File metadata and controls

220 lines (170 loc) · 7.26 KB
Raw

Developer Guide

Prerequisites

You need Java and Maven to build ModelMesh from source and etcd to run the unit tests. To build your custom modelmesh container image and deploy it to a ModelMesh Serving installation on a Kubernetes cluster, you need the docker and kubectl CLIs. On macOS you can install the required CLIs with Homebrew:

  • Java: brew install java
  • Maven: brew install maven
  • Etcd: brew install etcd
  • Docker: brew install docker
  • Kubectl: brew install kubectl

Generating sources

The gRPC stubs like the ModelMeshGrpc class have to be generated by the gRPC proto compiler from the .proto source files under src/main/proto. The generated sources should be created in the target directory target/generated-sources/protobuf/grpc-java.

To generate the sources run either of the following commands:

mvn package -DskipTests
mvn install -DskipTests

Project setup using an IDE

If you are using an IDE like IntelliJ IDEA or Eclipse to help with your code development you should set up source and target folders so that the IDE's compiler can find all the source code including the generated sources (after running mvn install -DskipTests).

For IntelliJ this can be done by going to File > Project Structure ... > Modules:

  • Source Folders
    • src/main/java
    • src/main/proto
    • target/generated-sources/protobuf/grpc-java (generated)
    • target/generated-sources/protobuf/java (generated)
  • Test Source Folders
    • src/test/java
    • target/generated-test-sources/protobuf/grpc-java (generated)
    • target/generated-test-sources/protobuf/java (generated)
  • Resource Folders
    • src/main/resources
  • Test Resource Folders
    • src/test/resources
  • Excluded Folders
    • target

You may also want to increase your Java Heap size to at least 1.5 GB.

Testing code changes

Note, before running the test cases, make sure you have etcd installed (see #prerequisites):

$ etcd --version

etcd Version: 3.5.5
Git SHA: 19002cfc6
Go Version: go1.19.1
Go OS/Arch: darwin/amd64

You can either run all test suites at once. You can use the -q flag to reduce noise:

mvn test -q

Or you can run individual test cases:

mvn test -Dtest=ModelMeshErrorPropagationTest
mvn test -Dtest=SidecarModelMeshTest,ModelMeshFailureExpiryTest

It can be handy to use grep to reduce output noise:

mvn test -Dtest=SidecarModelMeshTest,ModelMeshFailureExpiryTest | \
  grep -E " Running |\[ERROR\]|Failures|SUCCESS|Skipp|Total time|Finished"

[INFO] Running com.ibm.watson.modelmesh.ModelMeshFailureExpiryTest
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 10.257 s - in com.ibm.watson.modelmesh.ModelMeshFailureExpiryTest
[INFO] Running com.ibm.watson.modelmesh.SidecarModelMeshTest
[INFO] Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 17.302 s - in com.ibm.watson.modelmesh.SidecarModelMeshTest
[INFO] Tests run: 4, Failures: 0, Errors: 0, Skipped: 0
[INFO] BUILD SUCCESS
[INFO] Total time:  39.916 s
[INFO] Finished at: 2022-11-01T14:33:33-07:00

Building the container image

After testing your code changes locally, it's time to build a new modelmesh container image. Replace the value of the DOCKER_USER environment variable to your DockerHub user ID and change the IMAGE_TAG to something meaningful.

export DOCKER_USER="<your-docker-userid>"
export IMAGE_NAME="${DOCKER_USER}/modelmesh"
export IMAGE_TAG="dev"
export GIT_COMMIT=$(git rev-parse HEAD)
export BUILD_ID=$(date '+%Y%m%d')-$(git rev-parse HEAD | cut -c -5)

docker build -t ${IMAGE_NAME}:${IMAGE_TAG} \
    --build-arg imageVersion=${IMAGE_TAG} \
    --build-arg buildId=${BUILD_ID} \
    --build-arg commitSha=${GIT_COMMIT} .

docker push ${IMAGE_NAME}:${IMAGE_TAG}

Updating the ModelMesh Serving deployment

In order to test the code changes in an existing ModelMesh Serving deployment, the newly built container image needs to be added to the model-serving-config ConfigMap.

First, check if your ModelMesh Serving deployment already has an existing model-serving-config ConfigMap:

kubectl get configmap

NAME                            DATA   AGE
kube-root-ca.crt                1      4d2h
model-serving-config            1      4m14s
model-serving-config-defaults   1      4d2h
tc-config                       2      4d2h

If the ConfigMap list contains model-serving-config, save the contents of your existing configuration in a local temp file:

mkdir -p temp
kubectl get configmap model-serving-config -o yaml > temp/model-serving-config.yaml

And add the modelMeshImage property to the config.yaml string property:

      modelMeshImage:
        name: <your-docker-userid>/modelmesh
        tag: dev

Replace the <your-docker-userid> placeholder with your Docker username/login.

The complete ConfigMap YAML file might look like this:

apiVersion: v1
kind: ConfigMap
metadata:
  name: model-serving-config
  namespace: modelmesh-serving
data:
  config.yaml: |
    podsPerRuntime: 1
    restProxy:
      enabled: true
    scaleToZero:
      enabled: false
      gracePeriodSeconds: 5
    modelMeshImage:
      name: <your-docker-userid>/modelmesh
      tag: dev

Apply the ConfigMap to your cluster:

kubectl apply -f temp/model-serving-config.yaml

If you are comfortable using vi, you can forgo creating a temp file and edit the ConfigMap directly in the terminal:

kubectl edit configmap model-serving-config

If you did not already have a model-serving-config ConfigMap on your cluster, you can create one like this:

# export DOCKER_USER="<your-docker-userid>"
# export IMAGE_NAME="${DOCKER_USER}/modelmesh"
# export IMAGE_TAG="dev"

kubectl apply -f - <<EOF
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: model-serving-config
data:
  config.yaml: |
    modelMeshImage:
      name: ${IMAGE_NAME}
      tag: ${IMAGE_TAG}
EOF

The modelmesh-controller watches the ConfigMap and responds to updates by automatically restarting the serving runtime pods using the newly built modelmesh container image.

You can check which container images are used by running the following command:

kubectl get pods -o jsonpath='{range .items[*]}{"\n"}{.metadata.name}{"\t"}{range .spec.containers[*]}{.image}{", "}{end}{end}' | sort | column -ts $'\t' | sed 's/, *$//g'

etcd-78ff7867d5-45svw                            quay.io/coreos/etcd:v3.5.4
minio-6ddbfc9665-gtf7x                           kserve/modelmesh-minio-examples:latest
modelmesh-controller-64f5c8d6d6-k6rzc            kserve/modelmesh-controller:latest
modelmesh-serving-mlserver-1.x-84884c6849-s8dw6  kserve/rest-proxy:latest, seldonio/mlserver:1.3.2, kserve/modelmesh-runtime-adapter:latest, kserve/modelmesh:dev
modelmesh-serving-mlserver-1.x-84884c6849-xpdw4  kserve/rest-proxy:latest, seldonio/mlserver:1.3.2, kserve/modelmesh-runtime-adapter:latest, kserve/modelmesh:dev