The correct way of enforcing cpu device type under BentoML 1.0?

[AndreasLR]

Sometimes one might have a system locally for development that that differs from the production system. In my case my dev setup has a GPU, while the production system does not. It would be ideal to have some documentation on how device selection can be enforced so as to use the CPU on both systems.

For the TF2 tutorial I have tried by configuring the runner resources in bentofile.yaml without success:

service: "service:svc"
description: "file: ./README.md"
labels:
  owner: bentoml-team
  stage: demo
include:
- "*.py"
exclude:
- "locustfile.py"
python:
  lock_packages: false
  packages:
    - tensorflow
    - Pillow
runners:
- name: tensorflow_mnist
  runnable_type: TensorflowRunnable
  resource_config:
    cpu: 8.0
    max_batch_size: 8

In fact, here is bento.yaml produced by the bentoml build command:

service: service:svc
name: tensorflow_mnist_demo
version: vr2b5ixbr6fxxygv
bentoml_version: 1.0.0rc0.post8+g37b5502
creation_time: '2022-06-01T09:46:13.179835+00:00'
labels:
  owner: bentoml-team
  stage: demo
models:
- tag: tensorflow_mnist:alrlgsxa36za7ygv
  module: bentoml.tensorflow
  creation_time: '2022-05-31T12:41:37.238563+00:00'
runners:
- name: tensorflow_mnist
  runnable_type: TensorflowRunnable
  models:
  - tensorflow_mnist:alrlgsxa36za7ygv
  resource_config:
    cpu: 12.0
    nvidia_gpu: 1.0
apis:
- name: predict_ndarray
  input_type: NumpyNdarray
  output_type: NumpyNdarray
- name: predict_image
  input_type: Image
  output_type: NumpyNdarray

On a last note it might be efficient to have some feedback from bentoml build if there are parameters in bentofile.yaml that fail to take effect.

[timliubentoml]

hi @AndreasLR! We just released the rc1 version, we think this was actually a bug with rc0. Could you update with "pip install bentoml --pre -U" and try it out?