v0.18.0

🚀 Added

💪 New VLMs in Workflows

We've shipped blocks to integrate with Google Gemini and Anthropic Claude, but that's not everything! OpenAI block got updated. New "VLM Interface" of the block assumes that it can be prompted using pre-configured options and model output can be processed by set of formatter blocs to achieve desired end. It is now possible to:

• use classification prompting in VLM block and apply VLM as Classifier block to turn output string into classification result and process further using other blocks from ecosystem
• the same can be achieved for object-detection prompting and VLM as Detector block, which converts text produced by model into sv.Detections(...)

From now one, VLMs are much easier to integrate.

🧑‍🦱 USE CASE: PII protection when prompting VLM

Detect faces first, apply blur prediction visualisation and ask VLMs to tell what is the person eye colour - they won't be able to tell 🙃

👨‍🎨 USE CASE: VLM as object detection model

👓 USE CASE: VLM as secondary classifier

Turn VLM output into classification results and process using downstream blocks - here we ask Gemini to classify crops of dogs to tell what is dog's breed - then we extract top class as property.

🤯 Workflows previews in documentation 📖

Thanks to @joaomarcoscrs we can embed Workflows into documentation pages. Just take a look how amazing it is ❗

🌱 Changed

• E2E tests for workflows on hosted platform by @PawelPeczek-Roboflow in #622
• Allow model type "yolov8" without size by @SolomonLake in #627
• Fix GDAL issue in GHA by @PawelPeczek-Roboflow in #628
• Add support for DELETE in sqlite wrapper by @grzegorz-roboflow in #631
• Keep distinct exec sessions for inf pipeline usage by @grzegorz-roboflow in #632
• Insert usage payloads into redis and to sorted set atomically by @grzegorz-roboflow in #633
• remove height from workflow example by @capjamesg in #634

❗ BREAKING ❗ Batch[X] kinds removed from Workflows

What was changed and why?

In inference release 0.18.0 we decided to make drastic move to heal the ecosystem from the problem with ambiguous kinds names (Batch[X] vs X - see more here).

The change is breaking only for non-Roboflow Workflow plugins depending on imports from inference.core.workflows.execution_engine.entities.types module. To the best of our knowledge, there is no such plugin.

The change is not breaking in terms of running Workflows on Roboflow platform and on-prem given that external plugins were not used.

Migration guide

Migration should be relatively easy - in the code of a Workflow block, all instances of

from inference.core.workflows.execution_engine.entities.types import BATCH_OF_{{KIND_NAME}}

should be replaced with

from inference.core.workflows.execution_engine.entities.types import {{KIND_NAME}}

PR with changes as reference: #618

Full Changelog: v0.17.1...v0.18.0