Saving snapshot and clip locally to use as input for LLM Vision, possible?

[woody4165]

Hi all

I'm trying to use this fantastic notification system with LLM Vision integration in Home Assistant.

What I would like to achieve is to have frigate-notify pass a jpg or mp4 to HA (or eventually NodeRed, that I have separately installed) and use them as input in LLM Vision analysis.
My thought is to save them in a shared folder with the HA system, then have HA or NR use them as input.

Do you think is possible?

Thanks

[0x2142]

Hi,
Could this already be accomplished via NodeRed? The snapshot images and recordings are available via the Frigate API, so NodeRed could query those APIs directly for the content. If you needed a trigger, frigate-notify's webhook alert could forward the event info to NodeRed to kick off the automtion on that side.

[freefd]

@0x2142 exactly, you replied faster than I prepared the diagram.
The frigate-notify should not pass through the binary data such as MP4. Instead, @woody4165, you can use the webhook functionality to form the payload data as a metadata for the side worker:

[woody4165]

@0x2142 @freefd thanks for your replies

I'm using webhook to trigger to start the automation.
What I was trying to do is to re-use the clip and snapshot generated by frigate-notify to avoid another query to frigate via API and to be sure to use the exact frames sent to telegram for the llm vision analysis.

In any case, I'm using your suggestions and the automation works fine

Thanks