Web API for Hybrid AI

[kevinmoch]

Session description

Web apps are increasingly expected to gain access to a language model. We are proposing Web APIs that allow web developers to directly access both on-device and cloud-based language models, and securely share user data between multiple apps when using these models.

The following are the APIs goals:

• Provide web developers with a connection strategy for accessing both on-device and cloud-based models. For example, if no on-device models are available, attempt to access cloud-based models. Conversely, if cloud-based models are unavailable, try accessing on-device models.

• Provide web developers with a storage strategy for sharing user's private data. For example, one web app saves users' private data into a local vector database. Another web app, when accessing a on-device language model, can leverage this data through a local RAG system.

The following are not within our scope of concern:

• Design a uniform JavaScript API for accessing browser-provided language models, known as the Prompt API, which is currently being explored by Chrome's built-in AI team.

• Issues faced by hybrid AI, such as model management, elasticity through hybrid AI, and user experience, as this topic has already been discussed in Hybrid AI Presentations in the WebML IG, and will be covered in the sessions on AI Model Management.

This session will showcase how to use our proposed API for booking flights and hotels. It will also provide specific implementation details and references for these APIs. Example source code and implementation references can be found on GitHub web-hybrid-ai.

Session goal

Explore the potential of our proposed Web API for accessing hybrid AI through use case demonstrations and API implementations. Additionally, discuss concrete steps for moving forward.

Additional session chairs (Optional)

@espinr

Who can attend

Anyone may attend (Default)

IRC channel (Optional)

#web-hybrid-ai

Other sessions where we should avoid scheduling conflicts (Optional)

#15

Instructions for meeting planners (Optional)

No response

Agenda for the meeting.

1. Introduce the goals of the Web APIs we propose for hybrid AI (2m)
2. Introduce Connection API (5m)
3. Introduce Storage API (4m)
4. A Showcase of Hybrid AI App (3m)
5. Considerations for Connection Strategy (2m)
6. Considerations for Storage Strategy (2m)
7. Considerations for Native OS APIs (2m)
8. Discuss possible resolutions, followup actions and collaborations (20m)

Links

• Web API for Hybrid AI - Slides
• Web API for Hybrid AI - Explainer

Links to calendar

• 2024-09-25, 10:00 - 11:00

Meeting materials

• Slides
• Minutes

[tpac-breakout-bot]

Thank you for proposing a session!

You may update the session description as needed and at any time before the meeting, but please keep in mind that tooling relies on issue formatting: follow the instructions and leave all headings and other formatting intact in particular. Bots and W3C meeting organizers may also update the description, to fix formatting issues or add links and other relevant information. Please do not revert these changes. Feel free to use comments to raise questions.

Do not expect formal approval; W3C meeting organizers endeavor to schedule all proposed sessions that are in scope for a breakout. Actual scheduling should take place shortly before the meeting.

[mmccool]

Thanks for proposing this topic. I was going to propose something similar but you beat me to it :). I will do my best to attend, modulo conflicts. Reach out to me in advance if you'd like to coordinate with the related session on AI Model Management #15

[espinr]

Minutes of the session: https://www.w3.org/2024/09/25-web-hybrid-ai-minutes.html
Main slide deck: https://kevinmoch.github.io/web-hybrid-ai/Templates/Overview.html