OpenAI LLM Client?

[gladyshcodes]

Would it be worth adding the OpenAI LLM client?

Vision:
Creating an abstract class that allows developers to inject their preferred client
This is apparently possible with current OpenAI functionality

[slavingia]

This is a bit too vague for me to understand what you mean. A user story / example would be helpful!

[gladyshcodes]

Perhaps overcomplicated language. What I basically meant is that we currently only support the Anthropic client, but I’m suggesting adding support for the OpenAI client as well

[slavingia]

My bad! Makes sense now. I agree once they have launched their version of computer use / operator. Apparently may be released in Jan.

[gladyshcodes]

This is indeed possible with current OpenAI capabilities. We can annotate a screenshot by processing the DOM, sending it to the LLM for analysis, and receiving the ID of the annotated element on the page. However, this approach would require a lot of new functionality and a bit of shift in our strategy for processing pages

Maybe worth waiting Computer Use feat from OpenAI

[slavingia]

Have you tried that? It doesn't really work in my experience.

[jiz17043]

Curious, currently only Anthropic API key is supported, will you consider supporting OpenAI APIkey?

[slavingia]

Once they launch a feature similar to computer use!