Quarkus AI - starter samples and examples

[donbourne]

Hello,

I've been looking into how best to build new Java-based AI apps, and in particular investigating the Quarkus way vs. the Spring AI way. While both provide capabilities to interact with LLMs for generating responses, streaming, structuring input/output, using tools, etc. they do so in different ways. I wanted to share some of my observations here, and see which ones people think are worth addressing and to get ideas to address them. My hope is that we can improve the user experience of the Quarkus AI solution.

For this discussion, I'll focus on starters. Typically developers go to a "starter" website when they want to quickly get a template from which to copy patterns and dependencies from. The site lets them select the kind of app they want to build, then lets them download a zip containing the source code and build artifacts needed to try it out. Starter apps often have sample code giving a brief canonical example of how to work with the technology. For Quarkus, that site is https://code.quarkus.io.

AI Infused service tile

I went to https://code.quarkus.io to search for generative AI options. I knew Quarkus / LangChain4j works with a lot of LLMs. At that site, the first page shows a set of presets to choose from -- including one for AI infused service. I chose that, since my goal was to build an app using an LLM. The site then shows a list of 2 extensions (LangChaing4j OpenAI and LangChain4j Easy RAG). My first impression was that that was all there was for AI starters -- not very rich/extensive.

Langchain4j extensions

I eventually tried searching on LangChain4j and found a more significant set of extensions to choose from. I noticed that only the Easy RAG extension indicates it has starter code.

If you select one of the langchain4j * options, like "langchain4j Hugging Face" and generate a project, the popup will state "Follow the lanchgain4j * guide for your next steps". The link points to the top level documentation for langchain4j. I then have to navigate to find relevant docs (for example, /LLMs/Hugging Face) and since this is a doc and not a guide there are no complete code samples - just snippets.

Search box

I thought I saw "ai" in the list of categories from the Search box, but I don't currently see that.

Examples

Quarkus

• Samples are in https://github.com/quarkiverse/quarkus-langchain4j/tree/main/samples, which is not easy to find. All guides start with the expectation that OpenAPI is used, but some do provide an alternative of Ollama. We also found posts on Medium and Baeldung by searching for 'Quarkus langchain4j example'.

Suggested changes

I'd like to suggest a few changes for https://code.quarkus.io:

1. have the "AI infused service" tile return all of the AI related services, not just OpenAI and Easy RAG
2. add an "ai" category to the list of options from the Search. Have it contain all of the AI-related extensions (perhaps the same list as for the "AI infused service")
3. given there are code changes needed to switch between LLMs, I suggest having starter code for all of the LLM-related extensions. This could illustrate how to do something basic like completions or streaming.
4. include a link in the popup for each of the LangChain4j extensions that points directly to the LangChain4j doc for the corresponding LLM.

...and for the examples:

1. create a comprehensive set of concise examples, showing off each of the significant Quarkus/LangChain4j capabilities. Have links to corresponding runnable code in git.

Does this seem like a good set of changes?