Introducing GithubLLM integration for LangChain

[Josephrp]

Checked

• I searched existing ideas and did not find a similar one
• I added a very descriptive title
• I've clearly described the feature request and motivation for it

Feature request

GithubLLM:

I'm excited to introduce a new LLM integration for LangChain: the GithubLLM class. This integration allows users to leverage AI models available through GitHub's Models marketplace directly within the LangChain ecosystem.

The GithubLLM integration provides access to a wide range of models, including offerings from AI21, Cohere, Meta, Mistral, OpenAI, and Microsoft. This new class enables developers to experiment with these models for free, making it easier to prototype and develop AI applications using LangChain.
Key features of the GithubLLM integration:

• Support for 22 different models available in the GitHub Models marketplace
• Ability to set custom system prompts
• Support for both single-turn and multi-turn conversations
• Streaming capability for real-time responses
• Built-in model validation to ensure only supported models are used

Motivation

The motivation behind this integration is to provide LangChain users with easy access to the diverse range of models available through GitHub's Models marketplace. This allows for rapid prototyping and experimentation without the need for separate API integrations or accounts.

By leveraging GitHub's free API usage for these models, developers can quickly test different models and use cases within their LangChain applications before moving to production.

Proposal (If applicable)

from langchain_community.llms import GithubLLM

# Make sure GITHUB_TOKEN is set in your environment variables
llm = GithubLLM(model="gpt-4o")
response = llm("What is the capital of France?")
print(response)

Next steps:

A pull request has been submitted with the implementation of the GithubLLM class. We welcome feedback and suggestions from the community on how to further improve this integration.

For those interested in trying out this new integration, please note that the free API usage is currently in limited public beta. You can join the waitlist to request access.

Once the PR is merged, users will be able to experiment with these models using their GitHub personal access token. When ready for production, the same code can be used with a paid Azure account token, providing a seamless transition from development to deployment.

We're excited to see how the LangChain community will utilize this new integration in their projects!

OUTSTANDING TASKS:

• implement async
• handle empty system prompt
• map max tokens per model

[Josephrp]

#25083

[Josephrp]

Issue

GithubLLM is similar or analogous to Azure Endpoint Chat Model and might benefit from a merge or as an improvement instead

[kszicsillag]

I have managed to call GitHub's Models marketplace model with https://github.com/langchain-ai/langchain-azure (AzureAIChatCompletionsModel chat model)

[Strange-Watermelon]

How did you do it?

[kszicsillag]

Initialized AzureAIChatCompletionsModel similar to other llm classes in the python code samples in github models sandbox. Needed to pass:

• Github PAT as credential
• "https://models.github.ai/inference" as endpoint
• your target model name like "openai/gpt-4.1" as model. You can grab the name of the target model from the playground code.

sample

[Josephrp]

the good part is that it's quite free, and if you hit a rate limit you can fallback to the analogous azure endpoints :-)