Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Intra-vendor model routing feature enhancements #12948

Closed
wants to merge 35 commits into from
Closed

Intra-vendor model routing feature enhancements #12948

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
1898895
Fix endpoint get showing api key in plain text issue
ashera96 Feb 16, 2025
e1a90ec
Fix default vendors to have requestModel and remainingTokenCount fiel…
ashera96 Feb 18, 2025
0739029
Merge pull request #12947 from ashera96/intra-vendor-model-routing-po…
ashera96 Feb 18, 2025
53ae427
Fix EndpointDTO
PasanT9 Feb 18, 2025
e59b722
Fix parse errors due to custom parameter
PasanT9 Feb 18, 2025
1224219
Merge pull request #12950 from PasanT9/450-post-alpha-v1
PasanT9 Feb 18, 2025
860ebcb
Commit Design Assistant related generated files
ashera96 Feb 19, 2025
8c6548d
Fix apiKeyValue missing issue in synapse artifact
ashera96 Feb 20, 2025
ea03e28
Merge pull request #12964 from ashera96/intra-vendor-model-routing-po…
ashera96 Feb 20, 2025
3e6ba00
Handle default endpoint PUT DELETE calls
ashera96 Feb 20, 2025
77bd0f0
Handle deploymentStage invalid scenario
ashera96 Feb 20, 2025
2a34a96
Merge pull request #12967 from ashera96/intra-vendor-model-routing-po…
ashera96 Feb 20, 2025
53468d5
Improve error handling and set primary endpoint IDs on AI API creatio…
ashera96 Feb 20, 2025
8b8c158
Merge pull request #12972 from ashera96/intra-vendor-model-routing-po…
ashera96 Feb 20, 2025
d6394cd
Add failover policy for AI APIs
PasanT9 Feb 20, 2025
7bb0e0d
Refactor AI Gateway implementation
PasanT9 Feb 20, 2025
cdbb830
Remove AI API Endpoint sequences during API undeploy
PasanT9 Feb 21, 2025
fd95bdc
Refacter AI API failover implementation
PasanT9 Feb 21, 2025
fe27197
Add endpoint URL validation
ashera96 Feb 21, 2025
2cc9596
Merge pull request #12974 from ashera96/intra-vendor-model-routing-po…
ashera96 Feb 21, 2025
acc22ab
Merge pull request #12975 from PasanT9/post-alpha-v5
PasanT9 Feb 21, 2025
fc209a6
Add unique constraint on API_UUID, ENDPOINT_NAME, REVISION_UUID, and …
ashera96 Feb 21, 2025
b9dc2a9
Merge pull request #12977 from ashera96/intra-vendor-model-routing-po…
ashera96 Feb 21, 2025
4a403a0
Add missing java doc comments
PasanT9 Feb 21, 2025
e434de1
Merge pull request #12979 from PasanT9/post-alpha-v8
PasanT9 Feb 21, 2025
a836c28
Handle fail and exception scenarios
PasanT9 Feb 21, 2025
68be6e7
Merge pull request #12981 from PasanT9/post-alpha-v8
PasanT9 Feb 21, 2025
5f18fcb
Add API endpoint deletion on API deletion
ashera96 Feb 21, 2025
d5073d3
Fix review comments
PasanT9 Feb 21, 2025
d27bccb
Merge pull request #12983 from PasanT9/post-alpha-v8
PasanT9 Feb 21, 2025
85f58ca
Onboard models for default AI/LLM vendors
ashera96 Feb 21, 2025
066a2a3
Fix endpoint deletion failure when either production or sandbox don't…
ashera96 Feb 21, 2025
20d1f85
Merge pull request #12984 from ashera96/intra-vendor-model-routing-po…
ashera96 Feb 21, 2025
71e0777
Fix policy invoking when only single deployment stage is defined
PasanT9 Feb 21, 2025
8b0ed83
Merge pull request #12985 from PasanT9/post-alpha-v6
PasanT9 Feb 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .../apimgt/org.wso2.carbon.apimgt.api/src/main/java/org/wso2/carbon/apimgt/api/APIAdmin.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -675,7 +675,7 @@ List<KeyManagerConfigurationDTO> getGlobalKeyManagerConfigurations(String organi
* @param organization The organization name
* @param llmProviderId The ID of the LLM provider
* @return list of models registered under the LLM provider
* @throws APIManagementException
* @throws APIManagementException If an error occurs while retrieving the models
*/
List<String> getLLMProviderModels(String organization, String llmProviderId) throws APIManagementException;

Expand Down
27 changes: 22 additions & 5 deletions ...mgt/org.wso2.carbon.apimgt.api/src/main/java/org/wso2/carbon/apimgt/api/APIConstants.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,14 +44,17 @@ public class APIConstants {
public static final String ENDPOINT_SECURITY_PRODUCTION = "production";
public static final String ENDPOINT_SECURITY_SANDBOX = "sandbox";
public static final String ENDPOINT_CONFIG_SESSION_TIMEOUT = "sessionTimeOut";
public static final String OM_ELEMENT_NAME = "name";

public static class AIAPIConstants {
public static final int MILLISECONDS_IN_SECOND = 1000;
public static final String LLM_PROVIDERS = "llmProviders";
public static final String API_KEY_IDENTIFIER_TYPE_HEADER = "HEADER";
public static final String API_KEY_IDENTIFIER_TYPE_QUERY_PARAMETER = "QUERY_PARAMETER";
public static final String AI_API_REQUEST_METADATA = "AI_API_REQUEST_METADATA";
public static final String AI_API_RESPONSE_METADATA = "AI_API_RESPONSE_METADATA";
public static final String INPUT_SOURCE_PAYLOAD = "payload";
public static final String INPUT_SOURCE_HEADER = "header";
public static final String CONNECTOR_TYPE = "connectorType";
public static final String ID = "id";
public static final String NAME = "name";
Expand Down Expand Up @@ -82,12 +85,14 @@ public static class AIAPIConstants {
public static final String LLM_PROVIDER_SERVICE_METADATA_PROMPT_TOKEN_COUNT = "promptTokenCount";
public static final String LLM_PROVIDER_SERVICE_METADATA_COMPLETION_TOKEN_COUNT = "completionTokenCount";
public static final String LLM_PROVIDER_SERVICE_METADATA_TOTAL_TOKEN_COUNT = "totalTokenCount";
public static final String LLM_PROVIDER_SERVICE_METADATA_REMAINING_TOKEN_COUNT = "remainingTokenCount";
public static final String LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_MODEL = "$.model";
public static final String LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_PROMPT_TOKEN_COUNT = "$.usage" +
".prompt_tokens";
public static final String LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_COMPLETION_TOKEN_COUNT = "$.usage" +
".completion_tokens";
public static final String LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_TOTAL_TOKEN_COUNT = "$.usage.total_tokens";
public static final String LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_REMAINING_TOKEN_COUNT = "x-ratelimit-remaining-tokens";
public static final String LLM_PROVIDER_SERVICE_DEFAULT = "default";
public static final String NULL = "null";
public static final String LLM_PROVIDER = "LLM_PROVIDER";
Expand All @@ -96,12 +101,24 @@ public static class AIAPIConstants {
public static final String TRAFFIC_FLOW_DIRECTION_IN = "IN";
public static final String TRAFFIC_FLOW_DIRECTION_OUT = "OUT";
public static final String API_LLM_ENDPOINT = "_API_LLMEndpoint_";
public static final String TARGET_MODEL = "targetModel";
public static final String TARGET_ENDPOINT = "targetEndpoint";
public static final String SUSPEND_DURATION = "suspendDuration";
public static final String TARGET_MODEL = "TARGET_MODEL";
public static final String TARGET_ENDPOINT = "TARGET_ENDPOINT";
public static final String FAILOVER_TARGET_MODEL = "FAILOVER_TARGET_MODEL";
public static final String FAILOVER_TARGET_ENDPOINT = "FAILOVER_TARGET_ENDPOINT";
public static final String SUSPEND_DURATION = "SUSPEND_DURATION";
public static final String ENDPOINT_TIMEOUT = "ENDPOINT_TIMEOUT";
public static final String FAILOVER_ENDPOINTS = "FAILOVER_ENDPOINTS";
public static final String REJECT_ENDPOINT = "REJECT";
public static final String DEFAULT_ENDPOINT = "DEFAULT";
public static final String REMAINING_TOKEN_COUNT_HEADER = "x-ratelimit-remaining-tokens";
public static final String REMAINING_REQUEST_COUNT_HEADER = "x-ratelimit-remaining-requests";
public static final String EXIT_ENDPOINT = "EXIT";
public static final String REQUEST_PAYLOAD = "REQUEST_PAYLOAD";
public static final String REQUEST_HEADERS = "REQUEST_HEADERS";
public static final String REQUEST_HTTP_METHOD = "REQUEST_HTTP_METHOD";
public static final String REQUEST_REST_URL_POSTFIX = "REQUEST_REST_URL_POSTFIX";
public static final String REQUEST_MODEL = "REQUEST_MODEL";
public static final String CURRENT_ENDPOINT_INDEX = "CURRENT_ENDPOINT_INDEX";
public static final String DEFAULT_PRODUCTION_ENDPOINT_NAME = "DEFAULT PRODUCTION ENDPOINT";
public static final String DEFAULT_SANDBOX_ENDPOINT_NAME = "DEFAULT SANDBOX ENDPOINT";
public static final String ENDPOINT_SEQUENCE = "_EndpointsSeq";
}
}
8 changes: 4 additions & 4 deletions ...imgt/org.wso2.carbon.apimgt.api/src/main/java/org/wso2/carbon/apimgt/api/APIProvider.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1603,16 +1603,16 @@ APIEndpointInfo getAPIEndpointByUUID(String apiUUID, String endpointUUID, String
* @param apiUUID Unique identifier of API
* @param apiEndpoint API endpoint details
* @param organization Organization name
* @return created endpoint UUID
* @throws APIManagementException if an error occurs while inserting endpoint detail
* @return UUID of the added endpoint
* @throws APIManagementException if an error occurs while adding the endpoint
*/
String addAPIEndpoint(String apiUUID, APIEndpointInfo apiEndpoint, String organization)
throws APIManagementException;

/**
* Delete an API endpoint by providing the endpoint UUID.
* Delete endpoint by providing the endpoint UUID.
*
* @param endpointUUID Unique identifier of Endpoint
* @param endpointUUID Unique identifier of endpoint
* @throws APIManagementException if an error occurs while deleting the endpoint
*/
void deleteAPIEndpointById(String endpointUUID) throws APIManagementException;
Expand Down
26 changes: 20 additions & 6 deletions ...on.apimgt.api/src/main/java/org/wso2/carbon/apimgt/api/AzureOpenAiLLMProviderService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,24 +65,38 @@ public LLMProvider getLLMProvider()

List<LLMProviderMetadata> llmProviderMetadata = new ArrayList<>();
llmProviderMetadata.add(new LLMProviderMetadata(
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_MODEL,
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_REQUEST_MODEL,
APIConstants.AIAPIConstants.INPUT_SOURCE_PAYLOAD,
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_MODEL));
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_MODEL, false));
llmProviderMetadata.add(new LLMProviderMetadata(
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_RESPONSE_MODEL,
APIConstants.AIAPIConstants.INPUT_SOURCE_PAYLOAD,
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_MODEL, true));
llmProviderMetadata.add(new LLMProviderMetadata(
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_PROMPT_TOKEN_COUNT,
APIConstants.AIAPIConstants.INPUT_SOURCE_PAYLOAD,
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_PROMPT_TOKEN_COUNT));
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_PROMPT_TOKEN_COUNT, true));
llmProviderMetadata.add(new LLMProviderMetadata(
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_COMPLETION_TOKEN_COUNT,
APIConstants.AIAPIConstants.INPUT_SOURCE_PAYLOAD,
APIConstants.AIAPIConstants
.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_COMPLETION_TOKEN_COUNT));
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_COMPLETION_TOKEN_COUNT, true));
llmProviderMetadata.add(new LLMProviderMetadata(
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_TOTAL_TOKEN_COUNT,
APIConstants.AIAPIConstants.INPUT_SOURCE_PAYLOAD,
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_TOTAL_TOKEN_COUNT));
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_TOTAL_TOKEN_COUNT, true));
llmProviderMetadata.add(new LLMProviderMetadata(
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_REMAINING_TOKEN_COUNT,
APIConstants.AIAPIConstants.INPUT_SOURCE_HEADER,
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_REMAINING_TOKEN_COUNT, false));
llmProviderConfiguration.setMetadata(llmProviderMetadata);

// Set default model List
List<String> modelList = new ArrayList<>();
modelList.add("gpt-4o");
modelList.add("gpt-4o-mini");
modelList.add("o3-mini");
llmProvider.setModelList(modelList);

llmProvider.setConfigurations(llmProviderConfiguration.toJsonString());
return llmProvider;
} catch (Exception e) {
Expand Down
31 changes: 19 additions & 12 deletions ...t/org.wso2.carbon.apimgt.api/src/main/java/org/wso2/carbon/apimgt/api/ExceptionCodes.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -784,20 +784,27 @@ public enum ExceptionCodes implements ErrorHandler {
"Workflow status is not defined"),
RESOURCE_URI_TEMPLATE_NOT_DEFINED(902032, "Resource URI template value not defined", 400,
"Resource URI template value (target) not defined", false),
API_ENDPOINT_NOT_FOUND(902040, "Cannot find the required API Endpoint details.", 404,
API_ENDPOINT_NOT_FOUND(902040, "Cannot find the required API endpoint details.", 404,
"Requested API endpoint with id '%s' not found."),
ERROR_UPDATING_API_ENDPOINT_API(902041, "Error has occurred. Cannot update an API endpoint.", 500,
"Error when updating the API Endpoint."),
ERROR_INSERTING_API_ENDPOINT_API(902042, "Error has occurred. Fail to add an API endpoint to API.", 500,
"Error has occurred while inserting an API endpoint."),
ERROR_MISSING_ENDPOINT_CONFIG_OF_API_ENDPOINT_API(902043, "Missing mandatory API endpoint's endpoint config", 500,
"Required attributes %s for an API endpoint config specification %s are either missing or empty"),
ERROR_READING_API_ENDPOINTS_FILE(902044, "Error while reading API Endpoints from the endpoints file",
400, "Error while reading API Endpoints from the endpoints file"),
ERROR_ADDING_API_ENDPOINT(902045, "Error while adding API Endpoint to the API", 500,
"Error while adding API Endpoint with ID: %s to the API"),
ERROR_UPDATING_API_ENDPOINT(902041, "Error while updating the API endpoint.", 500,
"Error while updating the API endpoint."),
ENDPOINT_READONLY(902042, "API endpoint is read only", 400,
"API endpoint with UUID %s is read only"),
ERROR_ADDING_API_ENDPOINT(902043, "Failed to add endpoint to API.", 500,
"Error while adding API endpoint."),
ERROR_MISSING_ENDPOINT_CONFIG_OF_API_ENDPOINT_API(902044, "Mandatory endpoint config is missing " +
"in endpoint", 500, "Mandatory endpoint config is either missing or empty"),
ERROR_READING_API_ENDPOINTS_FILE(902045, "Error while reading API endpoints from the endpoints file",
400, "Error while reading API endpoints from the endpoints file"),
ERROR_ADDING_API_ENDPOINTS(902046, "Error while adding API Endpoints to the API", 500,
"Error while adding API Endpoints to the API");
"Error while adding API Endpoint to the API"),
ERROR_DELETING_API_ENDPOINT(902047, "Error while deleting API endpoint", 500,
"Error while deleting API endpoint with UUID '%s'."),
ERROR_DELETING_PRIMARY_API_ENDPOINT(902048, "Failed to delete API endpoint since endpoint is " +
"defined as a primary endpoint", 400,
"Failed to delete API endpoint with UUID '%s' since it is defined as a primary endpoint."),
API_ENDPOINT_URL_INVALID(902049, "Endpoint URL is invalid", 400,
"Endpoint URL is invalid");
private final long errorCode;
private final String errorMessage;
private final int httpStatusCode;
Expand Down
26 changes: 26 additions & 0 deletions ....wso2.carbon.apimgt.api/src/main/java/org/wso2/carbon/apimgt/api/LLMProviderMetadata.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,8 +31,16 @@ public class LLMProviderMetadata {
@JsonProperty("attributeIdentifier")
private String attributeIdentifier;

@JsonProperty("required")
private boolean required = true;

public LLMProviderMetadata() {}

/**
* @deprecated This constructor is deprecated. Use {@link #LLMProviderMetadata(String, String, String, boolean)}
* instead.
*/
@Deprecated
public LLMProviderMetadata(@JsonProperty("attributeName") String attributeName,
@JsonProperty("inputSource") String inputSource,
@JsonProperty("attributeIdentifier") String attributeIdentifier) {
Expand All @@ -42,6 +50,16 @@ public LLMProviderMetadata(@JsonProperty("attributeName") String attributeName,
this.attributeIdentifier = attributeIdentifier;
}

public LLMProviderMetadata(@JsonProperty("attributeName") String attributeName,
@JsonProperty("inputSource") String inputSource,
@JsonProperty("attributeIdentifier") String attributeIdentifier,
@JsonProperty("required") boolean required) {
this.attributeName = attributeName;
this.inputSource = inputSource;
this.attributeIdentifier = attributeIdentifier;
this.required = required;
}

public String getAttributeName() {

return attributeName;
Expand Down Expand Up @@ -71,4 +89,12 @@ public void setAttributeIdentifier(String attributeIdentifier) {

this.attributeIdentifier = attributeIdentifier;
}

public boolean isRequired() {
return required;
}

public void setRequired(boolean required) {
this.required = required;
}
}
26 changes: 20 additions & 6 deletions ...rbon.apimgt.api/src/main/java/org/wso2/carbon/apimgt/api/MistralAiLLMProviderService.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,24 +64,38 @@ public LLMProvider getLLMProvider()

List<LLMProviderMetadata> llmProviderMetadata = new ArrayList<>();
llmProviderMetadata.add(new LLMProviderMetadata(
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_MODEL,
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_REQUEST_MODEL,
APIConstants.AIAPIConstants.INPUT_SOURCE_PAYLOAD,
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_MODEL));
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_MODEL, false));
llmProviderMetadata.add(new LLMProviderMetadata(
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_RESPONSE_MODEL,
APIConstants.AIAPIConstants.INPUT_SOURCE_PAYLOAD,
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_MODEL, true));
llmProviderMetadata.add(new LLMProviderMetadata(
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_PROMPT_TOKEN_COUNT,
APIConstants.AIAPIConstants.INPUT_SOURCE_PAYLOAD,
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_PROMPT_TOKEN_COUNT));
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_PROMPT_TOKEN_COUNT, true));
llmProviderMetadata.add(new LLMProviderMetadata(
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_COMPLETION_TOKEN_COUNT,
APIConstants.AIAPIConstants.INPUT_SOURCE_PAYLOAD,
APIConstants.AIAPIConstants
.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_COMPLETION_TOKEN_COUNT));
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_COMPLETION_TOKEN_COUNT, true));
llmProviderMetadata.add(new LLMProviderMetadata(
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_TOTAL_TOKEN_COUNT,
APIConstants.AIAPIConstants.INPUT_SOURCE_PAYLOAD,
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_TOTAL_TOKEN_COUNT));
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_TOTAL_TOKEN_COUNT, true));
llmProviderMetadata.add(new LLMProviderMetadata(
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_REMAINING_TOKEN_COUNT,
APIConstants.AIAPIConstants.INPUT_SOURCE_HEADER,
APIConstants.AIAPIConstants.LLM_PROVIDER_SERVICE_METADATA_IDENTIFIER_REMAINING_TOKEN_COUNT, false));
llmProviderConfiguration.setMetadata(llmProviderMetadata);

// Set default model List
List<String> modelList = new ArrayList<>();
modelList.add("mistral-small-latest");
modelList.add("mistral-medium");
modelList.add("open-mistral-7b");
llmProvider.setModelList(modelList);
Comment on lines +92 to +97
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Verification agent

🧩 Analysis chain

Verify that the default models are up-to-date.

Please ensure that these are the latest supported models from Mistral AI.

🌐 Web query:

What are the latest available models from Mistral AI?

💡 Result:

As of February 2025, Mistral AI's latest models include several advanced offerings across different categories:

Core Models

  • Mistral Saba 25.02
    Released February 17, this model specializes in languages from the Middle East and South Asia while maintaining strong multilingual capabilities[3][4]. It balances performance with efficiency for regional applications.

  • Mistral Small v3 (25.01)
    A January release featuring:

    • Apache 2.0 license for open use
    • Competitive performance with larger models like Llama3-70B
    • Optimized latency (150 tokens/s)
    • High accuracy (81% MMLU)[9][10]

Specialized Models

  • Codestral 25.01
    January update to code-generation model:

    • Supports >80 programming languages
    • Enhanced speed (2.5× faster than predecessors)
    • Specializes in code completion/FIM/testing[1][2]

  • Pixtral Large (24.11)
    Multimodal November release combining:

    • Visual encoder for image understanding
    • Enhanced long-context handling
    • Integrated with text generation capabilities[3][7]

Edge-Optimized Models

  • Ministral Family
    October releases for local device deployment:
    Model Parameters Key Features
    Ministral 8B 8B 128k context for complex edge tasks
    Ministral3B 3B Low-latency mobile/robotics use[5][8]

Earlier November releases like Mistral Large24.11 remain current with improved reasoning/function calling[2], while foundational open-source models like Mathstral7B continue receiving updates[3].

These models reflect Mistral's focus on both frontier capabilities (through Large/Saba) and practical deployment efficiency (Small/Ministerial lines)[6][9].

Citations:

Update Default Model List to Reflect Latest Mistral AI Models

The current default list in MistralAiLLMProviderService.java (lines 92-97) contains:

  • "mistral-small-latest"
  • "mistral-medium"
  • "open-mistral-7b"

However, as of February 2025, the latest Mistral AI models include offerings such as Mistral Small v3 (25.01) and Mistral Saba 25.02, with no direct reference to a "medium" model or "open-mistral-7b". Please review and update the default model list to align with the current model naming and versions provided by Mistral AI.


llmProvider.setConfigurations(llmProviderConfiguration.toJsonString());
return llmProvider;
} catch (Exception e) {
Expand Down
Loading