Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bedrock performanceConfig #57

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Bedrock performanceConfig #57

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1880cc6
add performanceConfig in bedrock
Balamurugan355 Mar 10, 2025
c93bc16
add performanceConfig in bedrock
Balamurugan355 Mar 10, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions superagentx/llm/bedrock.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -84,6 +84,11 @@ def chat_completion(
if tools is not None:
kwargs["toolConfig"] = {"tools": tools}

if chat_completion_params.performance_config:
kwargs["performanceConfig"] = {
"latency": "optimized"
}

response = self.client.converse(**kwargs)

logger.debug(f"Bedrock Message ==> Bedrock no tool {response} ")
Expand Down Expand Up @@ -160,6 +165,11 @@ async def achat_completion(
if tools is not None:
kwargs["toolConfig"] = {"tools": tools}

if chat_completion_params.performance_config:
kwargs["performanceConfig"] = {
"latency": "optimized"
}

response = self.client.converse(**kwargs)

logger.debug(f"Bedrock Message ==> Bedrock no tool {response} ")
Expand Down
5 changes: 5 additions & 0 deletions superagentx/llm/models.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -97,3 +97,8 @@ class ChatCompletionParams(BaseModel):
description="Unique identifier for end-user monitoring and abuse detection.",
default=f'{uuid.uuid4()}'
)
performance_config: bool | None = Field(
description="Latency-optimized inference for foundation models in Amazon Bedrock delivers faster response"
" times and improved responsiveness for AI applications",
default=None
)
14 changes: 7 additions & 7 deletions superagentx/llm/types/base.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,11 +87,11 @@ def __validate_model_name__(self) -> Self:
)
logger.error(_msg)
raise ValueError(_msg)
elif self.model not in BEDROCK_MODELS:
_msg = (
f'Invalid Bedrock model - '
f'{self.model}. It should be one of the following {", ".join(BEDROCK_MODELS)}'
)
logger.error(_msg)
raise ValueError(_msg)
# elif self.model not in BEDROCK_MODELS:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why this commented?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@syedhabib39 The bedrock model names can have dynamic values from AWS Bedrock. Hence, validation in the Pydantic class may prevent to allow dynamic values.

# _msg = (
# f'Invalid Bedrock model - '
# f'{self.model}. It should be one of the following {", ".join(BEDROCK_MODELS)}'
# )
# logger.error(_msg)
# raise ValueError(_msg)
return self