Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

LiteLLM call to Sagemaker failing #9157

Open
swagulkarni opened this issue Mar 12, 2025 · 0 comments
Open

LiteLLM call to Sagemaker failing #9157

swagulkarni opened this issue Mar 12, 2025 · 0 comments

Comments

@swagulkarni
Copy link

I am trying to invoke model endpoint hosted on Sagemaker. Here is my entry in config.yaml

model_list:
  - model_name: jumpstart-model
    litellm_params:
      model: sagemaker/jumpstart-dft-meta-textgeneration-l-20250311-203442
      aws_access_key_id: <Access_key_id>
      aws_secret_access_key: <Secret_access_key>
      aws_region_name: us-east-1

Here is the curl command

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data ' {
      "model": "jumpstart-model",
      "hf_model_name": "meta-llama/Llama-2-7b-chat-hf",
      "messages": [
        {
          "role": "user",
          "content": "what llm are you"
        }
      ]
    }
'

But getting this error

{"error":{"message":"litellm.ServiceUnavailableError: SagemakerException - Too little data for declared Content-Length. Received Model Group=jumpstart-model\nAvailable Model Group Fallbacks=None","type":null,"param":null,"code":"503"}

What is the root cause here?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant