add deepseek-ai/deepseek-r1 on Replicate #1151
Conversation
zeke
requested review from
julien-c,
hanouticelina,
SBrandeis and
coyotte508
as code owners
| @@ -5,6 +5,9 @@ export const REPLICATE_API_BASE_URL = "https://api.replicate.com"; | |||
| type ReplicateId = string; | |||
|
|
|||
| export const REPLICATE_SUPPORTED_MODEL_IDS: ProviderMapping<ReplicateId> = { | |||
| conversational: { | |||
There was a problem hiding this comment.
I chose conversational here as the task type based on the existing Together AI code.
| @@ -930,6 +930,19 @@ describe.concurrent("HfInference", () => { | |||
| expect(res).toBeInstanceOf(Blob); | |||
| }); | |||
|
|
|||
| it("conversational unversioned", async () => { | |||
There was a problem hiding this comment.
Could use some help getting this test right.
|
Hi @zeke, it seems like Replicate does not support the chat completion API (or at least it's not documented on https://replicate.com/deepseek-ai/deepseek-r1/api/schema). By "chat completion" I mean the standard API introduced by OpenAI and now supported by many providers: https://platform.openai.com/docs/api-reference/chat/create. The main difference between this API and a classic "text-generation" API is that the user can pass a list of messages instead of a pre-formatted prompt. Users don't need to play with the chat template client side. So in order to have support for DeepSeek R1 on Replicate via HF, you should either set it as a "text-generation" task (but that wouldn't make it appear in the Widget on https://huggingface.co/deepseek-ai/DeepSeek-R1) or support the Chat Completion API on Replicate. (Note: "Message API", "Conversational", "Chat completion" all refer to the same API schema) |
|
Thanks for the context @Wauplin, that's very helpful. Let me take this back to the team and see what we can do. |
|
if it's public info, what are you running backend-wise to power R1 inference? might have an OpenAI compatible server |
This PR adds support for https://replicate.com/deepseek-ai/deepseek-r1
https://x.com/ClementDelangue/status/1884274486905827431
cc @bfirsh