Handle hf_inference in single file #2766
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hanouticelina
approved these changes
Jan 20, 2025
hanouticelina
merged commit 20e8d3a
into
inference-providers-compatibility
11 of 15 checks passed
hanouticelina
deleted the
inference-providers-compatibility-suggestion
branch
hanouticelina
added a commit
that referenced
this pull request
Jan 23, 2025
* Add first version of third-party providers support * add task level in model id mappings * raise error when task is not supported by a provider + some improvements * small (big) refactoring * multiple fixes * add hf inference tasks * Handle hf_inference in single file (#2766) * harmonize prepare_payload args and add automatic-speech-recognition task * backward compatibility with custom urls * first draft of tests * InferenceClient as fixture + skip if no api_key * give name to parametrized tests * upload cassettes * make quali * download sample files from prod * fix python3.8 * small improvement for better readability Co-authored-by: Lucain <lucain@huggingface.co> * make style * fixing more tests * test url building * fix and record async client tests * re-add cassettes * fix * add cassettes back * fix test * hopefully this will fix the test * fix sentence similarity test --------- Co-authored-by: Lucain <lucain@huggingface.co> Co-authored-by: Lucain Pouget <lucainp@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
cc @hanouticelina this is a suggestion for PR #2757
Main changes are:
provider_helper.prepare_payloadshould not have aexpect_binaryparameter => since theprovider_helperdepends on the task, we already know if the input must be a binary or notmodel: Optional[str]required inprovider_helper.prepare_payloadfor all providers and all tasks. At the moment themodelis only passed if we know the provider will use it. But in practice theInferenceClientand providers implementation should be independent. Hence passing it all the time and letting the provider decide what to do with the info.hf-inferenceprovider. This reduces by a lot the complexity / duplicated code. I'm definingHfInferenceTaskfor generic tasks,HFInferenceBinaryInputTaskfor tasks which require a binary input (so no need forexpect_binary: boolanymore) andHFInferenceConversationalfor specific task.Let me know what you think. We can either discuss some changes here or merge it and continue on your PR.