Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Added Adapters to Whisper Model from openai #572

Closed

Conversation

HenningBuhl
Copy link

This PR adds adapter support for the Whisper model from openai.

This feature has been requested in an issue before (#466).

@calpt
Copy link
Member

calpt commented Sep 9, 2023

Hey, thanks for your efforts in contributing new model architectures to adapter-transformers and sorry for the silence on our side.

In the last few weeks, we've been working on a large refactoring of our project, which will ultimately result in the release of Adapters, the next-generation adapters library. See #584.

As a consequence, we plan to merge any new model integrations directly to the new codebase, which currently can be found on this branch. Unfortunately, this necessitates some changes in the model integration code (detailed here, see already integrated models such as BERT, BART etc. for reference).

If you'd be willing to update your model integration to target the new library yourself, we'd be super happy to help you on this. Otherwise, we might look into upgrading and merging some of the open model integration PRs ourselves in the future. For more details, again see #584.

@TimoImhof
Copy link
Contributor

Hi, thanks again for your efforts in contributing Whisper as a new model to our adapters library.
We have decided to build upon your contribution and adapt your implementation to the new codebase in a new PR (see #693).

@TimoImhof TimoImhof closed this Apr 30, 2024
TimoImhof added a commit that referenced this pull request Aug 8, 2024
This PR adds adapter support for the Whisper model from openai and
builds upon work done previously in #572.

Key Additions:

1. Adapter Support for Whisper Model:

- Incorporates adapter functionality to enhance the flexibility and
adaptability of the Whisper model.

2. Enhanced Head Functions:
- Expanded the argument options for some heads by adding a layer
argument with a default value.

3. Preprocessing Scripts for Audio Datasets:

- Added preprocessing scripts tailored for audio datasets.
- These scripts are now utilized in the Whisper tests within the test
suite, replacing the use of arbitrary samples.

---------

Co-authored-by: Leon Engländer <leon.englaender@gmail.com>
Co-authored-by: calpt <calpt@mail.de>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants