Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The paper on the Bone structure has been updated #2312

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

JL-er
Copy link
Contributor

@JL-er JL-er commented Jan 8, 2025

The new version of the paper is more reliable and easier for readers to understand.
https://arxiv.org/abs/2409.15371

@BenjaminBossan
Copy link
Member

Thanks for the update.

I haven't compared the new paper, but checking the PR: Is DiSHA a method that could be used on its own, but is not available in PEFT, or is it the union of Bone and Bat? What I'd like to avoid is for users to be confused when they want to use DiSHA but don't find any method with that name in PEFT. Maybe this could be clarified in the description.

@JL-er
Copy link
Contributor Author

JL-er commented Jan 8, 2025

Simply put, DiSHA is the overall framework, while Bone and Bat are just subsets of it. So I even want to rename Bone to DiSHA, and then choose either Bone or Bat during init_weights.

@BenjaminBossan
Copy link
Member

So I even want to rename Bone to DiSHA, and then choose either Bone or Bat during init_weights.

When it comes to the PEFT code, let's avoid any renaming, as this would break backwards compatibility.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants