Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

FlashFFTConv can be definitely be implemented on Mamba, right? #23

Open
5thGenDev opened this issue Mar 6, 2024 · 2 comments
Open

FlashFFTConv can be definitely be implemented on Mamba, right? #23

5thGenDev opened this issue Mar 6, 2024 · 2 comments

Comments

@5thGenDev
Copy link

No description provided.

@DanFu09
Copy link
Contributor

DanFu09 commented Mar 6, 2024

Mamba does not have a convolutional form, so there isn't an exact mapping. For Mamba you'll have to use the scan formulation as documented in the paper.

@5thGenDev
Copy link
Author

For Mamba you'll have to use the scan formulation as documented in the paper.
I get what you mean (algorithm 2 in figure 2). I was confused a bit because figure 3 showing mamba architecture also has Conv layer???

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@DanFu09 @5thGenDev