Skip to content

Conversation

@smorad
Copy link
Owner

@smorad smorad commented Oct 7, 2025

This modifies the equinox S6 model to more closely match how the original paper discretizes the B matrix: https://arxiv.org/pdf/2312.00752. The results are very similar, but we should be consistent with the paper. Increasing the dimensionality of dt also seems to help quite a bit, and so we set dt to be recurrent_size by default. As a result, we split the S6 model into the more aptly named S6 and S6D (diagonal) modules.

@smorad smorad merged commit bcb813e into main Oct 8, 2025
1 check passed
@smorad smorad deleted the s6-fix branch October 8, 2025 06:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants