Add support for using basic block context to GRANITE models. #290
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Add fields to the `proto` specification to store context. * Add members to the Gematria `BasicBlock` data structure to store context and update methods on it and its Python binding accordingly. * Bonus: Remove dangling TODO.
* Update the graph builder and its Python bindings to add context instructions to basic block graphs and store context node mask to later be used by models. * Add tests for the new graph builder functionality.
* Update seq2seq models to only use non-context nodes as input to the readout network. * Add tests for training seq2seq models with context.
ondrasej
reviewed
Feb 7, 2025
Collaborator
ondrasej
left a comment
ondrasej
left a comment
There was a problem hiding this comment.
LGTM overall. I'll approve once the parent ones are ready (merged).
| basic_block_was_added = self._batch_graph_builder.add_basic_block(block) | ||
| # Add context to the basic block graph only for seq2seq models. | ||
| basic_block_was_added = self._batch_graph_builder.add_basic_block( | ||
| block, add_context=self.use_deltas |
Collaborator
There was a problem hiding this comment.
No action needed in this PR: I think we should either completely disable the model where use_deltas is False, or add the context mask info somehow to the update functions. Otherwise, these models would have a hard time knowing which nodes belong to the context...
But given the model precision, I'd go with just disabling the model without deltas :)
virajbshah
changed the base branch from
main
to
experiment/preceding-following-context
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
readout network.