-
Notifications
You must be signed in to change notification settings - Fork 89
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support where-clause evaluation in registration annotations #3841
Merged
stress-tess
merged 15 commits into
Bears-R-Us:master
from
jeremiah-corrado:annotation-where-clause-evaluation
Oct 21, 2024
Merged
Support where-clause evaluation in registration annotations #3841
stress-tess
merged 15 commits into
Bears-R-Us:master
from
jeremiah-corrado:annotation-where-clause-evaluation
Oct 21, 2024
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… register_commands.py Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
…ateAndRegister Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
…re clauses Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
stress-tess
requested review from
ajpotts,
stress-tess and
bmcdonald3
and removed request for
ajpotts and
stress-tess
October 16, 2024 17:33
bmcdonald3
approved these changes
Oct 16, 2024
ajpotts
approved these changes
Oct 16, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks! Look forward to using this!
stress-tess
approved these changes
Oct 17, 2024
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for doing this jeremiah!
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Oct 17, 2024
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Oct 17, 2024
github-merge-queue bot
pushed a commit
that referenced
this pull request
Oct 17, 2024
* add primitive where-clause evaluation (for instantiateAndRegister) to register_commands.py Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * remove uninstantiated overloads from commands annotated with instantiateAndRegister Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * remove match statements from register_commands.py Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * add error message for 'pad' w/ bigint Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * add preliminary support for where-clause evaluation in 'registerCommand' Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * fix misinterpretation of explicit type widths (e.g., uint(64)) in where clauses Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * fix error message Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * fix error message Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> --------- Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
github-merge-queue
bot
removed this pull request from the merge queue due to failed status checks
Oct 17, 2024
Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
…d in ReductionMsg. Fix 'axis' argument in 'ak.sum' Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
stress-tess
approved these changes
Oct 21, 2024
github-merge-queue
bot
removed this pull request from the merge queue due to no response for status checks
Oct 21, 2024
drculhane
pushed a commit
to drculhane/arkouda
that referenced
this pull request
Oct 23, 2024
…Us#3841) * add primitive where-clause evaluation (for instantiateAndRegister) to register_commands.py Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * remove uninstantiated overloads from commands annotated with instantiateAndRegister Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * remove match statements from register_commands.py Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * add error message for 'pad' w/ bigint Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * add preliminary support for where-clause evaluation in 'registerCommand' Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * fix misinterpretation of explicit type widths (e.g., uint(64)) in where clauses Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * fix error message Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * fix error message Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * refactor 'sum' to not rely on where-clauses for dispatching Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * refactor prod, max, and min to take their 'axis' argument as a list Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * fix mypy errors and multi-dim build failure Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * create list-specific implementations of aryUtil helper procedures used in ReductionMsg. Fix 'axis' argument in 'ak.sum' Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * fix return type change to reduction procedures Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> * fix reducedShape helper for lists Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com> --------- Signed-off-by: Jeremiah Corrado <jeremiah.corrado@hpe.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Add (primitive) support for where-clause evaluation to
registerCommand
andinstantiateAndRegister
. Both annotations now evaluate the procedure's where-clause and will only generate instantiations for sets of generic parameters where the where-clause would evaluate totrue
.The primary goal of this change is to be able to remove extraneous overloads of command procedures that simply return an error message indicating that some set of types and/or array ranks are not supported for that command. For example, in the following code snippet, the second overload of
asdf
can now be removed because the build system won't attempt to generate an instantiation for it:This feature is not totally general-purpose yet. Current limitations:
param
-function calls are currently supported. Specifically, support forisIntegralType
and other related procedures from Chapel's Types module is hard-coded in, but any other function calls within a where-clause will not workparam
ortype
methods/fields are accessibleetype
orrank
directly is not yet supported (ex:where x.etype != bool && x.rank < 3
)param
ortype
fields of a non-array argument is not yet supported (ex:where mySparseSym.matLayout == Layout.CSR
)Eventually,
registerCommands.py
should be modified to use the Compiler's resolver to speculatively evaluate where-clauses in a much more robust manner. This would in principle remove the above limitations, but is likely a more significant development effort than the solution in this PR.Limitations aside, this PR allows most of the extraneous procedures currently in Arkouda to be removed.
Note: one can opt out of this feature by setting the
ignoreWhereClause
argument in the annotation call. This could be useful in an example like the one below, or to get around one of the limitations listed above.(resolves: #3837)