-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Remove grouping trick #205
Conversation
Ran with and without trick on a gaussian field with 1 million parameters, 50 observations and 25 realizations. The sigma=10 controls the smoothness of the field.
I also ran Drogon with and without the trick and it runs much faster without the trick |
52445e4
to
a977d57
Compare
9d672d3
to
5ad992d
Compare
continue | ||
# Loop only over rows with significant correlations | ||
for indices_of_row in np.where(significant_rows)[0]: | ||
unique_row = significant_corr_XY[indices_of_row] | ||
|
||
# Get the parameters (X) that have identical significant responses (Y) | ||
X_subset = X[indices_of_row, :] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Since indices_of_row
is now an int, this variable X_subset
is now a 1D array. I am surprised that an error is not raised or that the code does not fail? Since X
or a subset should be a 2D array.
Perhaps look at this quickly before merging. Also, perhaps rename
indices_of_row -> parameter_num
unique_row -> correlated_responses
or something like that. No other comments from me. Looks good.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have not looked into it deeply but I'm assuming broadcasting is helping us out in this: X_subset + cov_XY_subset @ T
.
Thanks for the better variable names 👍
5ad992d
to
ca1b6e0
Compare
I want to run a final test using Drogon before merging, but we have some problems with |
ca1b6e0
to
fee5737
Compare
It seems to only be faster when the Gaussian Field is very smooth, which is not realistic for real fields.
fee5737
to
124fd46
Compare
Resolves: #200