-
Notifications
You must be signed in to change notification settings - Fork 3
use small datasets #88
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Definitely a step in the right direction. I think we can simplify the fixtures a bit, see here: #90
tests/test_idpi/test_ninjo_k2th.py
Outdated
@@ -36,7 +36,7 @@ def test_ninjo_k2th(data_dir, fieldextra): | |||
assert_allclose( | |||
fs_ds["POT_VORTIC_AT_THETA"], | |||
observed_at_theta["pot_vortic"], | |||
atol=1e-9, | |||
atol=1e-5, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why do we need to decrease the absolute tolerance here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure why this was necessary. Especially that 1e-5 is at the order of magnitude of the observable signal in the pot vortic field.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could be maybe an edge effect due to the differential operators and the reduced computational domain.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I do not know either, but I think a different location of boundaries can affect the differential operators, (since the stencil applied is different at the boundaries), even if the data is the same in the cropped domain.
I checked again and could increase this threshold again until 1e-8. But still the one below (1e-4) fails if I try to relax it.
However I still think that 1e-4 is still in the "ok" range, provide we do not really know how these operators propagate errors.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Considering that the expected values are in the 1e-5 range, it doesn't really make sense to me to put in an absolute tolerance of 1e-5. That would be comparable to setting an atol of 1e5 for the pressure field. The value for the rtol is fine in my opinion.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just realised that fieldextra is probably applying the same stencils on the same data so the argument about the edge effects does not hold.
## Purpose Run tests on a reduced test dataset by default to speed up the test suite. ## Code changes: - Added the custom pytest marker `data` - Remove the `system_defintion` module Superseeds #88
Purpose
Change the default datasets used for testing to a small dataset.
Code changes:
Checklist
Before submitting this PR, please make sure:
Additionally, if the PR updates the version of the package
Review
For the review process follow the guidelines at Checklist