You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
While running benchmarks on selFIe, I found that the test selfie_hellovar.sh is flaky, i.e. it fails sometimes.
The test selfie_hellovar.sh saves the current timestamp into the SELFIE_TEST variable. After the code under test finishes, it searches for the timestamp inside the error output of the code under test.
However, and I believe this causes the flaky behaivior, if the timestamp changes between the test itself and the code under test reading the current timestamp (e.g. by the test taking longer than a second or the test being started close to the "end of a second"), the grep call fails, causing the test to fail.
The text was updated successfully, but these errors were encountered:
While running benchmarks on selFIe, I found that the test
selfie_hellovar.sh
is flaky, i.e. it fails sometimes.The test selfie_hellovar.sh saves the current timestamp into the SELFIE_TEST variable. After the code under test finishes, it searches for the timestamp inside the error output of the code under test.
However, and I believe this causes the flaky behaivior, if the timestamp changes between the test itself and the code under test reading the current timestamp (e.g. by the test taking longer than a second or the test being started close to the "end of a second"), the
grep
call fails, causing the test to fail.The text was updated successfully, but these errors were encountered: