Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hash un-numeric input when used as PRNG seed, fixes #800 #801

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

brontolosone
Copy link

Closes #800

As discussed on Slack, but there may be more to discuss!

What has been done to verify that this works as intended?

Ran the tests, added some tests too. Ideally I'd put this JR version in an ODKCollect to witness some choice randomization change when altering the seed value in this test form, but I didn't manage that today ;-)

Why is this the best possible solution? Were any other approaches considered?

The nice thing is that it doesn't change existing randomizations that were working well (eg, strings interpretable as numerics). It only changes randomizations for which the seed wasn't working (eg, strings not interpretable as numerics).

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

In very specific use cases this can cause an unwelcome change.

For instance: In some survey the randomization of the order of choice fields is part of the survey. Also, it wasn't actually working due to issue #800, but anyway. Now if ODKCollect is updated with this new OpenRosa version halfway through the survey, some users will have seen randomization X for seed Z while others will have seen randomization Y for that same seed Z. If the research draws conclusions based on either X or Y, then the researchers need to know which version of ODKCollect was used so they know (or can deduce) which choice randomization went with seed Z.

Do we need any specific form for testing your changes? If so, please attach one.

as mentioned above, running ODKCollect on this form and witnessing the choice list reshuffle based on string input variations in the seed field, would be extra convincing.

Does this change require updates to documentation? If so, please file an issue here and include the link below.

Maybe. We could say something like:

"As of ODKCollect version X, the choice randomization based on using an RNG seed from some other field actually works for more inputs (notably, any text). If you have ongoing research that depends on this type of randomization, and where the analysis is dependent on the specific choice list ordering that used to go with certain seeds, you may want to test with this new version of ODKCollect to see if your choice list randomization will change, and decide to not upgrade ODKCollect while this research is ongoing".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Seed for randomization derived from non-numeric-looking string is always 0
1 participant