Apply custom thresholds and weights to Streamlit IQB score computation#160
Open
atulbhardwaj-io wants to merge 1 commit intom-lab:mainfrom
Open
Apply custom thresholds and weights to Streamlit IQB score computation#160atulbhardwaj-io wants to merge 1 commit intom-lab:mainfrom
atulbhardwaj-io wants to merge 1 commit intom-lab:mainfrom
Conversation
4 tasks
Collaborator
|
After listing our project for GSoC, we received a large amount of pull requests across several repositories. We are dealing with the backlog, but this would take time. We will get back to this pull request eventually. In the meanwhile, if you are a GSoC applicant, please read our updated GSoC policy: https://github.com/m-lab/gsoc/. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR fixes a correctness bug in the Streamlit prototype where edited thresholds/weights in the configuration editor were not applied to IQB score computation.
Why this was happening
The editor updated thresholds/weights in Streamlit session state, but score calculation paths still used the default calculator instance (
state.iqb) initialized with default config.So UI changes were visible, but the score logic ignored them.
Root cause
state.iqb.calculate_iqb_score(...)directly.state.iqbused default config only.IQBCalculator.set_configdid not support in-memory dict config.What changed
library/src/iqb/calculator.pyprototype/utils/calculation_utils.pycalculate_iqb_score_with_custom_settings(...)helper usage in scoring paths.prototype/Home.pyandprototype/pages/IQB_Map.py.library/tests/iqb/calculator_test.py.prototype/tests/calculation_utils_test.py.Validation
Automated tests
Ran:
uv run pytest library/tests/iqb/calculator_test.py prototype/tests/calculation_utils_test.pyResult:
30 passed in 4.02sKey tests:
calculator_test.py::TestIQBCalculatorScoreCalculation::test_calculate_iqb_score_changes_with_custom_thresholdscalculation_utils_test.py::test_calculate_iqb_score_with_custom_settings_changes_resultBehavior proof (script)
With fixed baseline inputs and strict thresholds:
baseline=0.571429strict=0.000000decreased=TrueManual proof of concept
Fixed inputs:
15.0020.0075.000.700Strict thresholds (all use-case tabs):
50050050.100Observed:
Screenshots
Before score (

proof_before_score.png):Config changed (

proof_config_changed.png):After score (

proof_after_score.png):Impact
Issue
Closes #158