Streamlit editor thresholds/weights are not applied to IQB score calculation

[atulbhardwaj-io]

Bug: Streamlit editor thresholds/weights are not applied to IQB score calculation

Summary

The Streamlit prototype allows users to edit thresholds and weights, but the IQB score shown in the app is still computed using the default library config.

This creates a correctness problem: UI customization appears to work, but score outputs do not reflect those settings.

Affected area

• prototype/Home.py
• prototype/pages/IQB_Map.py
• prototype/utils/calculation_utils.py
• library/src/iqb/calculator.py
• prototype/app_state.py

Evidence from code

1. A helper that builds a custom config from session state exists:
   • prototype/utils/calculation_utils.py:25 (get_config_with_custom_settings)
2. That helper is not used anywhere else in the repo.
3. Score computation calls the default calculator directly:
   • prototype/Home.py:72
   • prototype/pages/IQB_Map.py:273
   • prototype/pages/IQB_Map.py:1632
   • prototype/pages/IQB_Map.py:1766
4. IQBCalculator currently cannot accept non-default config:
   • library/src/iqb/calculator.py:22
   • library/src/iqb/calculator.py:28 raises NotImplementedError for non-None config.
5. App state initializes calculator as default:
   • prototype/app_state.py:44 (iqb: IQB = field(default_factory=IQB))

Reproduction steps

1. Run the prototype:
   • cd prototype
   • uv run streamlit run Home.py
2. Open the settings/editor controls and change a requirement threshold or weight significantly.
3. Keep measurement inputs fixed.
4. Observe the displayed IQB score.

Proof of concept

Use the Home page with fixed manual measurements:

• Download: 15.00
• Upload: 20.00
• Latency: 75.00
• Packet Loss: 0.700

Then apply strict thresholds in Modify Configurations -> Thresholds by Use Case.
Repeat for each use case tab (Web Browsing, Video Streaming, Audio Streaming, etc.):

• Download (Mbps): 500
• Upload (Mbps): 500
• Latency (ms): 5
• Packet Loss (%): 0.100

Attach three screenshots to the issue:

1. proof_before_score.png (fixed inputs + baseline score).
2. proof_config_changed.png (threshold values changed as above).
3. proof_after_score.png (same fixed inputs + score after changes).

Example result text to include under screenshots:

• Before: score = <baseline_score>
• After strict thresholds: score = <after_score>
• Expected: score should decrease significantly.
• Actual: score remains unchanged (or nearly unchanged), indicating editor changes are not applied to score computation.

Expected behavior

Changing thresholds/weights in the editor should change the IQB score if input values are near affected thresholds or if weights are rebalanced.

Actual behavior

Score is computed via state.iqb.calculate_iqb_score(...) with default calculator config; editor state changes are not applied to that calculation path.

Impact

• User-visible correctness bug in the prototype.
• Misleading results for researchers/users testing policy/weight scenarios.
• Reduced trust in “what-if” settings UI.

Proposed fix (implementation direction)

1. Add support for config dict injection in IQBCalculator:
   • Accept dict config in constructor / set_config.
2. In prototype score path, build updated config from state:
   • Use get_config_with_custom_settings(state).
3. Compute score using a calculator instance configured with that updated config.
4. Keep visualization and score paths consistent (both should use same effective settings).
5. Add tests:
   • Unit: score changes when threshold/weight changes.
   • Integration/prototype: editor state affects score output path.

Acceptance criteria

• Changing a threshold in editor changes computed score for a crafted input case.
• Changing requirement/use-case weights changes computed score for a crafted input case.
• No regression when editor values are default (scores match current baseline).
• Tests cover custom-config scoring path.

Suggested labels

• bug
• prototype
• good first issue
• help wanted (optional)

[bassosimone]

I appreciate that you opened an issue describing the problem. If you also have bandwidth to fix this, it would be great to read a diff that solves this issue. Thank you!

[atulbhardwaj-io]

Thanks @bassosimone for your encouragement. I’ve submitted a PR for this issue.

PR: #160

I’d really appreciate your review and judgment on whether this is the right direction.