🐛[Bug] : Calibration Failure with Low Sample Counts (e.g., 5x10)

[thechillbasu]

Description

When performing calibration with configurations involving low sample counts (e.g., 5 points × 10 samples), the application crashes after clicking the "FINISH" button. The user is not redirected, and the console displays a TypeError.

Steps to Reproduce

1. Set the calibration configuration to 5 points and 10 samples/point (or similar low-sample configs like 3x12, 4x15).
2. Run the calibration sequence.
3. Complete the calibration/validation process.
4. Click the "FINISH" button.
5. Observation: The application crashes and does not redirect. Console shows TypeError: Cannot read properties of undefined (reading 'PrecisionSD').

Environment

• Component: app/services/gaze_tracker.py
• Browser: All browsers
• OS: All OS

Root Cause Analysis

The issue resides in the backend's data splitting logic (train_test_split in gaze_tracker.py).

By default, the split is random and not stratified. It selects test samples from the entire pooled dataset of all points. With random_state=42, the random selection deterministically excludes all samples of specific points from the Test Set for the following configurations:

Points	Samples per Point	Missing Point (Excluded from Test Set)
3	12	Point 0
4	15	Point 1
5	10	Point 0
6	10	Point 2
8	10	Point 5
8	17	Point 5

Why this happens:
The train_test_split function treats the dataset as a flat list. With random_state=42, the Pseudo-Random Number Generator (PRNG) produces a fixed sequence of indices for the Test Set.

• For 5x10 (50 total samples), the indices selected for the Test Set happen to completely skip the index range [0-9] (which corresponds to Point 0).
• Since no index from [0-9] is picked, Point 0 has 0 samples in the Test Set.
• This is mathematically deterministic for these specific Sample/Point combinations with seed 42.

When a point is missing from the Test Set:

1. No predictions are generated for that point.
2. The backend returns incomplete results (missing keys for that point).
3. The frontend attempts to access metrics for that point (PrecisionSD) and crashes because the key is undefined.

Related Issue

• Frontend Rendering Bug: If testing with fewer than 4 points (e.g., 2 or 3 points), you may also encounter an issue where calibration points fail to render. See the corresponding Frontend Issue and PR (Issue #110 and PR #111) for the fix (generateCalibrationPattern).

Solution

The fix involves implementing stratified splitting. By stratifying the split based on point IDs, we enforce that every point is represented in both the Training and Test sets proportionally. This guarantees that metrics can be calculated for all calibration points, preventing the crash.

[thechillbasu]

Hey @jvJUCA @marcgc21 ! Kindly have a look into this. Thanks!