Skip to content

[TF FE][Tests] Fix non-deterministic FILM model test failure#33977

Open
evkotov wants to merge 2 commits intoopenvinotoolkit:masterfrom
evkotov:CVS-180301
Open

[TF FE][Tests] Fix non-deterministic FILM model test failure#33977
evkotov wants to merge 2 commits intoopenvinotoolkit:masterfrom
evkotov:CVS-180301

Conversation

@evkotov
Copy link
Contributor

@evkotov evkotov commented Feb 5, 2026

Details:

The test test_tf_hub_api_notebook1[CPU-film] was failing randomly in CI with shape mismatch error:
ValueError: operands could not be broadcast together with shapes (1,200,200,2) (1,6,6,2)

Root cause: The test used list(film_layer.values())[0] to select the model output. This is incorrect because:

  1. Python dict iteration order depends on hash randomization (PYTHONHASHSEED), which is random by default since Python 3.3
  2. The FILM model returns 7 different outputs including flow pyramids with various shapes
  3. When dict iteration returns a flow pyramid instead of 'image', the shapes don't match

The fix selects the output by name (film_layer['image']) instead of by position.

Why the fix is in the test, not in OpenVINO TF Frontend

Fixing this in OpenVINO is not possible because TensorFlow itself does not guarantee output order:

  1. SavedModel signature uses protobuf map<string, TensorInfo> for outputs, which has no defined iteration order per protobuf specification
  2. TensorFlow documentation states that signature inputs/outputs are "identified by string names", not by position
  3. Even within pure TensorFlow (without OpenVINO), the dict key order changes between runs depending on PYTHONHASHSEED

OpenVINO cannot preserve an order that TensorFlow does not define. The only reliable way to select a specific output is by name.

Tickets:

  • 180301

The test test_tf_hub_api_notebook1[CPU-film] was randomly failing with
"ValueError: operands could not be broadcast together with shapes
(1,200,200,2) (1,6,6,2)" due to non-deterministic output selection.

Root cause:
- The test used list(film_layer.values())[0] to select the first output
- Python dict iteration order is non-deterministic (PYTHONHASHSEED)
- TensorFlow SavedModel signature uses protobuf map (no order guarantee)
- Different runs could return different outputs (image vs flow pyramid)

Fix:
- Select output by name (film_layer['image']) instead of by position
- The 'image' output is the interpolated frame - the main FILM result

Investigation showed that fixing this in TF Frontend is impossible because:
1. TensorFlow itself doesn't guarantee output order in SavedModel signatures
2. Protobuf map fields have undefined iteration order
3. Python dict hash randomization affects order at runtime

CVS-180301
@evkotov evkotov self-assigned this Feb 5, 2026
@evkotov evkotov requested a review from a team as a code owner February 5, 2026 09:04
@github-actions github-actions bot added the category: TF FE OpenVINO TensorFlow FrontEnd label Feb 5, 2026
@evkotov evkotov requested a review from rkazants February 5, 2026 09:05
@evkotov evkotov requested review from dorloff and mvafin February 5, 2026 13:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: TF FE OpenVINO TensorFlow FrontEnd

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants