Fix subflow type

[jgyasu]

The develop branch was recently removed, and the main branch is now used for both development and releases (possibly via a rename from develop to main). The develop branch previously had an issue with the type of subflows, which caused Pydantic errors; this issue was fixed on main. However, that fix is no longer present on main. This PR restores the fix.

Summary by Sourcery

Bug Fixes:

• Correct subflow field type on Flow to use a dedicated Subflow structure instead of self-referential Flow instances.

Summary by CodeRabbit

• New Features

  • Introduced explicit subflow type structure to improve representation and handling of nested flows within compositions.

• Refactor

  • Updated flow schema with explicit subflow definitions instead of self-referential types, providing clearer structure and enhanced type safety for improved maintainability across flow composition operations.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

[sourcery-ai]

Reviewer's guide (collapsed on small PRs)

Reviewer's Guide

Refactors the Flow.subflows field to use a dedicated Subflow TypedDict type instead of a recursive Flow model, restoring the previously correct typing that avoids Pydantic validation issues.

Class diagram for updated Flow and Subflow types

classDiagram
    class BaseModel
    class TypedDict

    class Flow {
        +str identifier
        +str title
        +str description
        +str~dict~ metadata
        +str language
        +str dependencies
        +list~Parameter~ parameter
        +list~Subflow~ subflows
        +list~str~ tag
    }

    class Subflow {
        +str identifier
        +Flow flow
    }

    Flow --|> BaseModel
    Subflow --|> TypedDict
    Flow o--> Subflow
    Subflow o--> Flow

Loading

File-Level Changes

Change	Details	Files
Change Flow.subflows from a recursive Flow type to a dedicated Subflow TypedDict type and define the Subflow structure.	Update Flow.subflows annotation from list of Flow (Self) to list of Subflow Introduce a Subflow TypedDict representing a subflow item with identifier and flow fields Specify that each Subflow.flow is a Flow instance and identifier is optional	src/schemas/flows.py

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.

[coderabbitai]

Walkthrough

The change replaces recursive self-referential typing in the Flow schema with an explicit TypedDict-based structure. A new public type Subflow is introduced with identifier and flow fields, while Flow.subflows is updated to reference this new type instead of using Self.

Changes

Cohort / File(s)	Change Summary
Flow schema type refactoring src/schemas/flows.py	Added new public TypedDict Subflow with identifier: str | None and flow: Flow fields. Updated Flow.subflows annotation from list[Self] to list[Subflow]. Replaced Self import with TypedDict.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

• Verify that the new Subflow TypedDict structure aligns with how subflows are used elsewhere in the codebase
• Check for any runtime type validation or serialization logic that may reference the old Self pattern
• Consider whether existing code creating or manipulating subflows needs adjustment

Poem

🐰 From Self to explicit form we bound,
No cycles now, just structure sound,
Subflow emerges, clear and true,
Schema's dance—recursion's adieu! ✨

Pre-merge checks and finishing touches

✅ Passed checks (3 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'Fix subflow type' directly and accurately summarizes the main change: replacing Flow.subflows typing from Self to an explicit Subflow TypedDict structure to resolve Pydantic errors.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing touches

• 📝 Generate docstrings

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[sourcery-ai]

Hey - I've left some high level feedback:

• Since Subflow is referenced in Flow before it is defined, consider either moving the Subflow definition above Flow or changing the annotation to a string literal (list["Subflow"]) to avoid issues with tools/environments that don't enable postponed evaluation of annotations.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- Since `Subflow` is referenced in `Flow` before it is defined, consider either moving the `Subflow` definition above `Flow` or changing the annotation to a string literal (`list["Subflow"]`) to avoid issues with tools/environments that don't enable postponed evaluation of annotations.

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[coderabbitai]

Actionable comments posted: 0

🧹 Nitpick comments (1)

src/schemas/flows.py (1)

34-36: Consider using BaseModel for consistency and validation.

The Subflow TypedDict provides type hints but no runtime validation, unlike the Flow BaseModel. While this lighter approach might be intentional to avoid nested validation overhead, using a Pydantic BaseModel would ensure consistent validation across the schema and catch potential data issues earlier.

🔎 Alternative using BaseModel

-class Subflow(TypedDict):
+class Subflow(BaseModel):
     identifier: str | None
     flow: Flow

Note: Keep this as TypedDict if the performance/simplicity trade-off is intentional.

📜 Review details

Configuration used: defaults

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 8aaf051 and 6457dc6.

📒 Files selected for processing (1)

• src/schemas/flows.py

🧰 Additional context used

🧬 Code graph analysis (1)

src/schemas/flows.py (1)

tests/conftest.py (2)

• flow (79-89)
• Flow (70-75)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

• GitHub Check: Sourcery review

🔇 Additional comments (2)

src/schemas/flows.py (2)

4-4: LGTM! Import change aligns with the structural shift.

The import change from Self to TypedDict correctly supports the new explicit subflow structure.

---

28-28: Verify circular reference handling in Flow.subflows implementation.

The change from list[Self] to list[Subflow] resolves Pydantic's handling of recursive type definitions by using a TypedDict intermediary. Note that Subflow is a TypedDict containing identifier and a flow: Flow field—meaning subflows are validated as dictionaries that contain Flow BaseModel instances, not dictionaries alone.

Verify that:

• Database queries properly construct Subflow dictionaries with valid Flow instances
• Serialization correctly handles the nested Flow objects within the TypedDict structure

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@8aaf051). Learn more about missing BASE report.

Additional details and impacted files

@@           Coverage Diff           @@
##             main     #222   +/-   ##
=======================================
  Coverage        ?   54.44%           
=======================================
  Files           ?       32           
  Lines           ?     1135           
  Branches        ?      100           
=======================================
  Hits            ?      618           
  Misses          ?      516           
  Partials        ?        1           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[PGijsbers]

I believe the typing is correct in main because the structure got updated too. As you can see, the tests no longer pass with the typing reverted.

That said, I made the change because I misremembered the role of flow.identifier. It's poor naming, but it's actually a "nickname" of the subflow within the context of the relationship between flow and subflow only. So we may want to preserve that. I can revert the change altogether.

[jgyasu]

I believe the typing is correct in main because the structure got updated too

Hmm, I haven't looked into the changes, but I updated my main from the upstream and deleted and rebuilt the docker services, the http://localhost:8001/docs endpoint for SwaggerUI on the openml-python-rest-api leads to a type related Pydantic error. I should perhaps create an issue for discussion then?

[PGijsbers]

You're right. It does lead to an issue, perhaps I was loading a cached version on the page when I checked. I'll look into this and fix it or revert the change 👍

[PGijsbers]

Reverted change in #225