Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix multiple bugs in Hera Input and Output classes #1193

Merged
merged 10 commits into from
Sep 17, 2024

Conversation

alicederyn
Copy link
Collaborator

@alicederyn alicederyn commented Sep 5, 2024

Pull Request Checklist

Description of PR
Currently, the methods that consume annotations in InputMixin and OutputMixin have multiple bugs:

  • ignoring fields with a workflow annotation with no name
  • ignoring fields with a Pydantic annotation and no workflow annotation
  • OutputMixin._get_outputs additionally was:
    • mutating the original annotations without copying them
    • ignoring the model default if a workflow annotation was present but had no default

This PR pulls out a common function to iterate through fields and yield annotations, which:

  • copies the annotations so they cannot be accidentally changed
  • ignores unrecognized annotations (this is a requirement of the Annotated spec, and fixes the issue with Pydantic annotations)
  • defaults the annotation name to the field name if unset

This ensures all functions treat an explicit Parameter and a missing workflow annotation the same way, solving the second issue with OutputMixin._get_outputs.

The current code will silently skip fields with:

 - a Pydantic annotation but no workflow annotation
 - a workflow annotation with no name field

Fix the code to ignore unrecognized annotations (this is a requirement
of the Annotated spec) and to default the model name to the field name
if none is given explicitly.

Signed-off-by: Alice Purcell <alicederyn@gmail.com>
Signed-off-by: Alice Purcell <alicederyn@gmail.com>
Signed-off-by: Alice Purcell <alicederyn@gmail.com>
Signed-off-by: Alice Purcell <alicederyn@gmail.com>
Signed-off-by: Alice Purcell <alicederyn@gmail.com>
OutputMixin._get_outputs was mutating the original annotation instead of
copying it; not defaulting the name to the field name; and not copying
the model default into the annotation.

Signed-off-by: Alice Purcell <alicederyn@gmail.com>
@alicederyn alicederyn added semver:patch A change requiring a patch version bump type:bug A general bug labels Sep 5, 2024
Copy link

codecov bot commented Sep 5, 2024

Codecov Report

Attention: Patch coverage is 97.43590% with 1 line in your changes missing coverage. Please review.

Project coverage is 86.0%. Comparing base (d553546) to head (c6a2606).
Report is 147 commits behind head on main.

Files with missing lines Patch % Lines
src/hera/workflows/io/_io_mixins.py 97.4% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@           Coverage Diff           @@
##            main   #1193     +/-   ##
=======================================
+ Coverage   81.7%   86.0%   +4.3%     
=======================================
  Files         54      60      +6     
  Lines       4208    4038    -170     
  Branches     889     838     -51     
=======================================
+ Hits        3439    3475     +36     
+ Misses       574     391    -183     
+ Partials     195     172     -23     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Multiple methods in _io_mixins resolve the same problem of iterating
through all fields, constructing a workflow annotation based on the
user's annotations, defaulting the name if unset or falling back to a
Parameter if no annotation is found. Factor that logic out into an
iterator, simplifying the various methods and reducing the risk of
discrepancies and bugs in future.

Signed-off-by: Alice Purcell <alicederyn@gmail.com>
Copy link
Collaborator

@elliotgunton elliotgunton left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great to have loads of tests! Saw a couple of places we could do some extra cleanup

src/hera/workflows/io/_io_mixins.py Outdated Show resolved Hide resolved
src/hera/workflows/io/_io_mixins.py Outdated Show resolved Hide resolved
src/hera/workflows/io/_io_mixins.py Outdated Show resolved Hide resolved
@elliotgunton
Copy link
Collaborator

This also fixes #1165, updated the PR description

samj1912 pushed a commit that referenced this pull request Sep 13, 2024
**Pull Request Checklist**
- [x] Fixes #1198 
- [ ] ~Tests added~
- [x] Documentation/examples added
- [x] [Good commit messages](https://cbea.ms/git-commit/) and/or PR
title

**Description of PR**
From issue description:

* [Script basics "explicit"
example](https://hera.readthedocs.io/en/stable/user-guides/script-basics/#script-decorator)
passes `source=echo` in a Task - this is not possible
* [Script
annotations](https://hera.readthedocs.io/en/stable/user-guides/script-annotations/)
is quite confusing to read. Examples are overly complex/wordy, some
being incorrect. We can also show that `name` can be inferred when #1193
is merged
* [expr
guide](https://hera.readthedocs.io/en/stable/user-guides/expr/)'s first
link is dead, also some formatting can be tidied up
* README is duplicated due to mascot image link, but we can fix this.
Also note the [PyPI page](https://pypi.org/project/hera/) has a broken
image

![image](https://github.com/user-attachments/assets/1ac93d04-9f9a-4816-8123-9b8bab7ffc50)

---------

Signed-off-by: Elliot Gunton <elliotgunton@gmail.com>
@elliotgunton elliotgunton self-assigned this Sep 16, 2024
Signed-off-by: Elliot Gunton <elliotgunton@gmail.com>
Signed-off-by: Elliot Gunton <elliotgunton@gmail.com>
@elliotgunton elliotgunton enabled auto-merge (squash) September 17, 2024 10:06
@elliotgunton elliotgunton merged commit 3e58f22 into argoproj-labs:main Sep 17, 2024
20 checks passed
@alicederyn alicederyn deleted the fix.io.mixin.bugs branch September 17, 2024 10:39
@elliotgunton elliotgunton changed the title Fix multiple bugs in IO mixins Fix multiple bugs in Hera Input and Output classes Sep 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
semver:patch A change requiring a patch version bump type:bug A general bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Fields silently ignored in Pydantic I/O types Artifact name should be optional for runner inputs
3 participants