Script annotations artifacts

[dejamiko]

Pull Request Checklist

• Fixes Support annotations for input artifacts for script #717
• Tests added
• Documentation/examples added
• Good commit messages and/or PR title

Description of PR
In Hera we are able to specify Artifacts in inputs:

@script(inputs=Artifact(name="my-artifact", path="/tmp/file"))
def read_artifact():
    with open("/tmp/file") as a_file:
        print(a_file.read())

But by using annotations we can avoid repeating the path of the file, and access the artifact as if it's a regular Python argument of the given type:

@script()
def read_artifact(an_artifact: Annotated[Path, Artifact(name="my-artifact", path="/tmp/file")]):
    print(an_artifact.read_text())

The path duplication is no longer necessary since an_artifact is a Path object.

The fields allowed in the Artifact annotations are: name, path, and loader.
For Artifacts, we allow three types of loaders. Using ArtifactLoader, we have file, json,
and None.
With no loader, the path attribute of Artifact is extracted and can be subsequently used in the function body by referring to the function parameter. This can be seen above.
When the loader is set to file, the function parameter will be the contents of the file
stored at path.

@script()
def read_artifact(
    an_artifact: Annotated[str, Artifact(name="my-artifact", path=ARTIFACT_PATH, loader=ArtifactLoader.file)]
) -> str:
    return an_artifact

This loads the contents of the file at ARTIFACT_PATH to the argument an_artifact and subsequently can be used as a string inside the function.

When the loader is set to json, the contents of the file at path are read and parsed to json.

class MyArtifact(BaseModel):
    a = "a"
    b = "b"


@script()
def read_artifact(
    an_artifact: Annotated[MyArtifact, Artifact(name="my-artifact", path=ARTIFACT_PATH, loader=ArtifactLoader.json)]
) -> str:
    return an_artifact.a + an_artifact.b

Here, we have a json representation of MyArtifact stored at ARTIFACT_PATH. We can load it with ArtifactLoader.json and then use an_artifact as an instance of MyArtifact inside the function.

[elliotgunton]

We'll need to merge #708 first, and then you can rebase this branch to still merge to main. atm this PR includes all the #708 changes

[codecov]

Codecov Report

Merging #718 (dea7696) into main (d835fcf) will increase coverage by 0.2%.
Report is 1 commits behind head on main.
The diff coverage is 88.5%.

@@           Coverage Diff           @@
##            main    #718     +/-   ##
=======================================
+ Coverage   77.7%   77.9%   +0.2%     
=======================================
  Files         45      45             
  Lines       3388    3472     +84     
  Branches     649     676     +27     
=======================================
+ Hits        2633    2707     +74     
- Misses       574     578      +4     
- Partials     181     187      +6     

Files Changed	Coverage Δ
src/hera/workflows/__init__.py	100.0% <ø> (ø)
src/hera/workflows/container_set.py	92.0% <ø> (ø)
src/hera/workflows/cron_workflow.py	90.0% <ø> (ø)
src/hera/workflows/dag.py	81.8% <ø> (ø)
src/hera/workflows/task.py	80.1% <ø> (ø)
src/hera/workflows/workflow.py	79.6% <ø> (ø)
src/hera/workflows/workflow_template.py	97.1% <ø> (ø)
src/hera/workflows/script.py	86.8% <81.6%> (-2.7%)	⬇️
src/hera/workflows/runner.py	74.4% <88.2%> (+0.3%)	⬆️
src/hera/workflows/artifact.py	97.7% <97.2%> (-0.3%)	⬇️
... and 1 more

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

[flaviuvadan]

Will this return work by default if someone does not use the experimental script runner?

[dejamiko]

It probably won't; I didn't really make sure the examples make sense when ran on Argo, I focused on generating the same yaml as the old version

[elliotgunton]

If someone's not using the runner then I think this would return an empty string? Not using the runner with Artifact annotations seems to be the wrong thing to be doing

[elliotgunton]

Wooo! 🚀