Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Task result not passed to nested range loop #1496

Open
typeness opened this issue Jun 27, 2024 · 0 comments
Open

Task result not passed to nested range loop #1496

typeness opened this issue Jun 27, 2024 · 0 comments
Labels

Comments

@typeness
Copy link

/kind bug

What steps did you take and what happened:
[A clear and concise description of what the bug is.]

The following DSL generates Tekton that does not pass task results to the inner range loop pipeline-84a66-for-loop-4 to parameter

from kfp import dsl, components
from kfp.components import load_component_from_text
from kfp_tekton.tekton import Loop

op1_yaml = '''\
name: 'my-in-coop1'
inputs:
- {name: item, type: Integer}
- {name: param, type: Integer}
implementation:
    container:
        image: library/bash:4.4.23
        command: ['sh', '-c']
        args:
        - |
          set -e
          echo op1 "$0" "$1"
        - {inputValue: item}
        - {inputValue: param}
'''




@dsl.pipeline(name='pipeline')
def pipeline():
    cel_run_bash_script = load_component_from_text(r"""
            name: cel-run-bash-script
            inputs: []
            outputs:
            - {name: to_1, type: Integer}
            implementation:
              container:
                image: aipipeline/cel-eval:latest
                command: [cel]
                args: [--apiVersion, custom.tekton.dev/v1alpha1, --kind, VariableStore, --name,
                  1234567-var-store, --lit_0, 'a', --taskSpec,
                  '{}']
                fileOutputs: {}
        """)()
    with Loop.range(start=1, step=1, end=cel_run_bash_script.output):
        with Loop.range(start=1, step=1, end=cel_run_bash_script.output):
            op1_template = components.load_component_from_text(op1_yaml)
            op1 = op1_template(1, 2)




if __name__ == '__main__':
  from kfp_tekton.compiler import TektonCompiler as Compiler
  from kfp_tekton.compiler.pipeline_utils import TektonPipelineConf
  tekton_pipeline_conf = TektonPipelineConf()
  tekton_pipeline_conf.set_tekton_inline_spec(True)
  tekton_pipeline_conf.set_resource_in_separate_yaml(True)
  Compiler().compile(pipeline, __file__.replace('.py', '.yaml'), tekton_pipeline_conf=tekton_pipeline_conf)
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
  name: pipeline
  annotations:
    tekton.dev/output_artifacts: '{}'
    tekton.dev/input_artifacts: '{}'
    tekton.dev/artifact_bucket: mlpipeline
    tekton.dev/artifact_endpoint: minio-service.kubeflow:9000
    tekton.dev/artifact_endpoint_scheme: http://
    tekton.dev/artifact_items: '{"my-in-coop1": []}'
    sidecar.istio.io/inject: "false"
    tekton.dev/template: ''
    pipelines.kubeflow.org/big_data_passing_format: $(workspaces.$TASK_NAME.path)/artifacts/$ORIG_PR_NAME/$TASKRUN_NAME/$TASK_PARAM_NAME
    pipelines.kubeflow.org/pipeline_spec: '{"name": "pipeline"}'
  labels:
    pipelines.kubeflow.org/pipelinename: ''
    pipelines.kubeflow.org/generation: ''
spec:
  pipelineSpec:
    tasks:
    - name: cel-run-bash-script
      params:
      - name: lit_0
        value: a
      taskSpec:
        apiVersion: custom.tekton.dev/v1alpha1
        kind: VariableStore
        spec: {}
        metadata:
          labels:
            pipelines.kubeflow.org/cache_enabled: "true"
          annotations:
            pipelines.kubeflow.org/component_spec_digest: '{"name": "cel-run-bash-script",
              "outputs": [{"name": "to_1", "type": "Integer"}], "version": "cel-run-bash-script@sha256=973232bc30856bbd7dc5fc280eb165e2f4ceface92c22e762f9ebd7b860e5be2"}'
    - name: pipeline-84a66-for-loop-2
      params:
      - name: cel-run-bash-script-to-1
        value: $(tasks.cel-run-bash-script.results.to-1)
      - name: from
        value: '1'
      - name: step
        value: '1'
      - name: to
        value: $(tasks.cel-run-bash-script.results.to-1)
      taskSpec:
        apiVersion: custom.tekton.dev/v1alpha1
        kind: PipelineLoop
        spec:
          pipelineSpec:
            params:
            - name: cel-run-bash-script-to-1
              type: string
            - name: loop-item-param-1
              type: string
            tasks:
            - name: pipeline-84a66-for-loop-4
              params:
              - name: from
                value: '1'
              - name: step
                value: '1'
              taskSpec:
                apiVersion: custom.tekton.dev/v1alpha1
                kind: PipelineLoop
                spec:
                  pipelineSpec:
                    params:
                    - name: loop-item-param-3
                      type: string
                    tasks:
                    - name: my-in-coop1
                      taskSpec:
                        steps:
                        - name: main
                          args:
                          - |
                            set -e
                            echo op1 "$0" "$1"
                          - '1'
                          - '2'
                          command:
                          - sh
                          - -c
                          image: library/bash:4.4.23
                        metadata:
                          labels:
                            pipelines.kubeflow.org/cache_enabled: "true"
                          annotations:
                            pipelines.kubeflow.org/component_spec_digest: '{"name":
                              "my-in-coop1", "outputs": [], "version": "my-in-coop1@sha256=8ccab3a28a39a406554d964865f2ccb0aed854a43b6de827f613eff2bccd6f8f"}'
                  iterateNumeric: loop-item-param-3
                metadata:
                  labels:
                    pipelines.kubeflow.org/cache_enabled: "true"
          iterateNumeric: loop-item-param-1
        metadata:
          labels:
            pipelines.kubeflow.org/cache_enabled: "true"

This is a follow-up to the issue #1350. Passing task result output to nested loop was fixed for list-based loops but it still does not work for range loops

What did you expect to happen:

Additional information:
[Miscellaneous information that will assist in solving the issue.]

Environment:

  • Python Version (use python --version):
  • SDK Version:
  • Tekton Version (use tkn version):
  • Kubernetes Version (use kubectl version):
  • OS (e.g. from /etc/os-release):
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

1 participant