Skip to content

Commit

Permalink
quick fix for SDK instructions (#358)
Browse files Browse the repository at this point in the history
* Quick fix in SDK documentation

Takes some excerpts from Christian's website PR. 

I will be revisiting after release to figure out the overlap with User guide.

* SDK Packages Overview (#359)

Co-authored-by: Christian Kadner <ckadner@us.ibm.com>
  • Loading branch information
animeshsingh and ckadner authored Nov 9, 2020
1 parent e6f2eb4 commit f6e6144
Showing 1 changed file with 32 additions and 16 deletions.
48 changes: 32 additions & 16 deletions sdk/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ backed by Tekton.

<!-- START of ToC generated by running ./tools/mdtoc.sh sdk/README.md -->

- [Extensions to the Kubeflow Pipelines SDK](#extensions-to-the-kubeflow-pipelines-sdk)
- [SDK Packages Overview](#sdk-packages-overview)
- [Project Prerequisites](#project-prerequisites)
- [Installation](#installation)
- [Compiling a Kubeflow Pipelines DSL Script](#compiling-a-kubeflow-pipelines-dsl-script)
Expand All @@ -26,23 +26,31 @@ backed by Tekton.
<!-- END of ToC generated by running ./tools/mdtoc.sh sdk/README.md -->


## Extensions to the Kubeflow Pipelines SDK
## SDK Packages Overview

In addition to the functionality provided by the Kubeflow Pipelines
[SDK](https://www.kubeflow.org/docs/pipelines/sdk/sdk-overview/) the `kfp-tekton`
SDK provides a `TektonCompiler` and a `TektonClient`:
The `kfp-tekton` SDK is an extension to the [Kubeflow Pipelines SDK](/docs/pipelines/sdk/sdk-overview/)
adding the `TektonCompiler` and the `TektonClient`:

`TektonCompiler`:
* `kfp_tekton.compiler` includes classes and methods for compiling pipeline Python DSL into a Tekton PipelineRun YAML spec. The methods in this package
include, but are not limited to, the following:

- `kfp_tekton.compiler.TektonCompiler.compile` compiles Python DSL code into a
YAML file containing a Tekton `PipelineRun` which can be deployed directly to
a Tekton enabled Kubernetes cluster or uploaded to the Kubeflow Pipelines
dashboard with the Tekton backend.

`TektonClient`:
* `kfp_tekton.compiler.TektonCompiler.compile` compiles your Python DSL code
into a single static configuration (in YAML format) that the Kubeflow Pipelines service
can process. The Kubeflow Pipelines service converts the static
configuration into a set of Kubernetes resources for execution.

- `kfp_tekton.TektonClient.create_run_from_pipeline_func` compiles DSL pipeline
function and runs the pipeline on a Kubernetes cluster with KFP and Tekton
* `kfp_tekton.TektonClient` contains the Python client libraries for the [Kubeflow Pipelines API](/docs/pipelines/reference/api/kubeflow-pipeline-api-spec/).
Methods in this package include, but are not limited to, the following:

* `kfp_tekton.TektonClient.upload_pipeline` uploads a local file to create a new pipeline in Kubeflow Pipelines.
* `kfp_tekton.TektonClient.create_experiment` creates a pipeline
[experiment](/docs/pipelines/concepts/experiment/) and returns an
experiment object.
* `kfp_tekton.TektonClient.run_pipeline` runs a pipeline and returns a run object.
* `kfp_tekton.TektonClient.create_run_from_pipeline_func` compiles a pipeline
function and submits it for execution on Kubeflow Pipelines.
* `kfp_tekton.TektonClient.create_run_from_pipeline_package` runs a local
pipeline package on Kubeflow Pipelines.


## Project Prerequisites
Expand Down Expand Up @@ -144,8 +152,16 @@ export DEFAULT_STORAGE_SIZE=2Gi
## Running the Compiled Pipeline on a Tekton Cluster

After compiling the `sdk/python/tests/compiler/testdata/parallel_join.py` DSL script
in the step above, we need to deploy the generated Tekton YAML to our Kubernetes
cluster with `kubectl`. The Tekton server will automatically start a pipeline run.
in the step above, we need to deploy the generated Tekton YAML to Kubeflow Pipeline engine.

You can run the pipeline directly using a pre-compiled file and KFP-Tekton SDK. For more details, please look at the [KFP-Tekton user guide SDK documentation](https://github.com/kubeflow/kfp-tekton/tree/master/guides/kfp-user-guide#2-run-pipelines-using-the-kfp_tektontektonclient-in-python)

```python
experiment = kfp_tekton.TektonClient.create_experiment(name=EXPERIMENT_NAME, namespace=KUBEFLOW_PROFILE_NAME)
run = client.run_pipeline(experiment.id, 'parallal-join-pipeline', 'pipeline.yaml')
```

You can also deploy directly on Tekton cluster with `kubectl`. The Tekton server will automatically start a pipeline run.
We can then follow the logs using the `tkn` CLI.

kubectl apply -f pipeline.yaml
Expand Down

0 comments on commit f6e6144

Please sign in to comment.