Skip to content

Commit

Permalink
Fix links and formatting for types submodule
Browse files Browse the repository at this point in the history
  • Loading branch information
smokestacklightnin committed Sep 14, 2024
1 parent 8f3bc05 commit 87ba7e8
Show file tree
Hide file tree
Showing 3 changed files with 38 additions and 34 deletions.
33 changes: 19 additions & 14 deletions tfx/types/channel.py
Original file line number Diff line number Diff line change
Expand Up @@ -96,21 +96,23 @@ class BaseChannel(abc.ABC, Generic[_AT]):
Component takes artifact inputs distinguished by each "input key". For
example:
trainer = Trainer(
examples=example_gen.outputs['examples'])
^^^^^^^^
input key
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
channel
``` python
trainer = Trainer(
examples=example_gen.outputs['examples'],
) # ^^^^^^^^
# input key
# ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
# channel
```
Here "examples" is the input key of the `Examples` artifact type.
`#!python example_gen.outputs['examples']` is a channel. Typically a single channel
refers to a *list of `Artifact` of a homogeneous type*. Since channel is a
`#!python example_gen.outputs["examples"]` is a channel. Typically a single channel
refers to a *list of [`Artifact`][tfx.v1.dsl.Artifact] of a homogeneous type*. Since channel is a
declarative abstraction it is not strictly bound to the actual artifact, but
is more of an *input selector*.
The most commonly used channel type is an `OutputChannel` (in the form of
`component.outputs["key"]`, which selects the artifact produced by the
`#!python component.outputs["key"]`, which selects the artifact produced by the
component in the same pipeline run (in synchronous execution mode; more
information on OutputChannel docstring), and is typically a single artifact.
Expand Down Expand Up @@ -732,7 +734,7 @@ def __init__(
"""Initialization of ExternalPipelineChannel.
Args:
artifact_type: Subclass of Artifact for this channel.
artifact_type: Subclass of [Artifact][tfx.v1.dsl.Artifact] for this channel.
owner: Owner of the pipeline.
pipeline_name: Name of the pipeline the artifacts belong to.
producer_component_id: Id of the component produces the artifacts.
Expand Down Expand Up @@ -780,11 +782,14 @@ class ChannelWrappedPlaceholder(artifact_placeholder.ArtifactPlaceholder):
yet reference its name/key wrt. the downstream component in which it is used.
So a ChannelWrappedPlaceholder simply remembers the original Channel instance
that was used. The Placeholder expression tree built from this wrapper is then
passed to the component that uses it, and encode_placeholder_with_channels()
passed to the component that uses it, and `encode_placeholder_with_channels()`
is used to inject the key only later, when encoding the Placeholder.
For instance, this allows making Predicates using syntax like:
channel.future().value > 5
``` python
channel.future().value > 5
```
"""

def __init__(
Expand All @@ -803,8 +808,8 @@ def set_key(self, key: Optional[str]):
setter technically violates this guarantee, but we control the effects of it
by _only_ calling the setter right before an `encode()` operation on this
placeholder or a larger placeholder that contains it, and then calling
set_key(None) right after. encode_placeholder_with_channels() demonstrates
how to do this correctly and should be the preferred way to call set_key().
`#!python set_key(None)` right after. `#!python encode_placeholder_with_channels()` demonstrates
how to do this correctly and should be the preferred way to call `#!python set_key()`.
Args:
key: The new key for the channel.
Expand Down
10 changes: 5 additions & 5 deletions tfx/types/standard_artifacts.py
Original file line number Diff line number Diff line change
Expand Up @@ -84,13 +84,13 @@ class Examples(_TfxArtifact):
The file and payload format must be specified as optional custom properties
if not using default formats.
Please see
https://www.tensorflow.org/tfx/guide/examplegen#span_version_and_split to
[https://www.tensorflow.org/tfx/guide/examplegen#span_version_and_split](https://www.tensorflow.org/tfx/guide/examplegen#span_version_and_split) to
understand about span, version and splits.
* Properties:
- `span`: Integer to distinguish group of Examples.
- `version`: Integer to represent updated data.
- `splits`: A list of split names. For example, ["train", "test"].
- `splits`: A list of split names. For example, `#!python ["train", "test"]`.
* File structure:
- `{uri}/`
Expand All @@ -101,10 +101,10 @@ class Examples(_TfxArtifact):
* Commonly used custom properties of the Examples artifact:
- `file_format`: a string that represents the file format. See
tfx/components/util/tfxio_utils.py:make_tfxio for
[tfx/components/util/tfxio_utils.py](https://github.com/tensorflow/tfx/blob/v1.15.1/tfx/components/util/tfxio_utils.py):make_tfxio for
available values.
- `payload_format`: int (enum) value of the data payload format.
See tfx/proto/example_gen.proto:PayloadFormat for available formats.
See [tfx/proto/example_gen.proto](https://github.com/tensorflow/tfx/blob/v1.15.1/tfx/proto/example_gen.proto):PayloadFormat for available formats.
"""
TYPE_NAME = "Examples"
TYPE_ANNOTATION = Dataset
Expand Down Expand Up @@ -299,7 +299,7 @@ class Schema(_TfxArtifact):
Schema artifact is used to store the
schema of the data. The schema is a proto that describes the data, including
the type of each feature, the range of values for each feature, and other
properties. The schema is usually generated by the SchemaGen component, which
properties. The schema is usually generated by the [SchemaGen][tfx.v1.components.SchemaGen] component, which
uses the statistics of the data to infer the schema. The schema can be used by
other components in the pipeline to validate the data and to generate models.
Expand Down
29 changes: 14 additions & 15 deletions tfx/types/value_artifact.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,30 +106,29 @@ def encode(self, value) -> Any:
def annotate_as(cls, type_annotation: Optional[Type[SystemArtifact]] = None):
"""Annotate the value artifact type with a system artifact class.
Example usage:
!!! example "Example usage"
```python
from tfx import v1 as tfx
OutputArtifact = tfx.dsl.components.OutputArtifact
String = tfx.types.standard_artifacts.String
Model = tfx.dsl.standard_annotations.Model
```python
from tfx import v1 as tfx
@tfx.dsl.components.component
def MyTrainer(
model: OutputArtifact[String.annotate_as(Model)]
):
...
```
OutputArtifact = tfx.dsl.components.OutputArtifact
String = tfx.types.standard_artifacts.String
Model = tfx.dsl.standard_annotations.Model
@tfx.dsl.components.component
def MyTrainer(model: OutputArtifact[String.annotate_as(Model)]): ...
```
Args:
type_annotation: the standard annotations used to annotate the value
artifact type. The possible values are in
`tfx.v1.dsl.standard_annotations`.
Returns:
A subclass of the method caller class (e.g., standard_artifacts.String,
standard_artifacts.Float) with TYPE_ANNOTATION attribute set to be
`type_annotation`; returns the original class if`type_annotation` is None.
A subclass of the method caller class (e.g., [`standard_artifacts.String`][tfx.v1.types.standard_artifacts.String],
[`standard_artifacts.Float`][tfx.v1.types.standard_artifacts.Float]) with TYPE_ANNOTATION attribute set to be
`type_annotation`; returns the original class if`type_annotation` is None.
"""
if not type_annotation:
return cls
Expand Down

0 comments on commit 87ba7e8

Please sign in to comment.