Skip to content

Python Support

Jump to bottom Edit New page
Dan Debrunner edited this page Jan 15, 2016 · 7 revisions

Python Support

Mechanisms to support execution of Python code within IBM Streams.

Python Functions as SPL Operators

Initial experimental version has been released: v1.2.2_experimental

Documentation is here: http://ibmstreams.github.io/streamsx.topology/experimental/python/doc/spldoc/html/tk$com.ibm.streamsx.topology/ns$com.ibm.streamsx.topology.python.html

Looking for feedback on if it's useful or what could be added, please provide feedback by creating an issue: https://github.com/IBMStreams/streamsx.topology/issues/new

Possible improvements:

  • Source operator - repeatably calls the Python function
  • Access to operator api features, such as custom metrics
  • Parameters for operators
  • JSON support to tie in with streamsx.json and streamsx.topology JSON support (rstring jsonString).
  • Checkpoint support
  • Python 2.6+ support

Python Topology API

Allows an application to be built using Python in a similar approach to the Java Application API, but with Python so that streams are created and sinked using Python functions and tuples are Python objects.

Initial experimental version has been released: v1.2.0_experimental

Possible improvements:

  • Stateful operators using Python classes (+)
  • Define module policy/importing across PEs.
  • Intermediate tranformations such as filter & transform
  • Non-functional graph virtual operators (e.g. UDP, isolate, low latency regions)
  • Sample applications
  • Getting started & other documentation
  • Connectors (+)
  • Integration with SPL:
    • SPL tuples as Python tuples similar to Java API (subscribe to an SPL Stream & process data using Python)
    • Add SPL operators from Python topology (allows to re-use any toolkit asset)
  • Publish/Subscribe
    • Python tuples
    • Common types (JSON, String, Binary) supported by streamsx.topology
    • SPL tuples.
  • Fanin (out done?)
  • Windowing (using C++ SPL window library, storing Python references in tuple list)
  • Aggregates
  • Joins
  • Split (in Python)
  • Job submission (e.g. job name etc.)
  • Host tag/PE placement
  • Publish/Subscribe of Python tuples (how to deserialize unknown types?)
  • lamda expressions
  • Support building topologies in the Python interpreter
  • Allow testing of Python applications similar to Java tester
Add a custom footer
Add a custom sidebar
Clone this wiki locally