-
Notifications
You must be signed in to change notification settings - Fork 43
Installing Python and com.ibm.streamsx.topology Python alpha
The following steps worked to install Python 3 and the com.ibm.streamsx.topology Python alpha on a clean IBM Streams 4.1 Quickstart Virtual Machine.
Download Python 3.5.1 from: https://www.python.org/downloads/
Click 3.5.1 button and save file
In a terminal window untar the downloaded Python archive.
$ mkdir tmp_python3
$ cd tmp_python3
$ ls ~/Downloads
Python-3.5.1.tar.xz
$ tar xf ~/Downloads/Python-3.5.1.tar.xz
CPython installs pip but requires SSL support which may be missing in your environment.
The packages openssl and openssl-devel (might be known as libssl-dev) need to be installed.
su
# yum install openssl
# yum install openssl-devel
See: http://stackoverflow.com/questions/22592686/compiling-python-3-4-is-not-copying-pip
If you are working on a Redhat machine, and do not have access to the openssl or openssl-devel packages, this guide can walk you through adding the CentOS yum repositories.
Now we are going to build Python from its source, the Python 3 documentation is at: https://docs.python.org/3/using/unix.html#building-python
However, we need to add flags ``--enable-shared, --prefix=/usr` and `--with-ensurepip=install` to `configure`.
Also a symbolic link must be created after the make install.
$ # Become root to install Python 3 at the system location
$ su
$ cd Python-3.5.1
$ ./configure --enable-shared --prefix=/usr --with-ensurepip=install
$ make install
$ ln -s /usr/lib/libpython3.5m.so.1.0 /usr/lib64/libpython3.5m.so.1.0
$ # stop being root
$ exit
This is inspired by the information here, but the link details are different: http://serverfault.com/questions/71601/compile-python-3-1-1-with-enable-shared
At this point the command python3 should be working:
$ python3
Python 3.5.1 (default, Apr 18 2016, 10:15:49)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-11)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
Download the release from: https://github.com/IBMStreams/streamsx.topology/releases/tag/v1.2.5_alpha
Specifically the tar file: https://github.com/IBMStreams/streamsx.topology/releases/download/v1.2.5_alpha/streamsx.topology-v1.2-20160414-1513.tgz
I install my third-party toolkits under $HOME/toolkits, but almost any location will work. I do not recommend installing a non-product toolkit under the IBM Streams product install ($STREAMS_INSTALL).
$ mkdir ~/toolkits
$ cd toolkits
$ tar xzf ~/Downloads/streamsx.topology-v1.2-20160414-1513.tgz
export PYTHONPATH=$HOME/toolkits/com.ibm.streamsx.topology/opt/python/packages:$PYTHONPATH
If your PYTHONPATH is currently not set you do not need to add it to the end, e.g.
export PYTHONPATH=$HOME/toolkits/com.ibm.streamsx.topology/opt/python/packages
Change directory to the top-level samples for the Python Application API.
cd $HOME/toolkits/samples/python/topology
For example run the Hello World sample in hello_world.py:
$ cd simple
$ python3 hello_world.py
Apr 18, 2016 10:35:21 AM com.ibm.streamsx.topology.internal.streams.InvokeSc getToolkitPath
INFO: ToolkitPath:/home/streamsadmin/toolkits/com.ibm.streamsx.topology:/opt/ibm/InfoSphere_Streams/4.1.0.0/toolkits:/opt/ibm/InfoSphere_Streams/4.1.0.0/toolkits
Apr 18, 2016 10:35:21 AM com.ibm.streamsx.topology.internal.streams.InvokeSc invoke
INFO: Invoking SPL compiler (sc) for main composite: hello_world::hello_world
Apr 18, 2016 10:35:21 AM com.ibm.streamsx.topology.internal.streams.InvokeSc invoke
INFO: /opt/ibm/InfoSphere_Streams/4.1.0.0/bin/sc --optimized-code-generation --num-make-threads=4 --standalone -M hello_world::hello_world -t /home/streamsadmin/toolkits/com.ibm.streamsx.topology:/opt/ibm/InfoSphere_Streams/4.1.0.0/toolkits:/opt/ibm/InfoSphere_Streams/4.1.0.0/toolkits
Apr 18, 2016 10:35:21 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
SEVERE: CDISP0120W WARNING: The following path is already in the SPL path list: /opt/ibm/InfoSphere_Streams/4.1.0.0/toolkits
Apr 18, 2016 10:35:22 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: Creating the types.
Apr 18, 2016 10:35:23 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: Creating the functions.
Apr 18, 2016 10:35:23 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: Creating the operators.
Apr 18, 2016 10:35:23 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: Creating the processing elements.
Apr 18, 2016 10:35:23 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: The standalone application is being created.
Apr 18, 2016 10:35:23 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: Creating the application model.
Apr 18, 2016 10:35:23 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: Building the binaries.
Apr 18, 2016 10:35:23 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: [CXX-type] tuple<blob __spl_po>
Apr 18, 2016 10:35:23 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: [CXX-operator] source_tuples_0
Apr 18, 2016 10:35:23 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: [CXX-operator] print_flush_1
Apr 18, 2016 10:35:23 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: [CXX-pe] pe hello_world.hello_world-a
Apr 18, 2016 10:35:24 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: [CXX-standalone] standalone
Apr 18, 2016 10:35:24 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: [LD-standalone] standalone
Apr 18, 2016 10:35:25 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: [LN-standalone] standalone
Apr 18, 2016 10:35:27 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: [LD-pe] pe hello_world.hello_world-a
Apr 18, 2016 10:35:27 AM com.ibm.streamsx.topology.internal.process.ProcessOutputToLogger run
INFO: [Bundle] hello_world.hello_world.sab
Apr 18, 2016 10:35:30 AM com.ibm.streamsx.topology.internal.streams.InvokeSc invoke
INFO: SPL compiler complete: return code=0
Apr 18, 2016 10:35:30 AM com.ibm.streamsx.topology.internal.context.BundleStreamsContext doSPLCompile
INFO: Streams Application Bundle produced: hello_world.hello_world.sab
Apr 18, 2016 10:35:30 AM com.ibm.streamsx.topology.internal.streams.InvokeStandalone invoke
INFO: Invoking standalone application
Apr 18, 2016 10:35:30 AM com.ibm.streamsx.topology.internal.streams.InvokeStandalone invoke
INFO: /opt/ibm/InfoSphere_Streams/4.1.0.0/java/jre/bin/java -jar /home/streamsadmin/toolkits/samples/python/topology/simple/hello_world.hello_world.sab
Hello
World!
Apr 18, 2016 10:35:31 AM com.ibm.streamsx.topology.internal.streams.InvokeStandalone$ProcessFuture get
INFO: Standalone application completed: return code=0
Run the publish subscribe sample, note any number of instances of the applications can be submitted as jobs:
Build and submit a single copy of the publish application and the subscribe application. (Note: Ensure your Streams instance is running)
cd $HOME/toolkits/samples/python/topology/pubsub
python3 publish.py
python3 subscribe.py
Note that the bundles for the application are left in the current directory can be submitted directly to the instance, for example:
# Submit another copy of the subscribe application
streamtool submitjob SubscribeSimple.SubscribeSimple.sab