Skip to content

TensorFlow Serving API

Jump to bottom
David Salek edited this page Nov 19, 2017 · 14 revisions

This wiki describes how to install the TensorFlow Serving python API that is used to query the server from Raspberry Pi. An SSH tunnel is necessary for this. At the beginning, instructions are given on how to setup such tunnel manually and how to test that TensorFlow Serving is working. The subsequent sections talk about enabling an SSH connection without password, using public-key authentication, and setting up a persistent SSH tunnel using AutoSSH.

Install python API

In order to run the client code of TensorFlow Serving, install the python API.

sudo pip install tensorflow-serving-api

The installation will fail with the error below in case you do not have gcc version 4.9.

c++: error: unrecognized command line option ‘-fstack-protector-strong’

This is the way to get version 4.9 of the gcc compiler:

sudo apt-get install gcc-4.9 g++-4.9
sudo update-alternatives --remove-all gcc
sudo update-alternatives --remove-all g++
sudo update-alternatives --install /usr/bin/gcc gcc /usr/bin/gcc-4.9 1 --slave /usr/bin/g++ g++ /usr/bin/g++-4.9
sudo update-alternatives --config gcc

Query the server

Create the ssh tunnel from Raspberry Pi to the machine on your local network where the server runs.

ssh -nNT -L 9000:localhost:9000 salekd@192.168.2.2

You can check that the tunnel is alive in the following way. On the server side, open the port for listening:

nc -l localhost 9000

Make connection from Raspberry Pi and type something

nc localhost 9000

The text you type here, on Raspberry Pi, will be mirrored on the server side.

Test that TensorFlow Serving works by querying the server from Raspberry Pi:

wget https://github.com/salekd/rpizero_smart_camera/raw/master/camera.JPG
python tensorflow_serving/example/inception_client.py --server=localhost:9000 --image=camera.JPG

SSH with public-key authentication

It is handy to enable the SSH connection without the need to write a password every time. This can be achieved with public-key authentication. The steps below are taken from here: https://www.howtoforge.com/set-up-ssh-with-public-key-authentication-debian-etch

Generate new pair of private and public keys on your Raspberry Pi, copy it to the remote machine and delete the public key.

chmod 700 ~/.ssh
ssh-keygen
scp ~/.ssh/id_rsa.pub salekd@192.168.2.2:~/.ssh/.
rm .ssh/id_rsa.pub

On the remote machine, add the public key to authorized keys.

chmod 700 ~/.ssh
cat .ssh/id_rsa.pub >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Make sure the following settings are in the /etc/ssh/sshd_config file on the remote machine and restart.

PermitRootLogin		no
PasswordAuthentication	no
UsePAM			no

You will be able to SSH to the remote machine from your Raspberry Pi without entering a password now.

Persistent SSH tunnel with AutoSSH

It is also possible to create a permanent SSH tunnel already during boot time. This means that you will not have to execute this command every time by hand:

ssh -nNT -L 9000:localhost:9000 salekd@192.168.2.2

The instructions below are taken from https://www.everythingcli.org/ssh-tunnelling-for-fun-and-profit-autossh/

Install AutoSSH:

sudo apt-get install autossh

Create new file /etc/systemd/system/autossh-tfserving-tunnel.service containing the following lines:

[Unit]
Description=AutoSSH tunnel service
After=network.target

[Service]
User=pi
Type=forking
ExecStart=/usr/bin/autossh -M 0 -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -nNT -L 9000:localhost:9000 salekd@192.168.2.2

[Install]
WantedBy=multi-user.target

Tell systemd that we have added some stuff:

sudo systemctl daemon-reload

Start the service:

sudo systemctl start autossh-tfserving-tunnel.service

Enable the service during boot time:

sudo systemctl enable autossh-tfserving-tunnel.service
Clone this wiki locally