collect data from Junos devices using SNMP with telegraf.
store collected data in influxdb
query influxdb database with cli and python to extract data
In this demo we will use SNMPv2 to collect on Junos devices (100.123.1.0, 100.123.1.1, 100.123.1.2) these details:
- Device
hostname
- BGP
bgpPeerState
andbgpPeerRemoteAddr
. With a filter on some neighbors (192.168.*) - Interface
ifName
,ifHCInOctets
,ifHCOutOctets
. With a filter on some interfaces (ge-0/0/0, ge-0/0/1, ge-0/0/2, ge-0/0/3)
telegraf is an open source collector written in GO.
Telegraf collects data and writes them into a database.
It is plugin-driven (it has input plugins, output plugins, ...)
influxdb is an open source time series database written in GO.
here's the SNMP configuration on junos devices
jcluser@vMX-addr-0> show configuration snmp
community public;
Check if Docker is already installed
$ docker --version
If it was not already installed, install it:
$ sudo apt-get update
$ sudo apt-get install \
apt-transport-https \
ca-certificates \
curl \
software-properties-common
$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
$ sudo add-apt-repository \
"deb [arch=amd64] https://download.docker.com/linux/ubuntu \
$(lsb_release -cs) \
stable"
$ sudo apt-get update
$ sudo apt-get install docker-ce
$ sudo docker run hello-world
$ sudo groupadd docker
$ sudo usermod -aG docker $USER
Exit the ssh session and open an new ssh session and run these commands to verify you installed Docker properly:
$ docker run hello-world
Hello from Docker!
This message shows that your installation appears to be working correctly.
To generate this message, Docker took the following steps:
1. The Docker client contacted the Docker daemon.
2. The Docker daemon pulled the "hello-world" image from the Docker Hub.
(amd64)
3. The Docker daemon created a new container from that image which runs the
executable that produces the output you are currently reading.
4. The Docker daemon streamed that output to the Docker client, which sent it
to your terminal.
To try something more ambitious, you can run an Ubuntu container with:
$ docker run -it ubuntu bash
Share images, automate workflows, and more with a free Docker ID:
https://hub.docker.com/
For more examples and ideas, visit:
https://docs.docker.com/engine/userguide/
$ docker --version
Docker version 18.03.1-ce, build 9ee9f40
pull docker images
$ docker pull influxdb
Verify
$ docker images influxdb
Instanciate an influxdb container
$ docker run -d --name influxdb -p 8083:8083 -p 8086:8086 influxdb
Verify
$ docker ps | grep influxdb
for troubleshooting purposes you can run this command
$ docker logs influxdb
start a shell session in the influxdb container
$ docker exec -it influxdb bash
run this command to read the influxdb configuration file
# more /etc/influxdb/influxdb.conf
create a user and a database
# influx
Connected to http://localhost:8086 version 1.7.0
InfluxDB shell version: 1.7.0
Enter an InfluxQL query
> CREATE DATABASE juniper
> show databases
name: databases
name
----
_internal
juniper
> CREATE USER "juniper" WITH PASSWORD 'juniper'
> show users
user admin
---- -----
juniper false
> exit
#
exit the influxdb container
# exit
get ip address used by containers
$ ifconfig docker0
pull docker images
$ docker pull telegraf
Verify
$ docker images telegraf
create a telegraf configuration file (use this file)
it will use SNMP input plugin and influxdb output plugin
SNMP will be used to collect interfaces details and BGP details.
Influxdb is database to store the data collected
$ vi telegraf.conf
instanciate a telegraf container
$ docker run --name telegraf -d -v $PWD/telegraf.conf:/etc/telegraf/telegraf.conf:ro telegraf
verify
$ docker ps | grep telegraf
for troubleshooting purposes you can run this command
$ docker logs telegraf
start a shell session in the telegraf container
$ docker exec -it telegraf bash
verify the telegraf configuration file
# more /etc/telegraf/telegraf.conf
exit the telegraf container
# exit
start a shell session in the influxdb container
$ docker exec -it influxdb bash
query the database
# influx
Connected to http://localhost:8086 version 1.7.0
InfluxDB shell version: 1.7.0
Enter an InfluxQL query
> show databases
name: databases
name
----
_internal
juniper
> use juniper
Using database juniper
> show measurements
name: measurements
name
----
vMX_BGP
vMX_interfaces
>
> select * from "vMX_interfaces" order by desc limit 12
name: vMX_interfaces
time agent_host host hostname ifHCInOctets ifHCOutOctets ifName
---- ---------- ---- -------- ------------ ------------- ------
1544801695000000000 100.123.1.2 803d485462fa vMX-addr-2 653842 899652 ge-0/0/0
1544801695000000000 100.123.1.2 803d485462fa vMX-addr-2 654685 900070 ge-0/0/3
1544801695000000000 100.123.1.2 803d485462fa vMX-addr-2 654482 899569 ge-0/0/2
1544801695000000000 100.123.1.2 803d485462fa vMX-addr-2 654763 898987 ge-0/0/1
1544801695000000000 100.123.1.1 803d485462fa vMX-addr-1 760903 1006250 ge-0/0/0
1544801695000000000 100.123.1.1 803d485462fa vMX-addr-1 654603 899057 ge-0/0/3
1544801695000000000 100.123.1.1 803d485462fa vMX-addr-1 761474 1006808 ge-0/0/2
1544801695000000000 100.123.1.1 803d485462fa vMX-addr-1 761948 1006178 ge-0/0/1
1544801695000000000 100.123.1.0 803d485462fa vMX-addr-0 761686 1007986 ge-0/0/3
1544801695000000000 100.123.1.0 803d485462fa vMX-addr-0 654786 900235 ge-0/0/2
1544801695000000000 100.123.1.0 803d485462fa vMX-addr-0 760969 1007085 ge-0/0/1
1544801695000000000 100.123.1.0 803d485462fa vMX-addr-0 761056 1006102 ge-0/0/0
>
> select * from "vMX_BGP" order by desc limit 12
name: vMX_BGP
time agent_host bgpPeerRemoteAddr bgpPeerState host hostname
---- ---------- ----------------- ------------ ---- --------
1544801015000000000 100.123.1.2 192.168.3.7 6 db58b9842e56 vMX-addr-2
1544801015000000000 100.123.1.2 192.168.3.5 6 db58b9842e56 vMX-addr-2
1544801015000000000 100.123.1.2 192.168.3.3 6 db58b9842e56 vMX-addr-2
1544801015000000000 100.123.1.2 192.168.3.1 6 db58b9842e56 vMX-addr-2
1544801015000000000 100.123.1.1 192.168.2.7 6 db58b9842e56 vMX-addr-1
1544801015000000000 100.123.1.1 192.168.2.5 6 db58b9842e56 vMX-addr-1
1544801015000000000 100.123.1.1 192.168.2.3 6 db58b9842e56 vMX-addr-1
1544801015000000000 100.123.1.1 192.168.2.1 6 db58b9842e56 vMX-addr-1
1544801015000000000 100.123.1.0 192.168.1.7 6 db58b9842e56 vMX-addr-0
1544801015000000000 100.123.1.0 192.168.1.5 6 db58b9842e56 vMX-addr-0
1544801015000000000 100.123.1.0 192.168.1.3 6 db58b9842e56 vMX-addr-0
1544801015000000000 100.123.1.0 192.168.1.1 6 db58b9842e56 vMX-addr-0
>
exit
> exit
#
exit the influxdb container
# exit
install the influxdb python library. This python library is a client for interacting with InfluxDB.
$ pip install influxdb
Verify
$ pip list | grep influx
you can now interact with InfluxDB using Python
run this command to start a python interactive session
$ python
connect to InfluxDB using Python
>>> from influxdb import InfluxDBClient
>>> influx_client = InfluxDBClient('localhost',8086)
list the databases
>>> influx_client.query('show databases')
ResultSet({'(u'databases', None)': [{u'name': u'_internal'}, {u'name': u'juniper'}]})
list measurements for a database
>>> influx_client.query('show measurements', database='juniper')
ResultSet({'(u'measurements', None)': [{u'name': u'vMX_BGP'}, {u'name': u'vMX_interfaces'}]})
query data from a particular measurement and database
>>> gp = influx_client.query('select * from "vMX_interfaces" order by desc limit 4', database='juniper').get_points()
>>> for item in gp:
... print item['hostname']
... print item['ifName']
... print item['ifHCInOctets']
... print "*"*10
...
vMX-addr-2
ge-0/0/0
656258
**********
vMX-addr-2
ge-0/0/3
657120
**********
vMX-addr-2
ge-0/0/2
656977
**********
vMX-addr-2
ge-0/0/1
657238
**********
>>>