Riots Data Flow Analytics

The aim of this analytics is to detect if there is a change in the device network data flow. This analytics will notice if the device gets overactive or inactive suddenly.

The analytics is based on statistical method called Interquartile range (IQR). Algorithm stores data parameter to fixed size array and runs analytics over that array. Array size must be adjusted accoring to application. If array size is big algorithm uses more memory and analytics speed is a bit slower but detection speed is not affected. In case of small array less resources are used but algorithm can give more false alarms.

Stored parameters are data length in bytes (2 bytes), data time (8 bytes) in 0.1 second accuracy, time difference to previous packet (4 bytes) in 0.1 second accuracy and calculated byterate (4 bytes). Totally one entry takes 18 bytes of memory and if array size has been set to 1000 then analytics in one device will be using 18 kB memory. Analytics parameters can be adjusted in parameters.py file.

The time used to calculate byterate is also adjustable. If byterate time is small and device sends packets with big intervals then byterate is same as packet size. When byterate time is big algorithm allows short time and high datarate databursts.

Analytics triggers overactive alarm when byterate gets too high and inactive alarm is triggered when time difference between data packets gets too high.

Build and run instruction:

Using docker

docker build --pull -t test . && docker run -t 8888 -h 8877 --rm -it test

Directly from command line

python3 src/analytic_server.py -t 8888 -h 8877

Parameters are:

 [-t | --tcp_port] TCP port, used by data clients to connect to server
 [-h | --http_port] HTTP port, used by analytics api to ask for analytics results 

Files in this directory are as below:

"src\parameters.py": Set analytic parameters
"src\analytic_server.py": Server to receive data flow to be analyzed
"src\analytic_device.py": Class for single device
"src\analytic_ui.py": User interface for the analytics
"src\data_flow_service.py": The analytics algorithm
"demo_client\demo.py": Client to produce random data
"demo_client\parameters.py": Set server port for client

License

Released under the MIT License.

Acknowledgements

This software has been developed in the scope of the H2020 project SIFIS-Home with GA n. 952652.