-
Notifications
You must be signed in to change notification settings - Fork 0
/
Copy pathREADME.txt
88 lines (62 loc) · 4.14 KB
/
README.txt
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
ceph-influxDB-metricsCollector By Ignacy Debicki
This is a script to collect data about ceph and send it to influxDB >= v0.9.x
Recommended python version of >=2.7
Has been tested and works correctly on verison 2.6.6, however, optimisations in the json module are not implemented, so parsing json files takes significantly longer. If using version 2.6.x, it is highly recommended to also install simplejson.
To run it has the following dependencies:
-[influxdb-python](https://github.com/influxdb/influxdb-python)
|
-->[Requests](http://docs.python-requests.org/)
Optional libraries for 2.6.x to speed up JSON processing:
-[simplejson](https://github.com/simplejson/simplejson)
Also please ensure python has these libraries:
-os
-imp
-ConfigParser
-inspect
-logging
-sys
-traceback
-json
-collections
-functools
-gzip
-cStringIO
-time
-sys
-subprocess
-argparse (if python version >= 2.7.x)
-optparse (if python version <= 2.6.x)
Furthermore, you will require a ceph keychain on the machine this script will be running on.
Installation & configuration:
1.Download this repository wherever you want to have it.
2.Configure the default settings you would like to use if reading a config file fails using the single `default.conf` config file
3.Copy `default.conf` to `/etc/ceph-influxdb-metricsCollector.conf`, and configure your normal configuration.
4.Make sure the user the scripts are going to be ran by can read and write in the desired log location and the plugins directory in the package.
5.Schedual the loader.py to run at your desired interval by creating a CRON job. For testing, you can just run plugins/./runLoop.py, which will run the scripts at the start of every minute. For one time runs, you can use plugins/./loader.py
Defining alternative config file:
You can define an alternative config file by using `plugins/./loader.py -c path/to/config` or `plugins/./runLoop.py -c path/to/config`
Option `--config path/to/config` can also be used.
Defining different intervals for runLoop.py:
You can define a different minute interval by using `plugins/./runLoop.py -i intervalInMinutes`
Option `—interval` may also be used
Hotswapping plugins:
You can switch plugins whilst the program is running by turning them off or on in the configuration file.
If altering or changing the code of a plugin please remember to delete its corresponding .pyc file.
Documentation:
The documentation folder contains some useful information about the way items are tagged, and a spreadsheet showing the sources of all metrics with a small calculator which estimates the number of points that are gathered with each run.
Creating plugins:
All plugin classes must inherit from base.Base and implement the function gather_metrics(), which will return an array of all the measurements
To log events in your plugin, use 'self.logger.' followed by the logging level this should show up under as defined by the logging library
e.g. 'self.logger.info('Gathering metrics')'
Please see example.py for an example plugin structure.
To call commands use the function self.execute_command(isJson,*args) as the results of these operations are memoized, so multiple plugins calling and parsing the same command is not as resource intensive.
e.g. self.execute_command(True,'ceph','osd','tree','--format','json')
if isJson = True, the command will return a python dictionary from the parsed JSON.
If parsing a JSON which is not retrived from a command, please use self.readJson(jsonFile), as the result will be memoized.
To encode a measurement into a point, use self.create_measurement(tags,fields)
The built in methods, of escaping data can slow the system significantly whe nprocessing a large amount of points (~1 second extra per 100,000 points).
Due to this, they are commented out in influxLineProtocol.py, however can be commented back in if you require this functionality.
Furthermore, please make sure your passed in values for fields can be formatted as floats.
N.B. when testing plugins and the source code is changed, but the change is not reflected when running it, please delete the script's corresponding .pyc file.
To enable the plugin, just add into the config file under plugins:
pluginName=[ceph-cluster]