Skip to content

Latest commit

 

History

History
63 lines (48 loc) · 3.1 KB

README.md

File metadata and controls

63 lines (48 loc) · 3.1 KB
Raw

s3-response-time

Python package Lint Code Coverage Code style: black

Python script to record the time it takes to preform various operations on an S3 object storage. Errors are reported in Nagios format. Script tested with Python 3.6 and 3.8.

Tested S3 operations:

  • Create random or specific bucket (optional)
  • Upload random object of a specified size (default 1MB)
  • Compare original file MD5 hash to Etag
  • Download object
  • Compare original file MD5 hash to downloaded file MD5 hash
  • Delete random object
  • Delete random or specific bucket (optional)

Example command:

./s3_response_time.py -c credentials.json

Example configuration file:

{
  "s3_host": "https://server.com",          # Required
  "aws_access_key_id": "abcdefghijklmnop",  # Required
  "aws_secret_access_key": "1234567890ab",  # Required
                                            # Everything below is optional; defaults shown
  "addressing_style": "auto",               # S3 addressing style, options: 'auto', 'path', 'virtual'
  "object_size": "1",                       # Size of the object upload in MB 
  "create_bucket": "True",                  # Create bucket; if False bucket must already exist
  "bucket_name": "",                        # Omit for random bucket name
  "influxdb_enabled": "False",              # True to enable writing to influxb
  "influxdb_url": "http://localhost:8086",  # Influxdb server url
  "influxdb_token": "",                     # Influxdb auth token (Influxdb 2.0)
                                            #   Use "username:password" for Influxdb 1.8
  "influxdb_org": "",                       # Influxdb orgization where the data is saved (Influxdb 2.0)
                                            #   Use "-" for Influxdb 1.8
  "influxdb_bucket": "",                    # Influxdb bucket where the data is saved (Influxdb 2.0)
                                            #   Use "database/retention_policy" for Influxdb 1.8
  "influxdb_host": ""                       # Influxdb tag that will be assigned to the data
}

CentOS install instructions:

yum install epel-release
yum install python36 python36-virtualenv git (CentOS 7)
yum install python3 python3-virtualenv git (CentOS 8)

git clone https://github.com/kevincoakley/s3-response-time.git

virtualenv-3 <virtual-environment>
source <virtual-environment>/bin/activate

cd s3-response-time

pip install -r requirements.txt

python3 ./s3_response_time.py -c <config>.json