InfluxDB: save blacklisted domains

Sending blacklist entries every hour to InfluxDB involves a combination of:

1. Extracting the blacklist entries from your source.
2. Formatting these entries in a manner compatible with InfluxDB.
3. Sending this data to InfluxDB.
4. Automating the above steps to run every hour.

Below, I'll guide you through these steps.

1. Setup:

First, ensure you have the necessary Python libraries:

pip install influxdb-client

2. Script to Send Blacklist Entries to InfluxDB:

Here's a simple Python script (send_to_influxdb.py) that will send blacklist entries to InfluxDB:

from influxdb_client import InfluxDBClient, Point
from influxdb_client.client.write_api import SYNCHRONOUS
import sqlite3

# SQLite database path
DB_PATH = "path_to_your_blacklist_database.db"

# InfluxDB configurations
INFLUXDB_URL = 'http://localhost:8086'  # Modify as needed
INFLUXDB_TOKEN = 'YOUR_INFLUXDB_TOKEN'
INFLUXDB_ORG = 'YOUR_INFLUXDB_ORGANIZATION'
INFLUXDB_BUCKET = 'YOUR_INFLUXDB_BUCKET'

# Connect to SQLite database and fetch blacklisted domains
def fetch_blacklisted_domains():
    conn = sqlite3.connect(DB_PATH)
    cursor = conn.cursor()
    cursor.execute("SELECT domain FROM blacklist")
    domains = [row[0] for row in cursor.fetchall()]
    conn.close()
    return domains

def send_to_influxdb(domains):
    client = InfluxDBClient(url=INFLUXDB_URL, token=INFLUXDB_TOKEN, org=INFLUXDB_ORG)
    write_api = client.write_api(write_options=SYNCHRONOUS)

    points = []
    for domain in domains:
        point = Point("blacklist").tag("source", "your_source_name").field("domain", domain)
        points.append(point)

    write_api.write(INFLUXDB_BUCKET, INFLUXDB_ORG, points)

if __name__ == '__main__':
    domains = fetch_blacklisted_domains()
    send_to_influxdb(domains)

Replace placeholders (YOUR_INFLUXDB_TOKEN, etc.) with appropriate values.

3. Automate the Process:

To send the blacklist entries every hour, you can schedule the script to run as a cron job (on Linux) or a scheduled task (on Windows).

For Linux (using cron):

Edit the crontab:

crontab -e

Add the following line to schedule the script to run every hour:

0 * * * * /path/to/python3 /path/to/send_to_influxdb.py

Replace /path/to/python3 with the path to your Python interpreter (you can get this with which python3) and /path/to/send_to_influxdb.py with the full path to the script.

For Windows (using Task Scheduler):

1. Open Task Scheduler.
2. Create a new basic task.
3. Set the trigger to repeat every hour.
4. For the action, choose "Start a program" and point it to your Python interpreter and add the path to the script in the "Add arguments" section.

With these steps, your blacklist entries should be sent to InfluxDB every hour. Ensure you handle any potential exceptions or errors, especially when dealing with external systems.