Skip to content

Add Support for New Sensors to the MHN

Jump to bottom
Sean Maloney edited this page Oct 1, 2019 · 2 revisions

A Brief how-to for adding support for new honeypots or sensors to MHN. In this howto we provide an overview of the steps involved as well as providing links to a real example of adding support to MHN for Wordpot, a Word Press honeypot that originally had no support for hpfeeds.

Hpfeeds Support

Hpfeeds is a transport protocol use by many honeypots released by the honeypot project. MHN uses it for collecting data from honeypots. Make sure the honeypot/sensor has support for reporting its data through hpfeeds. Add support if necessary; this is usually pretty straight forward. Ideally the sensor reports its data as JSON. Any binary format should work though.

Example use of hpfeeds for publishing data

import hpfeeds, json

hpclient = hpfeeds.new(host, port, ident, secret)
hpclient.publish("channel.events", json.dumps({"src_ip": "4.4.4.4", "dst_ip": "2.1.12.1" }))

Full Example for Wordpot: https://github.com/Pwnlandia/wordpot/pull/1/files

Honeymap Support

Add a function that will take a record from your sensor, normalize it and perform IP Geolocation on the record. This enables the record to be rendered on HoneyMap. The normalized record should be a dictionary with these fields:

'type'
'sensor'
'time'
'latitude'
'longitude'
'source'
'city'
'country'
'countrycode'
# optional
'latitude2'
'longitude2'
'city2'
'country2'
'countrycode2'

Full example for Wordpot: https://github.com/Pwnlandia/hpfeeds/pull/9/files

Also, update MHN's install script for honeymap so it subscribes to the new honeypot's channel. Example: https://github.com/Pwnlandia/mhn/blob/master/scripts/install_honeymap.sh

cat > /opt/hpfeeds/geoloc.json <<EOF
{
    "HOST": "localhost",
    "PORT": 10000,
    "IDENT": "geoloc", 
    "SECRET": "$SECRET",
    "CHANNELS": [
        "dionaea.connections",
        "dionaea.capture",
        "glastopf.events",
        "beeswarm.hive",
        "kippo.sessions",
        "conpot.events",
        "snort.alerts",
        "amun.events",
        "wordpot.events"
    ],
    "GEOLOC_CHAN": "geoloc.events"
}
EOF

Mnemosyne Support

Add support to nmemosyne so the data can be normalized and indexed into MongoDB. This consists of adding a normalizer module and importing this module into normalizer/normalizer.py. This normalizer takes the event as it came from the honeypot, decodes it (e.g. json.loads()) and transforms it into at least a session record with the following structure:

session = {
	'timestamp': submission_timestamp,
	'source_ip': o_data['source_ip'],
	'source_port': int(o_data['source_port']),
	'destination_ip': o_data['dest_ip'],
	'destination_port': int(o_data['dest_port']),
	'honeypot': 'wordpot',
	'protocol': 'http'
}

Full Example for Wordpot: https://github.com/Pwnlandia/mnemosyne/pull/7/files

When deploying this new code make sure the new channel name (e.g. wordpot.events) is appended to the channel variable in mnemosyne's config file /opt/mnemosyne/mnemosyne.cfg. If this is not done mnemosyne will not even see these new events.

MHN Deploy Script

Lastly, you need to create a script that can be used to automatically deploy the honeypot/sensor and register the honeypot and its data flows with MHN.

See the example script and script registration for Wordpot: https://github.com/Pwnlandia/mhn/pull/67/files

When deploying this new code, make sure to update the /opt/mhn/server/config.py with the mapping of the honeypot to the channels it is allowed to publish to. Example:

HONEYPOT_CHANNELS = {
    'dionaea': [
        'mwbinary.dionaea.sensorunique',
        'dionaea.capture',
        'dionaea.capture.anon',
        'dionaea.caputres',
        'dionaea.connections'
    ],
    'conpot': ['conpot.events'],
    'snort': ['snort.alerts'],
    'kippo': ['kippo.sessions'],
    'thug': ['thug.files', 'thug.events'],
    'glastopf': ['glastopf.files', 'glastopf.events'],
    'amun': ['amun.events'],
    'wordpot': ['wordpot.events'],  # <-- Needed to add this line for word pot.
}

Updating hpfeeds permissions

Most new honeypots will publish data to their own honeypot specific hpfeeds channels. Because of this, you need to be sure to update the accounts associated with mnemosyne and the geoloc process. Mnemosyne is responsible for indexing and storing the honeypot data so it need to be able subscribe to the honeypot's channel. Geoloc is responsible for normalizing and IP Geolocating events for honeymap and it needs to be able to subscribe to the honeypot's channel. Here are the MongoDB commands to accomplish this:

Open the mongodb console and connect to the "hpfeeds" database:

$ mongo hpfeeds
MongoDB shell version: 2.6.2
connecting to: hpfeeds

Append the new honeypot channel name to mnemosyne's "subscribe" field.

> db.auth_key.update({"identifier": "mnemosyne"}, {"$push": {"subscribe": "wordpot.events"} })
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

Append the new honeypot channel name to geoloc's "subscribe" field.

> db.auth_key.update({"identifier": "geoloc"}, {"$push":{ "subscribe": "wordpot.events" }})
WriteResult({ "nMatched" : 1, "nUpserted" : 0, "nModified" : 1 })

OLD: Here for reference only.

Here is general what is required to get a new honeypot/sensor to work with MHN:

  1. hpfeeds - The sensor should support the HPFeeds transport mechanism to ship log events (as JSON) in realtime and this should be configurable in some way so we can deploy automatically. This is how we did it for Kippo: https://github.com/Pwnlandia/kippo/compare/desaster:master...master. If the log data is simple, then it is much easier than this.
  2. deploy scrip - The sensor needs to be deployable and configurable in an automated fashion so we can create a deploy script for it. This is how we did it for Kippo: https://github.com/Pwnlandia/mhn/compare/8fbd8d5651d8df96d42a8e8839b5310b562d53e8...master
  3. mnemosyne normalizer - Add a normalizer to convert the sensor's JSON format/schema to mnemsyne's schema (kippo example: https://github.com/Pwnlandia/mnemosyne/blob/master/normalizer/modules/kippo_events.py). This is for data ingest into MongoDB.
  4. honeymap processsor - Lastly, add a processor for honeymap (similar to mnemosyne's normalizer). Kippo's processor: https://github.com/Pwnlandia/hpfeeds/blob/master/examples/geoloc/processors.py#L118. This enables the events to be visualized in realtime.
Clone this wiki locally