Customize

You can customize the darklight core engine on your demand.

Collector

A collector is an onion domain collecting part that collects onion domain from outer sources such as Hidden Wiki or Fresh onions, etc.

Add custom collector at source directory. This will be automatically detected when the engine starts.

from source.base import SourceBase

class CustomCollector(SourceBase):
    def collect(self):
        # TO-DO: Develop your own collecting code.
        # save urls into 'self.urls' list
	pass

Pipeline

A pipeline is a preprocessor for normalizing data before inserting onion domain report crawled from the core engine.

You can add your custom pipeline into pipeline/source directory. This will be automatically detected when the engine starts.

from pipeline import Pipeline

class CustomPipeline(Pipeline):
    def handle(self):
	# TO-DO: Add your own pipeline code and save into the database.
	pass