Getting Started

Data scraping and processing code is organised into modular and extendable jobs written in JavaScript or CoffeeScript. A typical node.io job consists of of taking some input, processing / reducing it in some way, and then outputting the emitted results, although no step is compulsory. Some jobs may not require any input, etc.

Jobs can be run from the command line or through a web interface. To run a job from the command line (extension can be omitted), run

$ node.io myjob

To run jobs through the web interface, copy your jobs to ~/.node_modules and run

$ node.io-web

Basic examples

Let's run through some simple examples highlighting the anatomy of a job. Each example includes a JavaScript and CoffeeScript version and omits the required var nodeio = require('node.io');

Example 1: Hello World!

hello.js

exports.job = new nodeio.Job({
    input: false,
    run: function () {
        this.emit('Hello World!');
    }
});

hello.coffee

class Hello extends nodeio.JobClass
    input: false
    run: (num) -> @emit 'Hello World!'
    
@class = Hello
@job = new Hello()

Example 2: Double each element of input

double.js

exports.job = new nodeio.Job({
    input: [0,1,2],
    run: function (num) {
        this.emit(num * 2);
    }
});

double.coffee

class Double extends nodeio.JobClass
    input: [0,1,2]
    run: (num) -> @emit num * 2
    
@class = Double
@job = new Double()

Example 3: Extend the previous example to quadruple elements

quad.js

var double = require('./double').job;

exports.job = double.extend({
    run: function (num) {
        this.__super__.run(num * 2);
        //Same as: this.emit(num * 4)
    }
});

quad.coffee

Double = require('./double').Class

class Quad extends Double
    run: (num) -> super num * 2
    
@class = Quad
@job = new Quad()

Goto the next tutorial: Working with input / output