Hidden Layers on Neat

[millefh]

Is it possible to create hidden layers in Neat Networks? It seems as it was possible in Neataptic, if it is not a feature, is there a specific reason for it?

I been trying out the network option when creating a new neat but it seems to do no difference in structure or otherwise.

[christianechevarria]

Thanks for your question @millefh ! Neural networks in Carrot, as well as Neataptic and Synaptic (the source of both libraries), are architecture free this basically means that although you can construct neural networks with layers like this:

let { architect } = require('@liquid-carrot/carrot');

// Add as many hidden layers as needed
let Perceptron = new architect.Perceptron(4, 5, 20, 5, 10, 1);

Once the network is constructed there is no specific "layer" that a neuron exists in, in fact, they are all stored in one common array on the network prototype: Network.nodes so that if you looped through it you would get every neuron in the network.

This has a BIG benefit to evolving networks as far as how complex the library is because we don't have to worry where a neuron is in terms of a layer, instead we (the algorithm) can just add or remove neurons as needed and the network will operate perfectly fine.

I been trying out the network option when creating a new neat but it seems to do no difference in structure or otherwise.

The network option in the Neat constructor is a template to start evolving from, so usually when you start evolving with NEAT you start with just the input and the output neurons, but with the network option you could start evolving from the big network we created in the previous example: new architect.Perceptron(4, 5, 20, 5, 10, 1)

If you wanted to create a network with specific "layer" structures of different kinds you could do something like this example:

let architect = require('@liquid-carrot/carrot').architect
let Layer = require('@liquid-carrot/carrot').Layer

let input = new Layer.Dense(1);
let hidden1 = new Layer.LSTM(5);
let hidden2 = new Layer.GRU(1);
let output = new Layer.Dense(1);

// connect however you want
input.connect(hidden1);
hidden1.connect(hidden2);
hidden2.connect(output);

let yourCustomNetwork = architect.Construct([input, hidden1, hidden2, output]);

And then pass yourCustomNetwork to the Neat constructor to start evolving from there. Although, I would caution against using the Layer portion of the library since it was notoriously buggy (we are actively working on a new version of the library in TypeScript that is set to replace the current version and bring more stability to the project). If you're interested in that version it's inside the typescript branch of the project

I hope I was able to answer all of your questions!

[millefh]

Thank you so much for you fast and informative answer!

Is it possible to use Layer without requiring carrot using Node.js?
Right now I am using
const {Neat, Network, methods, architect, Layer} = carrot;

Which gives an error when trying to make a new Layer
TypeError: Layer.Dense is not a constructor

[christianechevarria]

For the TypeScript version it's slightly different, I believe the correct way would be to do new DenseLayer per the working TypeScript docs here:

https://raimannma.github.io/carrot/classes/denselayer.html#constructor

I would reference these while using the TypeScript version as these document the latest API. I'll be updating the README to point to this as well so that newcomers can try it directly

Thanks again for asking!

[millefh]

Awesome! I have been having a lot of fun playing around with Carrot the past few days. Keep up the good work and thank you for your help. 👍