In this project I have implemented and tested simple neural network models for solving supervised problems. It is divided in two tasks. The regression task will consist in a simple function approximation problem. The classification task will consist in a simple image recognition problem, where the goal is to correctly classify images of handwritten digits (MNIST).
In both cases, but especially for the classification problem, I have explored the use of advanced optimizers and regularization methods (e.g., initialization scheme, momentum, ADAM, early stopping, L2, L1 / sparsity, dropout...) to improve convergence of stochastic gradient descent and promote generalization. I have tuned learning hyperparameters using appropriate search procedures. For the image classification task, I have implemented more advanced convolutional architectures and explored feature visualization techniques to better understand how the deep network is encoding information at different processing layers.
- Implement basic regression and classification tasks
- Explore advanced optimizers and regularization methods
- Optimize hyperparameters using grid/random search and cross-validation
- Implement CNN for classification task
- Visualize weight histograms, activation profiles and receptive fields
A detailed report is present in the nn_homework_01.pdf file