*Written by Spencer Ball, November 2020
A reccurent neural network which classifies sentences as either objective or subjective.
88.1% Validation Accuracy achieved. For confusion matrix and accuracy/loss plots see output at bottom of the notebook file.
Image normalization, input dataset structuring (torch DataLoader creation), CNN model definition and instantiation, full training loop, and model performance visualization.
The 6 first hyperparameters at the top of the file were optimized through a grid search.
Batch normalization allows for a more accurate loss calculation and backwards step by effectively separating the training of each fully connected layer. This phenomenon is evident because when batch normalization is used, training time to reach a given validation accuracy is reduced and maximum achievable validation accuracy rises. Additionally, I found using cross-entropy loss instead of mean-sqaured-error loss sped up the training process considerably. Specifically, with CELoss, ~70% validation accuracy is achieved in 3 epochs, whereas reaching this accuracy would take 5 or 6 epochs using MSELoss. This is likely because the cross entropy loss function grows very quickly as the trainable parameters move in the wrong direction.