A Pytorch LSTM Language model trained on Penn Tree Bank Dataset.
- Introduction
- Setup
- Run the code
- Training
- Sample Generation
- Play with the model
- Connect with me
- License
This is a PyTorch Implementation for an LSTM-based language model. The model is trained on Penn Tree Bank dataset using Adam optimizer with a learning rate 0.001 and 10 epochs.
The code is using pipenv as a virtual environment and package manager. To run the code, all you need is to install the necessary dependencies. open the terminal and type:
git clone https://github.com/Khamies/LSTM-Language-Generator.gitcd LSTM-Language-Generatorpipenv install
And you should be ready to go to play with code and build upon it!
-
To train the model, run:
python main.py -
To train the model with specific arguments, run:
python main.py --batch_size=64. The following command-line arguments are available:- Batch size:
--batch_size - bptt:
--bptt - Learning rate:
--lr - Embedding size:
--embed_size - Hidden size:
--hidden_size - Latent size:
--latent_size
- Batch size:
The model is trained on 10 epochs using Adam as an optimizer with a learning rate = 0.001 and batch size = 32, you can find all the model settings in settings.py. Here is the loss curve for the training step:
-
Negative Likelihood Loss
Here are some generated samples from the model:
he said <pad> is n't expected to be the first quarter of
the company said will be used to repay the company'sTo play with the model, a jupyter notebook has been provided, you can find it here
@misc{Khamies2021LSTM-Language-Generator, author = {Khamies, Waleed}, title = {A PyTorch Implementation for an LSTM-based language model}, year = {2021}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/Khamies/LSTM-Language-Generator}}, }
For any question or a collaboration, drop me a message here
Follow me on Linkedin!
Thank you ❤️
