Deep Learning Models with Stateset Network Data
This is going to be a working notebook of the latest developments in Deep Learning and Natural Language Processing (NLP) and how to use data on Stateset Network to train the deep learning models.
In 2014 I took a course in Computational Linguistics that introduced me to logic programming in Prolog.
In 2016 I worked on my first NLP based bots using a context, utterance, entity detection and was training the model based on chatbot interactions. I used a stack powered by luis.ai and microsoft botkit, node.js while working at Apttus which later become Max.
In 2020 and we now have BERT and Transformer based models, XLNet: Generalized Autoregressive Pretrianing, state-of-the art deep learning methods and other state of the art natural language processing libraries.
I have developed the export from Stateset Network functionality and am now working on using the Tabular library to train models.
Fast forward 3 years and now we can create nanoGPT's (https://github.com/karpathy/nanoGPT).
I am now working on creating nanoGPT based on data at domsteil.com/dom.txt.
Here is my model colab: https://colab.research.google.com/gist/domsteil/4dd97db4caef727e3c50aefb07a3bd94/copy-of-gpt-dev.ipynb
I am going to work on serving a version using flask that can respond to prompts.