Language Modeling using ngrams and Kneser-Ney Smoothing

Python implementation of 4-gram language models that use Witten-Bell, Kneser-Ney Smoothing

Demo: Markov password guesser with Backoff and Kneser-Ney smoothing

PyQt application to demonstrate the Kneser-Ney smoothing algorithm for bigram/word prediction.

This repository implements N-gram language modeling with Kneser-Kney and Witten Bell smoothing techniques, including an in-house tokenizer. It also features a neural model with LSTM architecture and calculates perplexities for comparing language and neural models.

Bigram and Trigram Language Models. This repository provides my solution for the 1st Assignment for the course of Text Analytics for the MSc in Data Science at Athens University of Economics and Business.

This repository contains the group projects undertaken during the course "Text Engineering and Analytics" taught by Prof. Ion Androutsopoulos in the context of Msc. in Data Science at Athens University of Economics and Business.

Text Prediction algorithm and app built for the Capstone Project of the Coursera: John Hopkins Data Science Certification. Utilizes a quadri-gram model with Kneser-Ney smoothing and Good Turning Frequency Estimation.

This repository implements an n-gram-based language model for the CS6320 NLP course at UT Dallas, focusing on word sequence prediction, text preprocessing, smoothing techniques, and model evaluation.

Part of Speech Tagger (POS) for Urdu Language with Hidden Markov Model (HMM) using Kneser-Ney Smoothing

Course Repository for ELL881 (Special Topics:Modern Natural Language Processing), 6th Semester, 2023, IITD

Language Modelling for various corpora, Natural Language Generation using LMs, Corpus Statistics Visualization

Course Materials (along with assignments) for Intro to NLP, done as a part for requirement of the course "Introduction to NLP" (course-code: CS7.401.S22) @ IIITH. Note: If you are cloning this or taking help of this repo, try to star the repo.

Interpolated Kneser-Ney smoothing with an out-of-vocabulary correction and discount estimated from training data