Named entity recognition in Malayalam using BiLSTM and TENER (Transformer Encoder)

This is a tool that encrypts a sequence of words (or pieces of texts) using the AES-256 algorithm and encodes the encrypted result into a PNG image by linking each byte value to a specific color. It also decodes the before image to get back the original sequence of words

This is project for sequence to sequence NLP task. We developed a custom model to understand the process of task using PyTorch. We also fine tuned pre-trained transformer models to improve the performance of translation task.

An Introduction to Natural Language Processing (NLP)

an efficient ranked retrieval system for English corpora, optimised with VBE and BPE.

Feature extraction from sequential data

Transformer implementation in pytorch trained on NVIDIA A100 in fp16

Byte-Pair Encoding (BPE) (subword-based tokenization) algorithm implementaions from scratch with python

Kotlin multiplatform BPE tokenizer library for OpenAI models

Modern Eager TensorFlow implementation of Attention Is All You Need

R package for Byte Pair Encoding based on YouTokenToMe

A byte-level Byte Pair Encoding (BPE) algorithm for tokenization in Large Language Models (LLMs), similar to those used in GPT, Llama, and Mistral.

Order-agnostic lossless compressor using BPE and Huffman Coding.

Fast BPE algorithm to generate byte pair encodings from text corpus, it's written in rust and approximately 20x faster than it's python implementation

Byte-pair encoding implementation in Python.

Byte pair encoding tokenizer as used in some large language models.

This repository houses my assignments completed during the Deep Learning course as part of my Master's in Data Analytics program. Explore diverse projects showcasing hands-on applications of advanced neural networks and machine learning techniques.

A Visualizer to check how BPE Tokenizer in an LLM Works