Python implementation of the Multimodal Eigenwords (MM-Eigenwords) 🐍

Real-world photo sequence question answering system (MemexQA). CVPR'18 and TPAMI'19

A detailed description on how to extract and align text, audio, and video features at word-level.

Gowers Method for finding latent networks of multi-modal data

User modelling using Multi-modal fusion

Deep Multiset Canonical Correlation Analysis - An extension of CCA to multiple datasets

My master thesis: Siamese multi-hop attention for cross-modal retrieval.

Segment-level autoencoders for multimodal representation

PyTorch Implementation of HUSE: Hierarchical Universal Semantic Embeddings ( https://arxiv.org/pdf/1911.05978.pdf )

The code for our INTERSPEECH 2020 paper - Jointly Fine-Tuning "BERT-like'" Self Supervised Models to Improve Multimodal Speech Emotion Recognition

Code for COLING2020 paper: Probing Multimodal Embeddings for Linguistic Properties.

Multimodal Bi-Transformers (MMBT) in Biomedical Text/Image Classification

Companion Repo for the Vision Language Modelling YouTube series - https://bit.ly/3PsbsC2 - by Prithivi Da. Open to PRs and collaborations

All experiments were done to classify multimodal data.

Collects a multimodal dataset of Wikipedia articles and their images

This repository contains the implementation of the paper -- Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis

Yet another Ph.D. adventure.

Together Yet Apart: Multimodal Representation Learning for Personalised Visual Art Recommendation

IISAN: Efficiently Adapting Multimodal Representation for Sequential Recommendation with Decoupled PEFT