89 Open source projects that are alternatives of or similar to iMIX

[TPAMI Special Issue on ICCV 2021 Best Papers, Oral] Just Ask: Learning to Answer Questions from Millions of Narrated Videos

Utils and modules for Speech Language and Multimodal processing using pytorch and pytorch lightning

ROSITA: Enhancing Vision-and-Language Semantic Alignments via Cross- and Intra-modal Knowledge Integration

A modular framework for vision & language multimodal research from Facebook AI Research (FAIR)

An alternative EQA paradigm and informative benchmark + models (BMVC 2019, ViGIL 2019 spotlight)

A lightweight, scalable, and general framework for visual question answering research

MultiGraphGAN for predicting multiple target graphs from a source graph using geometric deep learning.

Research Code for ICCV 2019 paper "Relation-aware Graph Attention Network for Visual Question Answering"

CloudCV Visual Question Answering Demo

Official Repository for the paper Style Transfer for Co-Speech Gesture Animation: A Multi-Speaker Conditional-Mixture Approach published in ECCV 2020 (https://arxiv.org/abs/2007.12553)

Implementation for the paper "Hierarchical Conditional Relation Networks for Video Question Answering" (Le et al., CVPR 2020, Oral)

📷 ❓ Visual Question Answering Demo and Algorithmia API

Flexible time series feature extraction & processing

A PyTorch implementation of VIOLET

Strong baseline for visual question answering

GAKG is a multimodal Geoscience Academic Knowledge Graph (GAKG) framework by fusing papers' illustrations, text, and bibliometric data.

Tensorflow Implementation of Deeper LSTM+ normalized CNN for Visual Question Answering

MinkLoc++: Lidar and Monocular Image Fusion for Place Recognition

Bottom-up attention model for image captioning and VQA, based on Faster R-CNN and Visual Genome

A paper list of research conducted based on wikiHow

Visual Question Answering in Pytorch

(CVPR2021) Kaleido-BERT: Vision-Language Pre-training on Fashion Domain.

Implementation for the paper "Compositional Attention Networks for Machine Reasoning" (Hudson and Manning, ICLR 2018)

Oscar and VinVL

CALVIN - A benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks

Referring Expression Object Segmentation with Caption-Aware Consistency, BMVC 2019

A curated list of Visual Question Answering(VQA)(Image/Video Question Answering),Visual Question Generation ,Visual Dialog ,Visual Commonsense Reasoning and related area.

Neuro-Symbolic Visual Question Answering on Sort-of-CLEVR using PyTorch

[ICCV 2021- Oral] Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

Code for NeurIPS 2019 paper ``Self-Critical Reasoning for Robust Visual Question Answering''

Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

[CVPR 2021 Oral] Official PyTorch code for ClipBERT, an efficient framework for end-to-end learning for image-text and video-text tasks.

A Pytorch implementation of CVPR 2020 paper: Multi-Modal Graph Neural Network for Joint Reasoning on Vision and Scene Text

LipNet with gluon

读过的CV方向的一些论文，图像生成文字、弱监督分割等

Offical implementation of paper "MSAF: Multimodal Split Attention Fusion"

Hadamard Product for Low-rank Bilinear Pooling

[ICCV 2021] TRAR: Routing the Attention Spans in Transformers for Visual Question Answering -- Official Implementation

Pytorch implementation of NIPS 2017 paper "Modulating early visual processing by language"

It is the implementation of paper "Multi-Modal Sarcasm Detection in Twitter with Hierarchical Fusion Model"

PyTorch VQA implementation that achieved top performances in the (ECCV18) VizWiz Grand Challenge: Answering Visual Questions from Blind People

Official Tensorflow Implementation of the AAAI-2020 paper "Temporally Grounding Language Queries in Videos by Contextual Boundary-aware Prediction"

An efficient PyTorch implementation of the winning entry of the 2017 VQA Challenge.

[CVPR 2021] Counterfactual VQA: A Cause-Effect Look at Language Bias

[CVPR 2019 Oral] Social-IQ: A Question Answering Benchmark for Artificial Social Intelligence

Visual Q&A reading list

This repository contains the implementation of the paper -- Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis

PyTorch implementation of "Transparency by Design: Closing the Gap Between Performance and Interpretability in Visual Reasoning"

TensorFlow implementation of "Attentive Modality Hopping for Speech Emotion Recognition," ICASSP-20

Knowledge-Aware machine LEarning (KALE): accessible machine learning from multiple sources for interdisciplinary research, part of the 🔥PyTorch ecosystem

PyTorch implementation for the Neuro-Symbolic Concept Learner (NS-CL).

Multiple Meta-model Quantifying for Medical Visual Question Answering

X-VLM: Multi-Grained Vision Language Pre-Training (ICML 2022)

Code and Data for paper: Zero-shot Visual Question Answering using Knowledge Graph [ ISWC 2021 ]

Bottom-up features extractor implemented in PyTorch.

[CVPR21] Visual Semantic Role Labeling for Video Understanding (https://arxiv.org/abs/2104.00990)

Damage Identification in Social Media Posts using Multimodal Deep Learning: code and dataset

This repository contains the Pytorch implementation for our SCAI (EMNLP-2018) submission "A Knowledge-Grounded Multimodal Search-Based Conversational Agent"

End-to-End learning framework for circular RNA classification from other long non-coding RNA using multimodal deep learning