GitPlanet

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

Created with love in Canada, visit hostnodejs.com today

Feel like to post an Ad? Learn Details
All Projects → nerdimite → neuro-symbolic-ai-soc

nerdimite / neuro-symbolic-ai-soc

Licence: other
Neuro-Symbolic Visual Question Answering on Sort-of-CLEVR using PyTorch

Programming Languages

Jupyter Notebook
11667 projects
python
139335 projects - #7 most used programming language

Labels

program-synthesispytorchartificial-intelligencevqareasoningclevrneuro-symbolic-learningpytorch-implementationneuro-symbolic

Projects that are alternatives of or similar to neuro-symbolic-ai-soc

probnmn-clevr
Code for ICML 2019 paper "Probabilistic Neural-symbolic Models for Interpretable Visual Question Answering" [long-oral]
Stars: ✭ 63 (+53.66%)
Mutual labels:  vqa, clevr
Arc
The Abstraction and Reasoning Corpus
Stars: ✭ 1,598 (+3797.56%)
Mutual labels:  program-synthesis
Pytorch Vqa
Strong baseline for visual question answering
Stars: ✭ 158 (+285.37%)
Mutual labels:  vqa
plur
PLUR (Programming-Language Understanding and Repair) is a collection of source code datasets suitable for graph-based machine learning. We provide scripts for downloading, processing, and loading the datasets. This is done by offering a unified API and data structures for all datasets.
Stars: ✭ 67 (+63.41%)
Mutual labels:  program-synthesis
Openvqa
A lightweight, scalable, and general framework for visual question answering research
Stars: ✭ 198 (+382.93%)
Mutual labels:  vqa
L2
λ² is a tool for synthesizing functional programs from input-output examples.
Stars: ✭ 59 (+43.9%)
Mutual labels:  program-synthesis
Vqa regat
Research Code for ICCV 2019 paper "Relation-aware Graph Attention Network for Visual Question Answering"
Stars: ✭ 129 (+214.63%)
Mutual labels:  vqa
VT-UNet
[MICCAI2022] This is an official PyTorch implementation for A Robust Volumetric Transformer for Accurate 3D Tumor Segmentation
Stars: ✭ 151 (+268.29%)
Mutual labels:  pytorch-implementation
autogoal
A Python framework for program synthesis with a focus on Automated Machine Learning.
Stars: ✭ 153 (+273.17%)
Mutual labels:  program-synthesis
Popper
Popper is an inductive logic programming (ILP) system.
Stars: ✭ 95 (+131.71%)
Mutual labels:  program-synthesis
GAS
Generative Art Synthesizer - a python program that generates python programs that generates generative art
Stars: ✭ 42 (+2.44%)
Mutual labels:  program-synthesis
deepcoder
Deepcoder paper implementation
Stars: ✭ 96 (+134.15%)
Mutual labels:  program-synthesis
awesome-program-synthesis
An curated list of papers on program synthesis.
Stars: ✭ 56 (+36.59%)
Mutual labels:  program-synthesis
Clipbert
[CVPR 2021 Oral] Official PyTorch code for ClipBERT, an efficient framework for end-to-end learning for image-text and video-text tasks.
Stars: ✭ 168 (+309.76%)
Mutual labels:  vqa
self critical vqa
Code for NeurIPS 2019 paper ``Self-Critical Reasoning for Robust Visual Question Answering''
Stars: ✭ 39 (-4.88%)
Mutual labels:  vqa
Vqa Mfb
Stars: ✭ 153 (+273.17%)
Mutual labels:  vqa
neural inverse knitting
Code for Neural Inverse Knitting: From Images to Manufacturing Instructions
Stars: ✭ 30 (-26.83%)
Mutual labels:  program-synthesis
suslik
Synthesis of Heap-Manipulating Programs from Separation Logic
Stars: ✭ 107 (+160.98%)
Mutual labels:  program-synthesis
ResUNetPlusPlus-with-CRF-and-TTA
ResUNet++, CRF, and TTA for segmentation of medical images (IEEE JBIHI)
Stars: ✭ 98 (+139.02%)
Mutual labels:  pytorch-implementation
pytorch-convcnp
A PyTorch Implementation of Convolutional Conditional Neural Process.
Stars: ✭ 41 (+0%)
Mutual labels:  pytorch-implementation
View All Similar Projects ➔

Neuro-Symbolic AI for Visual Question Answering

Sort-of-CLEVR Dataset

Neuro-Symbolic AI allows us to combine Deep Learning’s superior pattern recognition abilities with the reasoning abilities of symbolic methods like program synthesis. This repository is an implementation of NSAI for Visual Question Answering on the Sort-of-CLEVR dataset using PyTorch. This implementation is inspired by the Neuro-Symbolic VQA paper by MIT-IBM Watson AI Lab.

The basic idea behind using NSAI for VQA is parsing the visual scene into a symbolic representation and using NLP to parse the query into an executable program which the program executor can use on the scene to find the answer. This implementation gets more than 99% on the Sort-of-CLEVR dataset.

Requirements

  • Pytorch <= 1.7
  • Torchtext <= 0.8.0
  • Torchvision <= 0.8.0
  • OpenCV
  • dlib
  • Scikit Learn
  • Pandas
  • Numpy

Usage

  • The Step-by-Step usage is in the NSAI on Sort-of-CLEVR.ipynb notebook from training the individual modules to plugging everything together to test it.
  • You can easily run this repository using Colab Open in Colab
  • To understand more about the design and workflow, check out NSAI Flow Diagram.pdf which contains the workflows of every component i.e. Perception Module, Semantic Parser and Program Executor.

References

Note that the project description data, including the texts, logos, images, and/or trademarks, for each open source project belongs to its rightful owner. If you wish to add or remove any projects, please contact us at [email protected].