Literature on Graph Neural Networks Acceleration

A reading list for deep graph learning acceleration, including but not limited to related research on software and hardware level. The list covers related papers, conferences, tools, books, blogs, courses and other resources. We have a team of Maintaners responsible for maintainance, meanwhile also welcome contributions from anyone.

Literatures in this page are arranged from a classification perspective, including the following topics:

Literature on Graph Neural Networks Acceleration

Click here to view these literatures in a reverse chronological order. You can also find Related Conferences, Graph Learning Tools, Learning Materials on GNNs and Other Resources in General Resources.

Hardware Acceleration for Graph Neural Networks

[EuroSys 2021] Accelerating Graph Sampling for Graph Machine Learning Using GPUs. Jangda, et al. [Paper]
[ASICON 2019] An FPGA Implementation of GCN with Sparse Adjacency Matrix. Ding et al. [Paper]
[MICRO 2021] AWB-GCN: A Graph Convolutional Network Accelerator with Runtime Workload Rebalancing. Geng et al. [Paper]
[DAC 2021] BlockGNN: Towards Efficient GNN Acceleration Using Block-Circulant Weight Matrices. Zhou et al. [Paper]
[FCCM 2021] BoostGCN: A Framework for Optimizing GCN Inference on FPGA. Zhang et al. [Paper]
[TCAD 2021] Cambricon-G: A Polyvalent Energy-efficient Accelerator for Dynamic Graph Neural Networks. Song et al. [Paper]
[ICCAD 2020] DeepBurning-GL: an automated framework for generating graph neural network accelerators. Liang et al. [Paper]
[DAC 2021] DyGNN: Algorithm and Architecture Support of Dynamic Pruning for Graph Neural Networks. Chen et al. [Paper]
[ICCAD 2021] DARe: DropLayer-Aware Manycore ReRAM architecture for Training Graph Neural Networks. Aqeeb et al. [Paper]
[arXiv 2022] Enabling Flexibility for Sparse Tensor Acceleration via Heterogeneity. Qin et al. [Paper]
[TC 2020] EnGN: A High-Throughput and Energy-Efficient Accelerator for Large Graph Neural Networks. Liang et al. [Paper]
[arXiv 2022] FlowGNN: A Dataflow Architecture for Universal Graph Neural Network Inference via Multi-Queue Streaming. Sarkar et al. [Paper]
[Access 2020] FPGAN: An FPGA Accelerator for Graph Attention Networks With Software and Hardware Co-Optimization. Yan et al. [Paper]
[HPCA 2022] GCoD: Graph Convolutional Network Acceleration via Dedicated Algorithm and Accelerator Co-Design. You et al. [Paper]
[FCCM 2022] GenGNN: A Generic FPGA Framework for Graph Neural Network Acceleration. Stefan, et al. [Paper]
[arXiv 2021] GNNIE: GNN Inference Engine with Load-balancing and Graph-Specific Caching. Mondal et al. [Paper]
[HPCA 2021] GCNAX: A Flexible and Energy-efficient Accelerator for Graph Convolutional Neural Networks. Li et al. [Paper]
[SC 2020] GE-SpMM: General-Purpose Sparse Matrix-Matrix Multiplication on GPUs for Graph Neural Networks. Huang, et al. [Paper]
[ATC 2021] GLIST: Towards In-Storage Graph Learning. Li, et al. [Paper]
[DAC 2021] GNNerator: A Hardware/Software Framework for Accelerating Graph Neural Networks. Stevens et al. [Paper]
[CCIS 2020] GNN-PIM: A Processing-in-Memory Architecture for Graph Neural Networks. Wang et al. [Paper]
[FPGA 2020] GraphACT: Accelerating GCN Training on CPU-FPGA Heterogeneous Platforms. Zeng et al. [Paper]
[arXiv 2020] GRIP: A Graph Neural Network Accelerator Architecture. Kiningham et al. [Paper]
[arXiv 2022] GROW: A Row-Stationary Sparse-Dense GEMM Accelerator for Memory-Efficient Graph Convolutional Neural Networks. Kang et al. [Paper]
[CAL 2021] Hardware Acceleration for GCNs via Bidirectional Fusion. Li et al. [Paper]
[DAC 2020] Hardware Acceleration of Graph Neural Networks. Auten, et al. [Paper]
[ASAP 2020] Hardware Acceleration of Large Scale GCN Inference. Zhang et al.[Paper]
[FAST 2022] Hardware/Software Co-Programmable Framework for Computational SSDs to Accelerate Deep Learning Service on Large-Scale Graphs. Kwon, et al. [Paper]
[HPCA 2020] HyGCN: A GCN Accelerator with Hybrid Architecture. Yan, et al. [Paper]
[arXiv 2021] LW-GCN: A Lightweight FPGA-based Graph Convolutional Network Accelerator. Tao, et al. [Paper]
[IPDPS 2022] Model-Architecture Co-Design for High Performance Temporal GNN Inference on FPGA. Zhou et al. [Paper]
[DAC 2021] PIMGCN: A ReRAM-Based PIM Design for Graph Convolutional Network Acceleration. Yang, et al. [Paper]
[MICRO 2021] Point-X: A Spatial-Locality-Aware Architecture for Energy-Efficient Graph-Based Point-Cloud Deep Learning. Zhang, et al. [Paper]
[DATE 2021] ReGraphX: NoC-Enabled 3D Heterogeneous ReRAM Architecture for Training Graph Neural Networks. Arka, et al. [Paper]
[TCAD 2021] Rubik: A Hierarchical Architecture for Efficient Graph Neural Network Training. Chen et al. [Paper]
[ICPADS 2020] S-GAT: Accelerating Graph Attention Networks Inference on FPGA Platform with Shift Operation. Yan et al. [Paper]
[CICC 2022] StreamGCN: Accelerating Graph Convolutional Networks with Streaming Processing. Sohrabizadeh et al. [Paper]
[EuroSys 2021] Tesseract: distributed, general graph pattern mining on evolving graphs. Bindschaedler, et al. [Paper]
[ICA3PP 2020] Towards a Deep-Pipelined Architecture for Accelerating Deep GCN on a Multi-FPGA Platform. Cheng et al. [Paper]
[SCIS 2021] Towards Efficient Allocation of Graph Convolutional Networks on Hybrid Computation-In-Memory Architecture. Chen et al. [Paper]
[arXiv 2021] VersaGNN: a Versatile accelerator for Graph neural networks. Shi et al. [Paper]
[arXiv 2021] ZIPPER: Exploiting Tile- and Operator-level Parallelism for General and Scalable Graph Neural Network Acceleration. Zhang et al. [Paper]

System Designs for Deep Graph Learning

[CLUSTER 2021] 2PGraph: Accelerating GNN Training over Large Graphs on GPU Clusters.Zhang et al. [Paper]
[MLSys 2022] Accelerating Training and Inference of Graph Neural Networks with Fast Sampling and Pipelining. Kaler et al. [Paper]
[JPDC 2021] Accurate, efficient and scalable training of Graph Neural Networks Zeng et al. [Paper]
[KDD 2019] AliGraph: a comprehensive graph neural network platform. Yang et al. [Paper] [GitHub]
[OSDI 2021] Dorylus: Affordable, Scalable, and Accurate GNN Training with Distributed CPU Servers and Serverless Threads. Thorpe et al. [Paper] [GitHub]
[EuroSys 2021] DGCL: an efficient communication library for distributed GNN training. Cai et al. [Paper]
[IA3 2020] DistDGL: Distributed Graph Neural Network Training for Billion-Scale Graphs. Zheng et al. [Paper]
[CoRR 2019] Deep Graph Library: Towards Efficient and Scalable Deep Learning on Graphs. Wang [Paper] [GitHub] [Home Page]
[TPDS 2021] Efficient Data Loader for Fast Sampling-Based GNN Training on Large Graphs. Bai et al. [Paper]
[TPDS 2020] EDGES: An Efficient Distributed Graph Embedding System on GPU Clusters. Yang et al. [Paper]
[GNNSys 2021] FedGraphNN: A Federated Learning System and Benchmark for Graph Neural Networks.He et al. [Paper] [Poster]
[EuroSys 2021] FlexGraph: a flexible and efficient distributed framework for GNN training. Wang et al. [Paper] [GitHub]
[ICCAD 2020] fuseGNN: accelerating graph convolutional neural network training on GPGPU. Chen et al. [Paper] [GitHub]
[SC 2020] FeatGraph: A Flexible and Efficient Backend for Graph Neural Network Systems. Hu et al. [Paper] [Github]
[ICLR 2019] Fast Graph Representation Learning with PyTorch Geometric. Fey et al. [Paper] [GitHub] [Documentation]
[IPDPS 2021] FusedMM: A Unified SDDMM-SpMM Kernel for Graph Embedding and Graph Neural Networks. Rahman et al. [Paper]
[GNNSys 2021] Graphiler: A Compiler for Graph Neural Networks.Xie et al. [Paper] [Poster]
[OSDI 2021] GNNAdvisor: An Adaptive and Efﬁcient Runtime System for GNN Acceleration on GPUs Wang et al. [Paper]
[AccML 2020] GIN : High-Performance, Scalable Inference for Graph Neural Networks. Fu et al. [Paper]
[JPDC 2021] High performance GPU primitives for graph-tensor learning operations. Zhang et al. [Paper]
[GNNSys 2021] IGNNITION: A framework for fast prototyping of Graph Neural Networks.Pujol-Perich et al. [Paper] [Poster]
[MLSys 2020] Improving the Accuracy, Scalability, and Performance of Graph Neural Networks with Roc. Jia [Paper]
[GNNSys 2021] Load Balancing for Parallel GNN Training.Su et al. [Paper] [Poster]
[CVPR 2020] L2-GCN: Layer-Wise and Learned Efficient Training of Graph Convolutional Networks.You et al. [Paper]
[ATC 2019] NeuGraph: Parallel Deep Neural Network Computation on Large Graphs. Ma et al. [Paper]
[arXiv 2021] PyTorch Geometric Temporal: Spatiotemporal Signal Processing with Neural Machine Learning Models. Rozemberczki et al. [Paper] [GitHub]
[SoCC 2020] PaGraph: Scaling GNN training on large graphs via computation-aware caching. Lin et al. [Paper]
[IPDPS 2020] Pcgcn: Partition-centric processing for accelerating graph convolutional network. Tian et al. [Paper]
[SysML 2019] PyTorch-BigGraph: A Large-scale Graph Embedding System. Lerer et al. [Paper] [GitHub]
[arXiv 2021] QGTC: Accelerating Quantized GNN via GPU Tensor Core. Wang et al. [Paper]
[arXiv 2018] Relational inductive biases, deep learning, and graph networks. Battaglia et al. [Paper] [GitHub]
[FPGA 2022] SPA-GCN: Efficient and Flexible GCN Accelerator with Application for Graph Similarity Computation. Atefeh, et al. [Paper]
[EuroSys 2021] Seastar: vertex-centric programming for graph neural networks. Wu, et al. [Paper]
[arXiv 2021] TC-GNN: Accelerating Sparse Graph Neural Network Computation Via Dense Tensor Core on GPUs. Wang, et al. [Paper]

Algorithmic Acceleration for Graph Neural Networks

[ICLR 2022] Adaptive Filters for Low-Latency and Memory-Efficient Graph Neural Networks. Tailor et al. [Paper]
[GNNSys 2021] Adaptive Filters and Aggregator Fusion for Efficient Graph Convolutions.Tailor et al. [Paper] [GitHub]
[PMLR 2021] A Unified Lottery Ticket Hypothesis for Graph Neural Networks. Chen et al. [Paper]
[PVLDB 2021] Accelerating Large Scale Real-Time GNN Inference using Channel Pruning. Zhou et al. [Paper]
[IPDPS 2019] Accurate, efficient and scalable graph embedding. Zeng et al. [Paper]
[MLSys 2022] BNS-GCN: Efficient Full-Graph Training of Graph Convolutional Networks with Boundary Node Sampling. Wan et al. [Paper]
[CVPR 2021] Bi-GCN: Binary Graph Convolutional Network. Wang et al. [Paper]
[CVPR 2021] Binary Graph Neural Networks. Bahri et al. [Paper]
[GLSVLSI 2021] Co-Exploration of Graph Neural Network and Network-on-Chip Design Using AutoML. Manu et al. [Paper]
[KDD 2021] DeGNN: Improving Graph Neural Networks with Graph Decomposition. Miao, et al. [Paper]
[FPGA 2022] DecGNN: A Framework for Mapping Decoupled GNN Models onto CPU-FPGA Heterogeneous Platform. Zhang, et al. [Paper]
[ICLR 2021] Degree-Quant: Quantization-Aware Training for Graph Neural Networks. Tailor et al. [Paper]
[ICLR 2022] EXACT: Scalable Graph Neural Networks Training via Extreme Activation Compression. Liu et al. [Paper]
[AAAI 2022] Early-Bird GCNs: Graph-Network Co-Optimization Towards More Efficient GCN Training and Inference via Drawing Early-Bird Lottery Tickets. You et al. [Paper]
[arXiv 2021] Edge-featured Graph Neural Architecture Search.Cai et al. [Paper]
[SC 2021] Efficient scaling of dynamic graph neural networks.Chakaravarthy et al. [Paper]
[GNNSys 2021] Efficient Data Loader for Fast Sampling-based GNN Training on Large Graphs.Bai et al. [Paper] [Poster]
[GNNSys 2021] Effiicent Distribution for Deep Learning on Large Graphs.Hoang et al. [Paper] [Poster]
[ICLR 2021 Open Review] FGNAS: FPGA-AWARE GRAPH NEURAL ARCHITECTURE SEARCH. Qing et al. [Paper]
[WWW 2022] Fograph: Enabling Real-Time Deep Graph Inference with Fog Computing. Zeng et al. [[Paper]](
[ICLR 2022] Graph-less Neural Networks: Teaching Old MLPs New Tricks Via Distillation. Zhang et al. [Paper]
[ICDM 2021] GraphANGEL: Adaptive aNd Structure-Aware Sampling on Graph NEuraL Networks. Peng et al. [Paper]
[MLSys 2022] Graphiler: Optimizing Graph Neural Networks with Message Passing Data Flow Graph. Xie et al. [Paper]
[ICML 2021] GraphNorm: A Principled Approach to Accelerating Graph Neural Network Training. Cai et al. [Paper]
[NeurIPS 2021] Graph Differentiable Architecture Search with Structure Learning. Qin et al. [Paper]
[KDD 2021] Global Neighbor Sampling for Mixed CPU-GPU Training on Giant Graphs. Dong et al. [Paper]
[arXiv 2021] GNNSampler: Bridging the Gap between Sampling Algorithms of GNN and Hardware. Liu et al. [Paper]
[ICCAD 2021] G-CoS: GNN-Accelerator Co-Search Towards Both Better Accuracy and Efficiency. Zhang et al. [Paper]
[ICLR 2020] GraphSAINT: Graph Sampling Based Inductive Learning Method. Xue et al. [Paper]
[NeurIPS 2020] Gcn meets gpu: Decoupling “when to sample” from “how to sample”.Morteza et al. [Paper]
[FPGA 2022] HP-GNN: Generating High Throughput GNN Training Implementation on CPU-FPGA Heterogeneous Platform. Lin, et al. [Paper]
[ICLR 2022] IGLU: Efficient GCN Training via Lazy Updates. Narayanan et al. [Paper]
[ICLR 2022] Learn Locally, Correct Globally: A Distributed Algorithm for Training Graph Neural Networks. Ramezani et al. [Paper]
[arXiv 2020] Learned Low Precision Graph Neural Networks. Zhao et al. [Paper]
[ICML 2021] Optimization of Graph Neural Networks: Implicit Acceleration by Skip Connections and More Depth. Xu et al. [Paper]
[RTAS 2021] Optimizing Memory Efficiency of Graph Neural Networks on Edge Computing Platforms. Zhou et al. [Paper] [GitHub]
[ICLR 2022] PipeGCN: Efficient full-graph training of graph convolutional networks with pipelined feature communication. Wan et al. [Paper]
[www 2022] PaSca: A Graph Neural Architecture Search System under the Scalable Paradigm. Zhang et al. [Paper]
[KDD 2021] Performance-Adaptive Sampling Strategy Towards Fast and Accurate Graph Neural Networks. Yoon et al. [Paper]
[www 2022] Resource-Efficient Training for Large Graph Convolutional Networks with Label-Centric Cumulative Sampling. Lin et al. [Paper]
[SC 2020] Reducing Communication in Graph Neural Network Training. Tripathy et al. [Paper] [GitHub]
[MLSys 2022] Sequential Aggregation and Rematerialization: Distributed Full-batch Training of Graph Neural Networks on Large Graphs. Mostafa. [Paper]
[arXiv 2022] SUGAR: Efficient Subgraph-level Training via Resource-aware Graph Partitioning. Xue et al. [Paper]
[ICTAI 2020] SGQuant: Squeezing the Last Bit on Graph Neural Networks with Specialized Quantization. Feng et al. [Paper]
[KDD 2020] TinyGNN: Learning Efficient Graph Neural Networks. Yan et al. [Paper]

Surveys and Performance Analysis on Graph Learning

[GNNSys 2021] Analyzing the Performance of Graph Neural Networks with Pipe Parallelism.T. Dearing et al. [Paper] [Poster]
[IJCAI 2021] Automated Machine Learning on Graphs: A Survey. Zhang et al. [Paper]
[arXiv 2021] A Taxonomy for Classification and Comparison of Dataflows for GNN Accelerators. Garg et al. [Paper]
[ISCAS 2021] Characterizing the Communication Requirements of GNN Accelerators: A Model-Based Approach. Guirado et al. [Paper]
[CAL 2022] Characterizing and Understanding Distributed GNN Training on GPUs. Lin et al. [Paper]
[CAL 2020] Characterizing and Understanding GCNs on GPU. Yan et al. [Paper]
[arXiv 2020] Computing Graph Neural Networks: A Survey from Algorithms to Accelerators. Abadal et al. [Paper]
[KDD 2020] Deep Graph Learning: Foundations, Advances and Applications. Rong et al. [Paper]
[TKDE 2020] Deep Learning on Graphs: A Survey. Zhang et al.[paper]
[arXiv 2021] Graph Neural Networks: Methods, Applications, and Opportunities. Waikhom et al. [Paper]
[ISPASS 2021] GNNMark: A Benchmark Suite to Characterize Graph Neural Network Training on GPUs. Baruah et al. [Paper]
[CAL 2021] Making a Better Use of Caches for GCN Accelerators with Feature Slicing and Automatic Tile Morphing. Yoo et al. [Paper]
[ISPASS 2021] Performance Analysis of Graph Neural Network Frameworks. Wu et al. [Paper]
[arXiv 2022] Survey on Graph Neural Network Acceleration: An Algorithmic Perspective. Liu, et al. [Paper]
[arXiv 2021] Sampling methods for efficient training of graph convolutional networks: A survey. Liu et al. [Paper]
[PPoPP 2021] Understanding and bridging the gaps in current GNN performance optimizations. Huang et al. [Paper]
[arXiv 2021] Understanding GNN Computational Graph: A Coordinated Computation, IO, and Memory Perspective. Zhang et al. [Paper]
[arXiv 2021] Understanding the Design Space of Sparse/Dense Multiphase Dataflows for Mapping Graph Neural Networks on Spatial Accelerators. Garg et al. [Paper]

Maintainers

Ao Zhou, Beihang University. [GitHub]
Yingjie Qi, Beihang University. [GitHub]
Tong Qiao, Beihang University. [GitHub]

Cheap and reliable Node.js hosting starts at $3/month, and $1/month static HTML hosting

BUAA-CI-Lab / Literatures-on-GNN-Acceleration

Labels

Projects that are alternatives of or similar to Literatures-on-GNN-Acceleration