QASMBench is moved to PNNL-QAMSBench.
An OpenQASM benchmark suite for NISQ evaluation. The .qasm code can be directly loaded in IBM Quantum Experience for execution. Please see our paper (attached and arXiv) for details.
Current version
Latest version: 1.2
About QASMBench
In this repository you will find a light-weighted benchmark suite based on IBM OpenQASM language (see spec). It collects commonly seen quantum algorithms and routines from various domains including chemistry, simulation, linear algebra, searching, optimization, arithmetic, machine learning, fault tolerance, cryptography, etc. QASMBench trades-off between generality and usability, covering the number of qubits ranging from 2 to 60K, and the circuit depth from 4 to 12M. We set most of the benchmarks with qubits less than 16 so they can be directly verified on IBM's public-available quantum machine -- IBM Quantum Experience. You may also want to use our Density-Matrix quantum simulator (DM-Sim) that can efficiently run on CPU and GPU-clusters.
OpenQASM
OpenQASM (Open Quantum Assembly Language) is a low-level quantum intermediate representation (IR) for quantum instructions, similar to the traditional Hardware-Description-Language (HDL) like Verilog and VHDL. OpenQASM is the open-source unified low-level assembly language for IBM quantum machines publically available on cloud that have been investigated and verified by many existing research works. Several popular quantum software frameworks use OpenQASM as one of their output-formats, including Qiskit, Cirq, Scaffold, ProjectQ, etc.
Qiskit
The Quantum Information Software Kit (Qiskit) is a quantum software developed by IBM. It is based on Python. OpenQASM can be generated from Qiskit via:
QuantumCircuit.qasm()
Cirq
Cirq is a quantum software framework from Google. OpenQASM can be generated from Cirq (not fully compatible) via:
cirq.Circuit.to_qasm()
Scaffold
Scaffold is a quantum programming language embedded in the C/C++ programming language based on the LLVM compiler toolchain. A Scaffold program can be compiled by Scaffcc to OpenQASM via "-b" compiler option.
ProjectQ
ProjectQ is a quantum software platform developed by Steiger et al. from ETH Zurich. The official website is here. ProjectQ can generate OpenQASM when using IBM quantum machines as the backends:
IBMBackend.get_qasm()
QASMBench Benchmarks
Depending on the number of qubits used, QASMBench includes three categories. For the introduction of the benchmarking routines under each category, please see our paper for detail. For each benchmark in the following tables, we list its name, brief description, and the algorithm category it belongs to, which is based on this Nature paper by adding the categories of quantum arithmetic, quantum machine learning and quantum communication.
The 'Gates' here refers to the number of Standard OpenQASM gates (see our paper) but excluding those gates in a branching if statement. It is known that physical qubits in an NISQ device follow a certain topology. Since the 2-qubit gates such as CNOT or CX can only be performed between two adjacent physical qubits, a series of SWAP operations can be required to move the relevant qubits until they become directly-connected. This is an important issue in machine-specific mapping and optimization, implying a significant potential overhead. Consequently, we also list the number of CNOT gates in the tables.
Small-scale
Qunatum circuits using 2 to 5 qubits.
| Benchmark | Description | Algorithm | Qubits | Gates | CNOT | Reference |
|---|---|---|---|---|---|---|
| wstate | W-state preparation and assessment | Logical Operation | 3 | 30 | 9 | OpenQASM |
| adder | Quantum ripple-carry adder | Quantum Arithmetic | 4 | 23 | 10 | Scaffold |
| basis_change | Transform the single-particle baseis of an linearly connected electronic structure | Quantum Simulation | 3 | 53 | 10 | OpenFermion |
| basis_trotter | Implement Trotter steps for molecule LiH at equilibrium geometry | Quantum Simulation | 4 | 1626 | 582 | OpenFermion |
| cat_state | Coherent superposition of two coherent states with opposite phase | Logical Operation | 4 | 4 | 3 | Scaffold |
| deutsch | Deutsch algorithm with 2 qubits for f(x) = x | Hidden Subgroup | 2 | 5 | 1 | OpenQASM |
| error_correctiond3 | Error correction with distance 3 and 5 qubits | Error Correction | 5 | 114 | 49 | Ref |
| fredkin | Controlled-swap gate | Logical Operation | 3 | 19 | 8 | Scaffold |
| grover | Grover’s algorithm | Search and Optimization | 2 | 16 | 2 | AgentANAKIN |
| hs4 | Hidden subgroup problem | Hidden Subgroup | 4 | 28 | 4 | Scaffold |
| inverseqft | Performs an exact inversion of quantum Fourier tranform | Hidden Subgroup | 4 | 8 | 0 | OpenQASM |
| ipea | Iterative phase estimation algorithm | Hidden Subgroup | 2 | 68 | 30 | OpenQASM |
| iswap | An entangling swapping gate | Logical Operation | 2 | 9 | 2 | OpenQASM |
| linearsolver | Solver for a linear equation of one qubit | Linear Equation | 3 | 19 | 4 | Ref |
| lpn | Learning parity with noise | Machine Learning | 5 | 11 | 2 | sampaio96 |
| pea | Phase estimation algorithm | Hidden Subgroup | 5 | 98 | 42 | OpenQASM |
| qec_sm | Repetition code syndrome measurement | Error Correction | 5 | 5 | 4 | OpenQASM |
| qft | Quantum Fourier transform | Hidden Subgroupe | 4 | 36 | 12 | OpenQASM |
| qec_en | Quantum repetition code encoder | Error Correction | 5 | 25 | 10 | sampaio96 |
| teleportation | Quantum teleportation | Quantum Communication | 3 | 8 | 2 | Ref |
| toffoli | Toffoli gate | Logical Operation | 3 | 18 | 6 | Scaffold |
| variational | Variational ansatz for a Jellium Hamiltonian with a linear-swap network | Quantum Simulation | 4 | 54 | 16 | OpenFermion |
| vqe_uccsd | Variational quantum eigensolver with UCCSD | Linear Equation | 4 | 220 | 88 | Scaffold |
| shor | Shor’s algorithm | Hidden Subgroup | 5 | 64 | 30 | Qiskit |
| bell | Circuit equivalent to Bell inequality test | Logic Operation | 4 | 33 | 7 | Cirq |
| qrng | Quantum random number generator | Quantum Arithmetic | 4 | 4 | 0 | Paper, Repo |
| qaoa | Quantum approximate optimization algorithm | Search and Optimization | 3 | 15 | 6 | Repo |
| quantumwalks | Quantum walks on graphs with up to 4 nodes | Quantum Walk | 2 | 11 | 3 | Repo |
| dnn | 3 layer quantum neural network sample | Machine Learning | 2 | 226 | 42 | Ref |
Medium-scale
Quantum circutis using 6 to 15 qubits.
| Benchmark | Description | Algorithm | Qubits | Gates | CNOT | Reference |
|---|---|---|---|---|---|---|
| adder | Quantum ripple-carry adder | Quantum Arithmetic | 10 | 142 | 65 | OpenQASM |
| bv | Bernstein-Vazirani algorithm | Hidden Subgroup | 14 | 41 | 13 | OpenQASM |
| cc | Counterfeit coin finding problem | Search and Optimization | 12 | 22 | 11 | OpenQASM |
| ising | Ising model simulation via QC | Quantum Simulation | 10 | 480 | 90 | Scaffold |
| multiply | Performing 3×5 in a quantum circuit | Quantum Arithmetic | 13 | 98 | 40 | AgentANAKIN |
| qf21 | Using quantum phase estimation to factor the number 21 | Hidden Subgroup | 15 | 311 | 115 | AgentANAKIN |
| qft | Quantum Fourier transform | Hidden Subgroup | 15 | 540 | 210 | OpenQASM |
| qpe | Quantum phase estimation algorithm | Hidden Subgroup | 9 | 123 | 43 | AgentANAKIN |
| sat | Boolean satisfiability problem | Search and Optimization | 11 | 679 | 252 | OpenQASM |
| seca | Shor's error correction algorithm for teleportation | Error Correction | 11 | 216 | 84 | AgentANAKIN |
| simons | Simon’s algorithm | Hidden Subgroup | 6 | 44 | 14 | AgentANAKIN |
| vqe_uccsd | Variational quantum eigensolver with UCCSD | Linear Equation | 6 | 2282 | 1052 | Scaffold |
| vqe_uccsd | Variational quantum eigensolver with UCCSD | Linear Equation | 8 | 10808 | 5488 | Scaffold |
| qaoa | Quantum approximate optimization algorithm | Search and Optimization | 6 | 270 | 54 | Cirq |
| bb84 | A quantum key distribution circuit | Quantum Communication | 8 | 27 | 0 | Cirq |
| multiplier | Quantum multiplier | Quantum Arithmetic | 15 | 574 | 246 | Cirq |
| dnn | 16-dimension quantum neural network sample | Machine Learning | 8 | 1008 | 192 | Ref |
Large-scale
Quantum circuits using more than 15 qubits.
| Benchmark | Description | Algorithm | Qubits | Gates | CNOT | Reference |
|---|---|---|---|---|---|---|
| dnn | quantum neural network sample | Machine Learning | 16 | 2016 | 384 | Ref |
| bigadder | Quantum ripple-carry adder | Quantum Arithmetic | 18 | 284 | 130 | OpenQASM |
| cc | Counterfeit coin finding problem via QC | Hidden Subgroup | 18 | 34 | 17 | OpenQASM |
| bv | Bernstein-Vazirani algorithm | Hidden Subgroup | 19 | 56 | 18 | OpenQASM |
| qft | Quantum Fourier tranform | Hidden Subgroup | 20 | 970 | 380 | OpenQASM |
| bwt | Binary Welded Tree: a quantum walk algorithm in continuous time domain | Quantum Walk | 21 | 462001 | 174800 | QASMBench |
| cat_state | Coherent superposition of two coherent states with opposite phase | Logical Operation | 22 | 22 | 21 | QASMBench |
| ghz_state | Greenberger-Horne-Zeilinger (GHZ) state for max entanglement | Logical Operation | 23 | 23 | 22 | QASMBench |
| ising | Ising model simulation via QC | Quantum Simulation | 26 | 280 | 50 | QASMBench |
| multiplier | Quantum multiplier | Quantum Arithmetic | 25 | 1743 | 750 | Cirq |
| square_root | Computing the square root of an number via amplitude amplification | Quantum Arithmetic | 18 | 2300 | 898 | Scaffold |
| swap_test | Swap test to measure quantum state distance | Machine Learning | 25 | 230 | 96 | QASMBench |
| vqe | Variational quantum eigensolver with UCCSD | Quantum Simulation | 24 | 2306072 | 1538240 | QASMBench |
| ising | Ising model simulation via QC | Quantum Simulation | 500 | 5494 | 998 | Scaffold |
| ising | Ising model simulation via QC | Quantum Simulation | 1000 | 10994 | 1998 | Scaffold |
| class_number | Compute the class group of a real quadratic number field | Hidden Subgroups | 60052 | 31110504 | 12460637 | Scaffold |
qelib1.inc
OpenQASM header file that defines all the gates. Please see OpenQASM and our paper for details.
QASMBenchmark Suite Structure
Each benchmark folder include the following file:
- bench.qasm: OpenQASM source file.
- bench.png: Visualization of the circuit from IBM QE.
- res_bench.png: Running results from IBM QE quantum backends (mainly 5-qubit Burlington, 15-qubit Melbourne, and 27-qubit Paris).
- bench.cuh: Source file for our DM-Sim quantum simulator.
Tests
The small-scale benchmarks (except basis-trotter) can be directly uploaded and verified on real Quantum Machines IBM Quantum Experience.
The medium-scale benchmarks can be either validated by real quantum machines or simulators in IBM Quantum Experience.
Some of the large-scale benchmarks can be validated on IBM simulators.
DM-Sim simulation
You may also want to use our density-matrix quantum circuit simulator DM-Sim for simulating the QASMBench benchmark circuits efficiently on modern CPU (Intel X86, AMD X86, IBM Power), GPU (NVIDIA GPU and AMD GPU) and Xeon-Phi workstations or clusters ORNL Summit, ANL Theta, and NERSC Cori Supercomputers.
Authors
Ang Li, Pacific Northwest National Laboratory (PNNL)
Sriram Krishnamoorthy, Pacific Northwest National Laboratory (PNNL)
And also the original authors that developed these quantum routines.
Citation format
For research articles, please cite our paper:
- Ang Li, Sriram Krishnamoorthy, "QASMBench: A Low-level QASM Benchmark Suite for NISQ Evaluation and Simulation" [arXiv:2005.13018].
Bibtex:
@article{li2020qasmbench,
title={QASMBench: A Low-level QASM Benchmark Suite for NISQ Evaluation and Simulation},
author={Li, Ang and Krishnamoorthy, Sriram},
journal={arXiv preprint arXiv:2005.13018},
year={2020}
}
License
This project is licensed under the BSD License, see LICENSE file for details.
Acknowledgments
We thank the many developers and open-source community for providing these awesome quantum circuits online so we are able to collect and form this benchmark suite. This work was originally supported by PNNL's Quantum Algorithms, Software, and Architectures (QUASAR) LDRD Initiative. It is now supported by U.S. DOE Co-design Center for Quantum Advantage (C2QA) Quantum Information Science (QIS) center XCITe crosscut. The Pacific Northwest National Laboratory (PNNL) is operated by Battelle for the U.S. Department of Energy (DOE) under contract DE-AC05-76RL01830.
Contributing
Please contact us If you'd like to add your circuits into the benchmark suite or you'd like to remove your circuits from the suite.