Quantum Benchmarking Initiative Aims for Commercial Viability

Measurement scientists are making significant strides in understanding the performance of quantum computers as they seek to establish a common framework for benchmarking. The effort involves leading technology companies like Google, Amazon, IBM, and Microsoft, alongside a range of innovative start-ups. These entities aim to transition quantum computing from a research-focused field to a mainstream commercial opportunity that can solve complex scientific and industrial challenges.

As advancements in quantum technology accelerate, government laboratories and academic institutions are urged to maintain their focus on foundational research. This includes developing essential quantum hardware and software, supported by theoretical frameworks and experimental systems. The goal is to progress along multiple research and development pathways simultaneously.

Understanding Quantum Performance Metrics

A crucial part of this initiative is to evaluate and compare the performance of quantum computers across various platforms, including superconducting circuits, trapped ions, and photonic processors. The National Quantum Computing Centre (NQCC) in the UK has funded a significant research collaboration aimed at creating a comprehensive taxonomy of metrics and benchmarks. Over the past four years, this initiative has involved collecting and organizing data to assess quantum computers against classical systems and each other.

Led by the National Physical Laboratory (NPL), the UK’s National Measurement Institute, the consortium is tackling a complex challenge due to the diversity of quantum hardware. The collaboration recognizes two distinct approaches to quantum computing: a gate-based framework for universal computation and an analogue method designed for specific tasks.

Ivan Rungger, a principal scientist at NPL and project lead, emphasizes the importance of establishing transparent and objective benchmarks. These benchmarks are vital for fostering trust and comparability among quantum technologies, aligning with NPL’s mission in quantum metrology and standards.

Challenges in Quantum Benchmarking

Performance metrics traditionally used for classical computers, such as operational speed and error rates, have limited applicability to quantum systems. In recent years, a variety of dedicated metrics have emerged to evaluate quantum performance, from hardware components to complete applications. The challenge lies in navigating extensive literature and the varying levels of maturity among these metrics.

Rungger notes that while many approaches yield similar qualitative performance assessments, differences in technical implementation complicate quantitative comparisons. This inconsistency can hinder progress toward achieving quantum advantage. Therefore, the goal is to streamline the metrics into a minimal, representative set that manufacturers, algorithm developers, and end-users can agree upon.

The rigorous review conducted by Rungger and his colleagues has resulted in a detailed collection of metrics. Each metric includes a clear definition, methodology, assumptions, limitations, and a linked open-source software package. This software will facilitate reproducible evaluations of quantum performance.

As the research on metrics continues, Rungger anticipates that the repository will evolve into a living resource, updated regularly to reflect community-driven advancements in the field.

Towards Standardization and Collaboration

The push for objective performance benchmarks has prompted international standards bodies to begin work in areas ripe for standardization. This will equip manufacturers, end-users, and investors with reliable evaluations of various quantum components and systems. The UK’s leadership in metrics and benchmarking is already influencing global standards development.

Deep Lall, a quantum scientist at NPL and lead author of the benchmarking study, highlights the international interest in their work. He has presented the findings at several global standardization meetings, fostering engagement with researchers and industry stakeholders worldwide. Lall asserts that the UK’s efforts in benchmarking will help shape the broader international initiative, enhancing the development of standardized benchmarks for quantum computers.

Cyrus Larijani, head of the quantum program at NPL, underscores the increasing importance of evidence-based decision-making as the field evolves. By grounding strategic choices in robust measurement science and real-world data, the aim is to ensure that innovations in quantum technology yield meaningful impacts across various industries and society at large.

The strides being made in quantum benchmarking reflect a collective effort to understand and improve the performance of quantum computers. As this field continues to grow, the establishment of standard metrics will play a pivotal role in the journey toward practical quantum advantage.