How to evaluate a quantum computing pilot proposal
On this page
Sooner or later, a proposal reaches a bank's steering committee: a quantum computing pilot for portfolio optimisation, risk simulation or fraud detection, often with a vendor attached and an impressive slide about speed-ups. This piece is for the people asked to approve, fund, govern or audit that pilot. It doesn't argue for or against quantum pilots. Some are worth doing. It sets out what's realistic today, the questions that separate a sound proposal from a marketing exercise, and what a well-run pilot looks like.
One distinction first. A pilot like this is about using quantum computers. It is a separate matter from post-quantum cryptography, the defensive migration that protects the bank from quantum attacks. The two need different owners, budgets and levels of urgency, and approving one says nothing about the other.
What's realistic today
For the problems banks are usually pitched, the published evidence is cautious:
- Optimisation. A 2024 review by a large international working group on quantum optimisation concludes that more tools are needed to reach a quantum advantage in optimisation, and calls for systematic, reproducible benchmarks against classical methods. Classical solvers can already find near-optimal answers to many large problems in reasonable time.
- Quantum annealers. A review of annealing for industry found no industrial application where annealing unquestionably outperforms classical methods. A later independent comparison found one vendor's hybrid annealing service competitive on a narrow class of problems, but with no advantage over leading commercial solvers on the others it tested.
- Machine learning and finance. As What quantum computers are good at explains, getting data into a quantum computer and answers out of it often erodes the theoretical speed-up, and clear advantages over the best ordinary methods aren't established.
None of that rules out a pilot. It means the honest purpose of most pilots today is learning: building skills, understanding the tools, and being ready if the hardware improves. A proposal that says so is easier to approve and govern than one that promises a near-term competitive edge.
A real example
In September 2025, HSBC and IBM reported a trial on predicting whether corporate bond trades would be filled at a quoted price. Their paper reports a relative gain of up to about 34% in out-of-sample test scores for models given data transformed on IBM's Heron quantum processor, compared with the same models using the original data or a noise-free simulation of the quantum step. That is a gain in a model's test score, not 34% more trades filled or 34% more profit, and HSBC described it as a promising trial, not a production system. Within days, critics including the computer scientist Scott Aaronson argued that the gain came from the particular methods compared rather than from anything quantum, and a data scientist proposed that a known classical technique could explain the effect; that was a suggested explanation, not a published replication.
This piece takes no side on that result. The episode shows why the questions below matter: the same experiment can look like a breakthrough or an artefact depending on the baseline, the comparison and what's reported.
Two different kinds of machine
Proposals don't always say which kind of quantum computer they use, and the answer changes what the pilot can show.
| Quantum annealer | General-purpose (gate-based) quantum computer | |
|---|---|---|
| What it does | Searches for low-cost answers to optimisation problems written in one specific mathematical form (known as QUBO or Ising); hybrid services add classical processing to handle wider problems | Runs programs built from quantum gates, in principle any quantum algorithm |
| Examples | D-Wave's systems | Machines from IBM, Google, Quantinuum, IonQ and others |
| Typical pitch | Scheduling, routing, portfolio selection | Optimisation, simulation, machine learning |
| Can it run Shor's algorithm? | No: it implements only part of what general-purpose computation needs | In principle yes, but no machine today is anywhere near large or reliable enough |
Where the work happens in a hybrid algorithm
Most pilots on today's hardware use hybrid algorithms, in which an ordinary computer and a quantum computer work in a loop:
The ordinary computer does much of the work: it chooses the settings, scores the results and decides what to try next. That's a strength for running on today's machines, but it means an improvement can come from the classical side, from the extra data handling around the quantum step, or from the comparison chosen. A sound pilot tests this directly, by running the same pipeline with the quantum step replaced by a classical alternative.
Questions to ask before approving
- What business problem, and what's it worth if solved? A pilot without a named decision or process it would improve is research, which may be fine, but should be funded as such.
- What's the best classical method for this problem, and who ran it? The comparison should be against leading commercial or academic methods, tuned by people who know them, with the same data, the same objective and scoring, and the same time and budget. For prediction, that means testing on data the model hasn't seen and guarding against leakage; for optimisation, it means comparing answer quality, whether constraints are met, run time and total cost. A comparison against a simple or untuned method proves little.
- Which kind of machine, and why that one? An annealer and a general-purpose machine answer different questions. Ask how the choice was made and whether the vendor has a stake in it.
- What happens with the quantum step swapped out? For hybrid methods, compare the complete pipeline against a well-tuned classical alternative. Where feasible, also run the quantum part as a noise-free simulation, to help show how much of the result comes from hardware noise. That simulation is a useful diagnostic, but it doesn't replace the fair classical comparison.
- Does it get better as the problem grows? Small problems can often be solved or simulated classically. The interesting question is the trend as size increases.
- What data leaves the bank, and where does it go? Most banking quantum pilots reach the hardware through cloud services, so the provider arrangement, data classification and processing location are part of the approval decision. Third-party risk, data classification and location reviews apply, and for federally regulated institutions OSFI's Guideline B-10 on third-party risk applies, in proportion to the risk and criticality of the arrangement. Synthetic or properly anonymised data can reduce confidentiality and privacy risk, but it doesn't remove the need for supplier due diligence, contract terms, access controls and a check on where data is processed. If the pilot builds or tests a model that could influence banking decisions, involve the model risk function as well: OSFI's revised Guideline E-23 on model risk management takes effect in May 2027.
- What does the whole thing cost, end to end? Compare full cost and elapsed time, including data preparation, classical processing, quantum runs and repeated trials, and the time to interpret results, not just the speed of the quantum step.
- What counts as success, and what happens if it isn't reached? Success criteria, a fixed time box and an exit decision should be agreed before the work starts, not after the results are in.
- Who can publicise the results? Vendors value bank case studies. Agree in the contract what can be announced, by whom, and after what review.
What a well-run pilot looks like
The strongest proposals share a few features. They state their purpose honestly, usually learning and readiness rather than advantage. They define success criteria and a classical baseline before starting, ideally with the baseline run by a team that doesn't benefit from the quantum result. They use the least sensitive data that will do the job, and put the provider through a third-party risk assessment proportionate to what the pilot involves. They're time-boxed, with a decision point at the end. And they report what didn't work as clearly as what did, because a well-documented negative result is still worth paying for.
- Published reviews have not established a quantum advantage for the optimisation and finance problems banks are usually pitched.
- Judge a pilot by its classical baseline and by what happens when the quantum step is swapped out.
- A quantum pilot is a supplier and data arrangement, and is separate from post-quantum migration.
Sources and further reading
Tags show what kind of source each one is. A standard or government guidance is an official document; a peer-reviewed paper has been checked by other experts; a preprint has not been peer-reviewed yet; an experiment reports a real-world demonstration; a company announcement is the company's own account; vendor documentation is a supplier's own technical guide; expert commentary is a specialist's published analysis or opinion; a news report is a journalist's account. Dates and figures were checked against these sources on 17 October 2026. Spotted an error? Email hello@plainquantum.com and it will be corrected, with a note.
- Peer-reviewed paperChallenges and opportunities in quantum optimization (Quantum Optimization Working Group)Abbas et al., Nature Reviews Physics, 2024
- Peer-reviewed paperQuantum annealing for industry applications: introduction and reviewYarkoni, Raponi, Bäck and Schmitt, Reports on Progress in Physics, 2022
- Peer-reviewed paperQuantum annealing applications, challenges and limitations for optimisation problems compared to classical solversQuinton et al., Scientific Reports, 2025
- Company announcementHSBC demonstrates world's first-known quantum-enabled algorithmic trading with IBMHSBC media release, 2025
- PreprintEnhanced fill probability estimates in institutional algorithmic bond trading using statistical learning algorithms with quantum computersCiceri et al. (HSBC, IBM), arXiv, 2025
- News reportQuantum computing 'breakthrough' has 'more red flags than a People's Liberation Army parade'Dan Barnes, The DESK, 2025
- Government guidanceGuideline B-10: Third-Party Risk ManagementOffice of the Superintendent of Financial Institutions (OSFI), 2023
- Expert commentaryRead the fine printScott Aaronson, Nature Physics, 2015
- Government guidanceGuideline E-23: Model Risk Management (2027), effective 1 May 2027Office of the Superintendent of Financial Institutions (OSFI), 2025