One of the main problems in implementing the new generation of quantum computers lies in their most basic customer: qubit . Qubits can interact with any objects in the immediate vicinity that transfer energy close to their own wandering photons (i.e. unwanted electromagnetic fields, phonons (mechanical vibrations of a quantum device) or quantum defects (irregularities on the chip surface that appeared during the manufacturing stage), which can unpredictably change the state of qubits on their own.
The matter is complicated by many tasks that put the tools used to control qubits. Qubits are processed and read by classical methods: analog signals in the form of electromagnetic fields, coupled with a physical board in which a qubit is integrated, for example, a superconducting microcircuit. Imperfections in control electronics (leading to white noise), interference from external radiation sources and fluctuations in digital-to-analog converters lead to even greater stochastic errors that worsen the operation of quantum microcircuits. These practical issues affect the accuracy of calculations and, thus, limit the application of the upcoming generation of quantum devices.
To increase the computing power of quantum computers and open the way to large-scale quantum computing, it is first necessary to build physical models that accurately describe these experimental problems.
In the article “Universal Quantum Control through Deep Reinforcement Learning” , published in the Nature Partner Journal (npj) Quantum Information (https://www.nature.com/npjqi/articles), we introduced a new structure of quantum control created using deep learning with reinforcement in which the practical problems of optimizing quantum control can be encapsulated with a single loss function. The structure under consideration provides a reduction in the average error of a quantum gate to two orders of magnitude compared to standard stochastic solutions of gradient descent and a significant reduction in the gate time to optimal values of gate synthesis analogs. Our results open up new horizons for quantum modeling, quantum chemistry and quantum excellence tests using quantum devices in the near future.
The innovation of this quantum control paradigm is based on the development of a quantum control function and an effective optimization method based on deep learning with reinforcement. In order to develop a comprehensive loss function, we first need to develop a physical model of a realistic quantum control process in which we can accurately predict the magnitude of the error. One of the most annoying mistakes in evaluating the accuracy of quantum computing is leakage: the amount of quantum information lost during the calculation. Such a leak usually occurs when the quantum state of a qubit changes to a higher energy level or to a lower due to spontaneous emission. Due to a leak-error, not only useful quantum information is lost, it also degrades “quantumness” and ultimately reduces the performance of a quantum computer to the performance of a computer with a classical architecture.
A common practice for accurately estimating lost information during a quantum computation is to model the entire computation first. However, this negates the whole point of creating large-scale quantum computers, since their advantage is that they are able to perform calculations impossible for classical computers. With the improvement of physical modeling, our common loss function allows us to jointly optimize accumulated leakage errors, violations of boundary control conditions, total valve time and valve accuracy.
With the new loss management function, the next step is to use an effective optimization tool to minimize it. Existing optimization methods are not good enough to search for high-precision solutions that are reliable for controlling fluctuations. Instead, we use the method based on the on-policy method of deep learning with reinforcement (RL), RL - a trusted area . Since this method demonstrates good performance in all test tasks, it is inherently resistant to sample noise and can optimize complex control problems with hundreds of millions of control parameters. A significant difference between this on-policy RL method and the previously studied off-policy RL methods is that the management policy appears to be independent of loss management. On the other hand, all RL policies, such as Q-learning , use a single neural network to represent the control path and the associated reward, where the control trajectory determines the control signals that should be associated with the qubits on different measures, and the associated reward measures how good the tact is quantum control.
On-policy RL is well known for its ability to use non-local features in control paths, which becomes critical when the control landscape is multidimensional and packed with a combinatorially large number of non-global solutions, as is often the case with quantum systems.
We encode the control path into a three-layer, fully connected neural network - NN policy, and the control loss function into the second neural network - NN value, which reflects the discounted future award. Reliable control solutions were obtained with reinforcement learning agents that train both neural networks in a stochastic environment that simulates realistic noise control. We offer a solution for controlling a set of continuously parameterized two-qubit quantum gates, which are of particular importance in application to quantum chemistry, but are too expensive to implement using a standard universal set of gates.
In the framework of this new structure, our numerical simulation shows a hundred-fold decrease in the errors of quantum gates and a reduction in the time of gates for the family of continuously parameterized simulation quantum gates by an average of one order of magnitude compared to traditional approaches using a universal set of gates.
This work emphasizes the importance of using new machine learning methods and the latest quantum algorithms that use the flexibility and additional processing power of a universal quantum control circuit. In order to fully integrate machine learning and increase computational capabilities, it is necessary to conduct additional experiments, similar to what was given in this work.