Higher order derivatives of quantum neural networks with barren plateaus

Quantum neural networks (QNNs) offer a powerful paradigm for programming near-term quantum computers and have the potential to speed up applications ranging from data science to chemistry to materials science. However, a possible obstacle to realizing that speed-up is the barren plateau (BP) phenomenon, whereby the gradient vanishes exponentially in the system size n for certain QNN architectures. The question of whether high-order derivative information such as the Hessian could help escape a BP was recently posed in the literature. Here we show that the elements of the Hessian are exponentially suppressed in a BP, so estimating the Hessian in this situation would require a precision that scales exponentially with n. Hence, Hessian-based approaches do not circumvent the exponential scaling associated with BPs. We also show the exponential suppression of higher order derivatives. Hence, BPs will impact optimization strategies that go beyond (first-order) gradient descent. In deriving our results, we prove novel, general formulas that can be used to analytically evaluate any high-order partial derivative on quantum hardware. These formulas will likely have independent interest and use for training QNNs (outside of the context of BPs).

Automated summary

The phenomenon of barren plateaus in quantum neural networks causes exponential suppression of elements in the Hessian matrix, making estimation at this situation require exponential precision with system size n. This indicates that Hessian-based approaches do not overcome the exponential scaling associated with barren plateaus. Additionally, higher order derivatives are also exponentially suppressed, impacting optimization strategies beyond first-order gradient descent.