4.6 Article

STOCHASTIC ROUNDING AND ITS PROBABILISTIC BACKWARD ERROR ANALYSIS

期刊

SIAM JOURNAL ON SCIENTIFIC COMPUTING
卷 43, 期 1, 页码 A566-A585

出版社

SIAM PUBLICATIONS
DOI: 10.1137/20M1334796

关键词

floating-point arithmetic; rounding error analysis; numerical linear algebra; stochastic rounding; round to nearest; probabilistic backward error analysis; stagnation

资金

  1. Engineering and Physical Sciences Research Council [EP/P020720/1]
  2. Royal Society
  3. EPSRC [EP/P020720/1] Funding Source: UKRI

向作者/读者索取更多资源

Stochastic rounding rounds real numbers to the nearest larger or smaller floating-point numbers based on certain probabilities, showing potential benefits in low precision computations for deep learning. It is compared with round to nearest method, revealing both similarities and significant differences in their properties. The analysis demonstrates that rounding errors in stochastic rounding are mean independent random variables, resulting in unconditionally bounded backward errors for a range of linear algebra computations.
Stochastic rounding rounds a real number to the next larger or smaller floating-point number with probabilities 1 minus the relative distances to those numbers. It is gaining attention in deep learning because it can increase the success of low precision computations. We compare basic properties of stochastic rounding with those for round to nearest, finding properties in common as well as significant differences. We prove that for stochastic rounding the rounding errors are mean independent random variables with zero mean. We derive a new version of our probabilistic error analysis theorem from [N. J. Higham and T. Mary, SIAM J. Sci. Comput., 41 (2019), pp. A2815-A2835], weakening the assumption of independence of the random variables to mean independence. These results imply that for a wide range of linear algebra computations the backward error for stochastic rounding is unconditionally bounded by a multiple of issurd nu to first order, with a certain probability, where n is the problem size and u is the unit roundoff. This is the first scenario where the rule of thumb that one can replace nu by \surd nu in a rounding error bound has been shown to hold without any additional assumptions on the rounding errors. We also explain how stochastic rounding avoids the phenomenon of stagnation in sums, whereby small addends are obliterated by round to nearest when they are too small relative to the sum.

作者

我是这篇论文的作者
点击您的名字以认领此论文并将其添加到您的个人资料中。

评论

主要评分

4.6
评分不足

次要评分

新颖性
-
重要性
-
科学严谨性
-
评价这篇论文

推荐

暂无数据
暂无数据