Numerical Reproducibility for the Parallel Reduction on Multi- and Many-Core Architectures

Sylvain Collange 1 David Defour 2 Stef Graillat 3 Roman Iakymchuk 3
1 PACAP - Pushing Architecture and Compilation for Application Performance
Inria Rennes – Bretagne Atlantique , IRISA_D3 - ARCHITECTURE
2 DALI - Digits, Architectures et Logiciels Informatiques
LIRMM - Laboratoire d'Informatique de Robotique et de Microélectronique de Montpellier, UPVD - Université de Perpignan Via Domitia
3 PEQUAN - Performance et Qualité des Algorithmes Numériques
LIP6 - Laboratoire d'Informatique de Paris 6
Abstract : On modern multi-core, many-core, and heterogeneous architectures, floating-point computations, especially reductions, may become non-deterministic and, therefore, non-reproducible mainly due to the non-associativity of floating-point operations. We introduce an approach to compute the correctly rounded sums of large floating-point vectors accurately and efficiently, achieving deterministic results by construction. Our multi-level algorithm consists of two main stages: first, a filtering stage that relies on fast vectorized floating-point expansion; second, an accumulation stage based on superaccumulators in a high-radix carry-save representation. We present implementations on recent Intel desktop and server processors, Intel Xeon Phi co-processors, and both AMD and NVIDIA GPUs. We show that numerical reproducibility and bit-perfect accuracy can be achieved at no additional cost for large sums that have dynamic ranges of up to 90 orders of magnitude by leveraging arithmetic units that are left underused by standard reduction algorithms.
Complete list of metadatas

https://hal-lirmm.ccsd.cnrs.fr/lirmm-01206348
Contributor : David Defour <>
Submitted on : Thursday, June 6, 2019 - 1:01:20 PM
Last modification on : Wednesday, June 12, 2019 - 1:33:07 AM

File

superaccumulator.pdf
Files produced by the author(s)

Identifiers

Citation

Sylvain Collange, David Defour, Stef Graillat, Roman Iakymchuk. Numerical Reproducibility for the Parallel Reduction on Multi- and Many-Core Architectures. Parallel Computing, Elsevier, 2015, 49, pp.83-97. ⟨10.1016/j.parco.2015.09.001⟩. ⟨lirmm-01206348⟩

Share

Metrics

Record views

648

Files downloads

13