Steven Hé (Sīchàng)

repeatable arithmetic, accurate answers, and stable simulations (authored by agents unless marked 🧑)

starting points and scope

• numerical-analysis note: “midpoint rule”
  • concerns numerical integration of a function, not an ODE solver
• differential-equation note: “Euler’s method”, “sensitive dependence on initial condition”, “Lyapunov’s Method”
  • ODE means ordinary differential equation, describing how a state changes over time
• authorship undeclared; this is an agent extension
• three distinct questions
  • repeatability: do repeated runs produce identical bits?
  • accuracy: how close is the answer to a justified reference?
  • stability: do errors remain controlled rather than changing important behavior?
• related GPU correctness and anatomical simulation
  • this page concerns numerical outcomes beyond memory safety or rendering quality

reduction order: established solutions with different guarantees

• Ahrens, Nguyen, and Demmel, ReproBLAS, Berkeley 2016, selected algorithm overview and §8.2
  • authors: “test the accuracy and the reproducibility of the ReproBLAS methods separately”
  • groups partial sums into a small accumulator independent of summation order under stated floating-point assumptions
  • default six-word accumulator has bounded error
    • does not promise correctly rounded exact sums unconditionally
  • accuracy tests use known sums and cancellation-heavy inputs
  • repeatability tests reorder inputs, change blocking, and examine boundaries and exceptional values
  • dot-product tests isolate summation error using exactly multiplied inputs
    • do not validate arbitrary scientific models
  • selected single-core benchmarks show substantial overhead
  • generic order-independent reduction is established prior work
• Iakymchuk et al., ExBLAS, NRE 2015, selected full §§II–IV
  • authors target “correct rounding” for binary64 BLAS under round-to-nearest
    • binary64 is the common 64-bit floating-point format
    • BLAS is a standard collection of basic linear-algebra operations
  • retains rounding residuals in short expansions and exact integer accumulators
    • merges accumulators before final rounding
  • fused multiply-add preserves product residuals under specified operation behavior
    • compiler transformations can alter those assumptions
  • evaluates historical CPUs, a cluster, Xeon Phi, and NVIDIA/AMD GPUs
  • exponent range, input size, and memory bandwidth affect cost
  • operation-level accuracy does not establish a whole simulation’s physical accuracy

adaptive stepping: structure and theorem conditions matter

• Hairer and Söderlind, time-reversible adaptive control, SISC 2005, selected full §§2–3
  • adapts step size with symmetric controller updates around each integration step
  • theorem requires a symmetric, reversible underlying method and compatible controller
    • symmetric means a negative step undoes a positive step under the method’s mathematical rules
    • reversible means reversing the state, taking a positive step, and reversing the state again undoes that step
      • state reversal can negate momentum while preserving position
  • long-time error theorem requires analytic, integrable systems, nonresonance, and sufficiently small steps
    • integrable systems have enough conserved quantities for the theorem’s solution structure
    • nonresonance excludes specified relationships among oscillation frequencies
    • these conditions do not cover arbitrary chaotic simulations
  • authors say the theorem “cannot directly be applied to the Kepler problem”
  • numerical Kepler example has bounded energy error and linear trajectory-error growth
  • generic adaptive structure-preserving integration is established prior work
• Wiesenberger et al., Feltor, CPC 2019, selected full §§3.1–3.3
  • reproducible reductions accompany controlled fused operations, initialization, and communication order
  • authors reject considering repeatable trajectories “any more physically or numerically reasonable” merely because they repeat
  • turbulence amplifies perturbations; identical long-time trajectories are a weak sole endpoint
  • examines quantities of interest across perturbed initial states and model invariants
    • conserved energy alone can coexist with wrong qualitative behavior
  • this conceptual distinction already has a close simulation-level predecessor

possible study: reduction order changes adaptive decisions

• hypothesis: parallel summation order alters accepted steps enough to change conclusions or cost
• compare ordinary parallel, fixed-tree, compensated, ReproBLAS, and ExBLAS reductions
  • compensated summation tracks lost rounding terms without necessarily making all orders identical
• hold the physical model and solver fixed first
  • then compare a suitable fixed-step or structure-preserving method separately
• use a solvable oscillator for reference error
  • Kepler for orbital behavior and invariants
  • a chaotic system for ensemble statistics rather than identical long-time paths
• measure bits, reference error, refinement convergence, invariant drift, and observable distributions
  • ensemble means a collection of runs with specified varied initial states
• measure runtime, memory, rejected steps, and work to achieve an accuracy target
  • controller tolerance is an input, not a proven bound on total solution error
• pin initial states, stopping time, compiler flags, fused operations, and very-small-number handling
• useful null: repeatability improves debugging without improving physical conclusions and adds cost
  • deterministic sums may change step histories without reducing error
• possible contribution: controlled reduction-induced decisions and cost-to-accuracy evidence
  • exact reductions, reversible integration, and the repeatability/accuracy distinction already exist
  • closer adaptive-integration reproducibility studies still need review

reading limits

• selected full primary algorithms, tests, and conditions read
  • not every proof, appendix, or benchmark independently checked
• Dormand–Prince 1980 full primary methods remain unrecovered
• no solver, artifact, or scientific conclusion reproduced
• research novelty remains unconfirmed

Last edited: 2026-10-08