Binary Elementwise Performance Follow-up#

Date:

2026-08

Updated:

2026-10-23

complete

Objective#

The functional Binary roadmap is complete: all 19 registered operators share one prepared broadcast engine, execute through the session-owned executor, and have reproducible correctness and benchmark corpora. This follow-up addresses the performance gaps exposed by the complete end-to-end benchmark rather than adding more operator semantics.

The final gate is:

  • at least 1.0x ONNX Runtime median performance for every priority operator/type/loop-family group;

  • no priority case below 0.9x;

  • no small-tensor p90 regression greater than 2% against the current serial SIMD baseline;

  • no private scheduler, inference-time tuning, or registry access in the hot path.

Measured baseline#

The first tuning pass removed the fixed four-participant ceiling, increased executor granularity, and added typed bulk loops for integer and half-precision Add, Sub, and Mul. On the measured Sub corpus:

  • median speed-up over ONNX Runtime increased from 0.155x to 1.533x;

  • median onnx-light-cpu execution time improved by 3.0x;

  • the fraction of cases faster than ONNX Runtime increased from 6.0% to 51.8%;

  • the median n=4096 and n=65536 cases reached 2.36x and 1.93x;

  • the median n=1048576 and n=4194304 cases remained at only 0.39x and 0.56x.

These numbers are diagnostic, not a final parity claim. They cover one operator on one host, and ONNX Runtime timing dispersion was significant for some large broadcast cases. The complete unfiltered corpus must be rerun with raw samples, affinity, effective thread count, and selected tuning parameters recorded before accepting any optimization.

FP32 fixed-exponent broadcast follow-up#

The FP32 Pow scalar-exponent adapter already recognized exponents 2 through 5, but its per-element finite/underflow checks prevented the repeated-multiplication loop from vectorizing. It now uses an AVX2 fixed-exponent loop with vector validity masks and scalar std::pow repair only for exceptional lanes. The multiplication order, signed zeros, finite-boundary results, tails, and exact in-place operation match the existing scalar adapter. Integer-typed exponents, fractional exponents, and broadcast scheduling are unchanged.

Registered-runtime measurements on an Intel Xeon Platinum 8480C, with 96 configured threads and main 1fc3359 as the baseline, reduced the 1,048,576-element float32xfloat32 per-channel case from 0.000204432 s to 0.000097377 s (2.10x) and the swapped outer-broadcast case from 0.000206468 s to 0.000073306 s (2.82x). These are before/after gains, not ONNX Runtime ratios. Routing through the general AVX-512 Pow kernel and a two-pass scalar validity scan were both rejected after end-to-end measurements showed no improvement.

python -m onnx_light_cpu benchmark --dtype float32 --onnxruntime \
    --test '^test_cpu_pow_v15_(outer_float32xfloat32_to_float32_swapped|per_channel_float32xfloat32_to_float32)_n1048576_benchmark$' \
    --threads 96 -r 100 -w 30 -t 0.5 -o pow-broadcast.xlsx

FP32 GreaterOrEqual AVX2 follow-up (#996)#

The FP32 adapter previously used the baseline-ISA compiler-generated bulk loop. GCC 13.3 generated SSE comparisons with a long unpack sequence to produce boolean bytes. The new path reuses the existing AVX2 Greater packing loop: four eight-float comparisons, saturating packs, a lane-order permutation, and one 32-byte store. Ordered >= preserves NaN semantics; both scalar-broadcast directions broadcast once per range. Runtime scheduling, other types, general broadcasting, and the portable fallback are unchanged. Dispatch assertions and disassembly confirmed the AVX2 path, including vcmpge_oqps and vpermd. Hardware-counter profiling was unavailable (perf_event_paranoid=4); this is disassembly and end-to-end timing evidence, not a hardware-counter attribution.

Environment: AMD EPYC 7763 VM, two physical cores / four logical CPUs, Linux x86-64, AVX2 (SIMD level 3); GCC 13.3, CMake Release (-O3 -DNDEBUG), ONNX_LIGHT_CPU_MAX_SIMD_LEVEL=AVX2. Baseline: onnx-light-cpu 0.2.0, 91ce5e37016a5db17279847b86e63b71e475e0d5 (code based on main b3fc52ef3c52f0378b56664e4c11c02837c07e2c). Both measurements link the onnx-light 0.2.0 release wheel (tag commit 6400f0f812b13bc32469e405bd2f762ba3cba7f0), with Python 3.13 and matching non-stable nanobind ABI. ORT is the CPU wheel 1.31.0, Release, build commit 402979a. This differs from the issue’s global onnx-light 0.2.1 metadata.

The original named backend cases and inputs were used without changes. Every run used 200 warm-ups and 1,000 measured calls per backend (5 s cap, not reached), three trials, native tensor feeds for CPU and NumPy feeds for ORT. Both backends had the same explicit thread limit, no worker pinning, and unchanged default spin settings: adaptive for onnx-light, enabled for ORT. ORT used sequential graph execution and one inter-op thread. One-thread processes were restricted to CPU 0; two-thread processes to CPUs 0-3.

The existing benchmark CLI, which times CPU before creating the ORT session, did not consistently reproduce the published ORT advantage. Its median before/after CPU gains were 1.29x / 1.31x for contiguous / swapped contiguous and 2.01x for scalar-left at one thread; 1.16x / 1.08x / 1.76x at two threads. For example (the CLI dtype filter selects the boolean output):

taskset -c 0 python -m onnx_light_cpu benchmark --threads 1 \
    --tests '^test_cpu_greaterorequal_v16_(contiguous_float32xfloat32_to_bool(_swapped)?_n4194304|right_scalar_float32xfloat32_to_bool_swapped_n1048576)_benchmark$' \
    --dtypes bool -r 1000 -w 200 -t 5 --onnxruntime -o /tmp/greaterorequal.xlsx

An additional comparison used measure_alternating from tools/benchmark_binary_parity.py with those same warm-up/repetition limits, sessions and feeds. Both sessions were created before timing, and backend order reversed every iteration. Baseline dispatch was restored for the before measurement, then the AVX2 registration was re-enabled for the after measurement. With both pools active, the two-thread baseline showed an ORT advantage in all three trials for all three cases.

Below are medians of the three trial medians, in milliseconds. All case names have prefix test_cpu_greaterorequal_v16_ and suffix _benchmark:

  • A: contiguous_float32xfloat32_to_bool_n4194304.

  • B: right_scalar_float32xfloat32_to_bool_swapped_n1048576 (scalar left).

  • C: contiguous_float32xfloat32_to_bool_swapped_n4194304.

Alternating backend measurements (ms)#

Case / threads

CPU before

CPU after

ORT before

ORT after

A / 1

1.362993

1.138686

1.384113

1.385538

B / 1

0.157285

0.078908

0.159294

0.153147

C / 1

1.341067

1.142744

1.384046

1.315162

A / 2

1.155082

0.966794

0.998498

1.008897

B / 2

0.172895

0.082750

0.118428

0.108583

C / 2

1.149933

1.055295

0.973497

1.087200

The large two-thread cases remain noisy: after-trial CPU medians ranged 0.904963-1.109556 ms for A and 1.044899-1.287495 ms for C. ORT still won one of C’s three after trials. These results support a faster implementation, not a universal parity claim or an exact reproduction of the dashboard’s latencies (whose per-case thread/spin/build settings are unknown). The 17 float32 GreaterOrEqual backend cases, including these three benchmark inputs, matched onnx-light’s built-in kernels exactly at one and two threads.

Integer arithmetic and 64-bit comparisons#

Integer Pow uses exact checked integer multiplication, with a portable fallback for compilers without overflow intrinsics. Typed contiguous and scalar-broadcast adapters share the same overflow contract. Scalar exponents 0, 1, and 2 use fill, copy, and checked-square paths, including repeated broadcast inner blocks.

Integer Div and both integer Mod modes prepare an unsigned reciprocal once per scalar-divisor block. Quotients use multiply-high and an exact remainder correction, or shifts for powers of two; signed results retain truncation toward zero and Python-modulo sign correction. Short blocks retain hardware division. Both operators validate unique divisors in bulk and retain the signed-minimum divided by minus-one check before execution.

Signed and unsigned 64-bit comparisons have baseline, AVX2, and AVX-512F implementations for contiguous and either scalar-broadcast input. AVX2 biases the unsigned sign bit before signed comparison. Output bytes are canonical booleans and short tails remain scalar.

The parity matrix includes integer Pow/Div, Mod (including int16), and all four ordered int64/uint64 comparisons across its seven broadcast families. Integer outputs are compared exactly, rather than after conversion to float32. Each integer benchmark requires a recorded implementation path; the comparison ISA comes from runtime dispatch, not host CPU flags. used_kernel_paths(session) includes records such as Binary.Pow.integer_pow.scalar_exponent, Binary.Mod.integer_divmod.invariant_divisor, and Binary.Greater.compare64.avx2. used_kernel_names still returns only operator identities.

After updating the checkout and rebuilding, run a homogeneous campaign with the existing runner (repeat with an AVX2-ceiling build on AVX-512 hosts):

PYTHONPATH="$PWD" python tools/benchmark_binary_parity.py --no-calibrate \
    --case '^test_cpu_(pow|div|mod|greater|greaterorequal|less|lessorequal)_v[0-9]+_.*_(u?int16|u?int32|u?int64)x' \
    --threads 1 --threads physical --cpus 0-3 \
    -r 100 -w 30 --output /tmp/integer-binary-parity.json

The report retains raw alternating samples, shapes, types, effective threads, implementation paths, and latency percentiles. ORT worker spinning is disabled so idle workers cannot interfere with the other candidate. A filtered campaign does not satisfy the runner’s complete-matrix gate; inspect its per-case speedup values against 0.9 and retain profiles for remaining exceptions. Integer Mod benchmark fixtures use the released opset-13 schema, whose integer semantics are unchanged in Mod-28. Correctness fixtures retain the latest opset.

2026-09-17 diagnostic campaign#

Revision 1991b23 was measured on an AMD EPYC 9V74 VM with two physical cores, affinity 0-3, GCC Release/AUTO dispatch, onnx-light 0.1.27, and ORT 1.30.0. The 420 comparable rows cover 65,536 and 1,048,576 output elements, all seven families, and one/two configured threads. Recording confirmed the integer arithmetic adapters and AVX-512 comparison kernels. There were no unsupported cases. Median speedup was 2.017x, but 44 rows remained below 0.9x: this is not a completed parity gate or a dedicated-machine result. The longer four-size campaign was stopped without publishing partial results.

PYTHONPATH="$PWD" python tools/benchmark_binary_parity.py --no-calibrate \
    --case '^test_cpu_(pow|div|mod)_v[0-9]+_.*_int(16|32|64)x.*_n(65536|1048576)_benchmark$' \
    --case '^test_cpu_(greater|greaterorequal|less|lessorequal)_v[0-9]+_.*_(int64|uint64)x.*_n(65536|1048576)_benchmark$' \
    --threads 1 --threads physical --cpus 0-3 \
    -r 30 -w 10 -t 0.05 --output /tmp/integer-binary-final.json

Each row below summarizes 28 measurements; ratios are ORT/CPU median latency.

Operator

Type

Median ratio

Minimum ratio

Below 0.9x

Div

int32

0.450

0.397

28

Div

int64

1.025

0.599

9

Mod

int16

1.041

0.878

2

Mod

int32

1.035

0.880

2

Mod

int64

1.045

0.893

2

Pow

int32

5.152

1.002

0

Pow

int64

4.940

0.551

1

Greater

int64

2.146

1.524

0

Greater

uint64

2.258

1.563

0

GreaterOrEqual

int64

2.359

1.598

0

GreaterOrEqual

uint64

2.580

1.492

0

Less

int64

2.539

1.530

0

Less

uint64

2.718

1.566

0

LessOrEqual

int64

2.324

1.573

0

LessOrEqual

uint64

2.309

1.558

0

Exception profiles and follow-up:

  • All int32 Div families remain below target. Non-scalar divisors still use hardware division. Disassembly of the invariant-divisor adapter confirms multiply/shift execution, but only baseline SSE2 compiler vectorization, including reciprocal work alongside the power-of-two shift path. Dedicated wider integer-division loops remain necessary.

  • Int64 Div exceptions are the 1,048,576-element left-scalar case with one thread; contiguous/left-scalar at that size with two threads; and every 65,536-element family except right-scalar with two threads. The default block-size policy can keep these small workloads serial despite two configured threads.

  • Mod exceptions are contiguous and left-scalar at 1,048,576 elements with one thread, for each of int16/int32/int64. These retain the divisor scan and hardware remainder loop, unlike the optimized invariant-divisor path.

  • The remaining Pow exception is int64 per-channel, 65,536 elements, two configured threads: CPU 0.000375912 s versus ORT 0.0002072725 s. It lies at the default scheduling-granularity boundary. At 1,048,576 elements the same family reaches 0.933x with two threads and 0.998x with one.

  • The first campaign exposed repeated CPUID in checked-square dispatch. Caching it reduced one-thread int64 per-channel latency at 1,048,576 elements from 0.031368454 s to 0.005959802 s. This comparison is diagnostic: the independently profiled 65,536-element case improved from 0.001325910 s to 0.000379798 s.

Call-level profiles used python -m cProfile -o /tmp/integer.pstats before the runner command, selecting the four cases below at 65,536 elements, one thread, -r 100 -w 30 -t 0.2. Of 0.074665 s in 524 CPU wrapper calls, 0.074490 s was in the native-backed evaluator call; case collection consumed 0.672408 s outside the timed region. These profiles locate the remaining cost below the Python wrapper, not within case generation. Native PMU profiling was unavailable because the runner has perf_event_paranoid=4; instruction-level hotspot attribution remains unverified.

Operator/type/family

CPU median (s)

ORT median (s)

Ratio

Div/int32/right_scalar

0.0000769795

0.0000538700

0.700

Mod/int16/right_scalar

0.0000962025

0.0001174840

1.221

Pow/int64/per_channel

0.0003797975

0.0003817955

1.005

Greater/int64/left_scalar

0.0000118175

0.0000312865

2.647

An exception-focused repeat (-r 30 -w 10 -t 0.1) reproduced the remaining groups below. The 902 CPU wrapper calls consumed 1.038556 s, including 1.038014 s in the native-backed evaluator call.

Operator/type/family

Elements

Threads

CPU median (s)

ORT median (s)

Ratio

Div/int32/left_scalar

65536

2

0.0001487200

0.0000709755

0.477

Div/int64/left_scalar

65536

2

0.0001684095

0.0001050000

0.623

Mod/int16/contiguous

1048576

1

0.0023085260

0.0020299580

0.879

Mod/int32/contiguous

1048576

1

0.0023076340

0.0020325315

0.881

Mod/int64/contiguous

1048576

1

0.0026062220

0.0023398920

0.898

Pow/int64/per_channel

65536

2

0.0003763575

0.0002096610

0.557

Current execution and tuning contract#

Every worker combines both levels of parallelism: ExecuteRanges assigns an independent output range to a session worker, and that worker invokes the bulk SIMD loop for its range. Small inputs remain serial SIMD to avoid executor overhead.

Binary kernels now register tuning ABI 2 under exact operator and left-input element-type keys with library="onnx_light_cpu" and implementation="broadcast_plan". The immutable per-session parameters are:

  • parallel.bulk_threshold_bytes (portable default: 1 MiB);

  • parallel.block_threshold_bytes (portable default: 1 MiB);

  • parallel.scalar_threshold_bytes (portable default: 256 KiB);

  • parallel.target_block_bytes (portable default: 1 MiB).

  • parallel.max_participants (portable default: 0, meaning the complete effective session executor).

The registry validates and resolves these parameters before execution. The hot path reads only the configured typed values; it performs no registry lookup, string lookup, allocation, or lock. The participant ceiling is applied to both flat and multidimensional schedules.

A targeted 1,176-case pass covering Add, Sub, Mul, Div, Mod, Pow, PRelu, Equal, Less, And and BitwiseAnd improved the aggregate median from 0.394x to 0.669x ONNX Runtime. The fraction below parity fell from 50.8% to 37.5%. Comparison and bitwise median gains in onnx-light-cpu time were 3.0x to 4.4x. These figures remain diagnostic: repeated large arithmetic runs show enough host dispersion that they do not satisfy the final gate.

The latest arithmetic follow-ups are #563 and #577. #563 unrolls AVX-512 FP32/FP64 arithmetic, adds the missing FP32 PRelu vector paths, and extends the backend benchmark coverage. #577 validates the physical integer divisor tensor once, inspecting expanded broadcast pairs only for the signed INT_MIN / -1 overflow case.

Remaining bottlenecks#

True low-precision SIMD#

FP16/BF16 Add, Sub, Mul, Div and PRelu now widen and narrow vectorized blocks through the shared half-conversion kernels for contiguous and scalar-broadcast loops. Mod and Pow use the same block-conversion approach for contiguous loops. Add dedicated AVX-512FP16, AVX-512BF16 where applicable, F16C/AVX2 conversion-vector, NEON FP16, and SVE/SVE2 implementations. Unsupported instruction sets must retain the portable bulk loop.

Integer and predicate bulk coverage#

Div, Mod, comparisons, logical operators, bitwise operators, shifts, and integer PRelu now have typed contiguous and scalar-broadcast loops for every same-type signature. Comparisons emit canonical byte BOOL. Integer Div/Mod and BitShift validate the complete input first and then enter unchecked typed compute loops. The remaining work in this area is ISA-specific predicate packing and validation bulk loops; validation still uses the prepared strided traversal.

Expensive arithmetic#

FP32 Pow now stays in FP32 rather than promoting every operation to long double, and float/integer mixed signatures have typed bulk loops. On AVX-512, exponents 0 through 5 use exact multiplications, positive finite bases with finite exponents use the shared vector Log/Exp primitives, and exceptional values fall back to std::pow. Contiguous and left-scalar loops have dedicated vector paths; small repeated blocks avoid the large-loop dispatch overhead. The approximation is checked against std::pow with a 3e-5 relative tolerance.

At 4,194,304 FP32 elements on the development host, all measured contiguous, scalar, row, per-channel, outer, and general-strided orientations reached parity with ONNX Runtime. The narrowest result was the row broadcast at 1.03x; the other broadcast families reached 1.21x to 2.85x, scalar cases 3.81x to 4.17x, and contiguous 2.56x.

General broadcast traversal#

Contiguous, scalar, repeated-block, and vector-inner families can call bulk loops. Rank 2-4 prepared plans now use fixed-size traversal state. They seed offsets once per worker range, avoid heap allocation in workers, and dispatch their typed inner bulk loop without the arbitrary-rank counter. The generic strided traversal remains the correctness fallback for higher ranks and for adapters without a compatible inner bulk loop.

On the targeted 116-case general corpus, median onnx-light-cpu time for n=1048576 fell from 2.28 ms to 0.31 ms and n=4194304 from 2.43 ms to 0.36 ms. Small cases were retained (3.83 to 3.64 us at n=4096). ONNX Runtime timings varied substantially on the largest group, so these are implementation-time comparisons rather than parity claims.

Do not add an unbounded template matrix for arbitrary ranks. Retain the current prepared general loop as the correctness fallback and specialize only patterns demonstrated by the backend corpus and real models.

Executor granularity#

The fixed four-worker cap hid dispatch overhead but prevented large tensors from scaling. Removing it improved large inputs only after each submitted range was increased to roughly 1 MiB of useful traffic. One portable block size is still unlikely to fit every processor, data type, and operation.

Calibration must jointly search crossover and block size. It should measure candidate thresholds 0, 64 KiB, 256 KiB, 1 MiB, 4 MiB and 16 MiB, while the executor derives participant count from useful blocks and the session limit. Record the actual participant count after SIMD/cache-line alignment; the requested count alone is not sufficient evidence.

Runtime and allocation overhead#

Measure both a preallocated BinaryBroadcastPlan::Execute layer and the complete ReferenceEvaluator path. If the preallocated kernel reaches parity but end-to-end execution does not, profile output allocation, tensor metadata construction, plan-cache access, and runtime dispatch separately. Do not compensate for runtime allocation overhead by over-parallelizing the compute loop.

Calibration and persistence#

Deterministic calibration callbacks are registered for every Binary tuning key. Each callback jointly evaluates seven complete profiles spanning serial, coarse, balanced, fine-grained, compute-heavy and participant-limited schedules in addition to the portable profile. Calibration uses equal-shape, scalar-broadcast and prepared rank-4 cases and:

  • creates deterministic inputs and caller-preallocated outputs;

  • compares every candidate byte-for-byte against forced-serial output;

  • records repeated samples and scores candidates relative to serial execution;

  • jointly selects all four byte parameters and the participant ceiling;

  • rejects profiles that regress small-tensor p90 by more than 2%;

  • obeys explicit duration and cumulative live-memory limits;

  • publishes atomically through onnx-light’s execution-specific registry.

Portable defaults remain available when no exact processor profile exists. Existing sessions retain their resolved immutable configuration when a later calibration publishes a new generation. Python and CLI inspection must show the exact key, source profile, four configured byte values, participant ceiling, and effective session thread count.

Benchmark and acceptance matrix#

The benchmark matrix crosses:

  • all 19 operators and every manifest signature;

  • contiguous, left/right scalar, row, per-channel, outer, and general broadcasts, including swapped non-commutative operands;

  • output sizes 4,096, 65,536, 1,048,576, and 4,194,304;

  • session limits 1, 2, 4, physical cores, and logical cores;

  • portable scalar, available SIMD levels, portable tuning defaults, and exact calibrated profiles.

Published comparisons use ONNX Runtime’s normal CPU execution provider. The tool publishes in-memory calibrated Binary profiles before creating benchmark sessions and records those calibration reports with the backend case name, operator, complete type signature, loop family, shapes, byte traffic model, SIMD level, affinity, warmups, raw samples, median, dispersion, and correctness tolerance. ONNX Runtime CPU signatures that are not implemented, currently the priority BF16 arithmetic and PRelu cases on the measured build, are reported as not comparable rather than silently dropped or substituted with FP32.

The final implementation pass also replaced callback-per-element bulk loops for logical operators and FP32 PRelu with direct vectorizable/SIMD loops. FP32 Pow scalar exponents 0 through 5 use multiplication/copy/fill kernels instead of libm. At 65,536 elements on the development host this improved:

  • logical And by 4.1x to 12.5x in onnx-light-cpu time, reaching 1.46x to 2.62x ONNX Runtime across the measured families;

  • the previously slow PRelu families by about 5x, while the explicit FP32 SIMD contiguous and scalar paths reached 4.5x to 5.5x ONNX Runtime;

  • Pow right-scalar by 33x to 35x and integer-valued per-channel exponents to about 9.75x ONNX Runtime.

A complete 1,008-cell development-host pass found all comparable matrix cells and classified 280 BF16 cells as unsupported by ONNX Runtime CPU. Its aggregate median was 1.41x and every operator/type/loop-family group except contiguous FP32 Div (0.983x) reached a 1.0x median. The run did not pass the per-case or small-p90 gates: the shared 96-core host showed severe large-case executor dispersion. These results are diagnostic and do not replace the required pinned dedicated-machine acceptance run.

Pull-request sequence#

PR

Scope

Merge criterion

Depends on

Status

Binary Perf PR01

Reproducible baseline and diagnostics.

The unfiltered corpus records raw samples, selected tuning, actual participants, and separate preallocated/end-to-end timings. Results are grouped by operator, complete signature, loop family, and size.

Completed Binary roadmap

Completed

Binary Perf PR02

Arithmetic and low-precision bulk kernels.

Dedicated FP16/BF16 SIMD and complete typed Add/Sub/Mul/Div/Mod paths improve or retain every arithmetic priority group with exact invalid-input and overflow semantics.

PR01

Implemented through #563 and #577; native half ISA paths remain

Binary Perf PR03

Comparison, logical, bitwise, shift, and PRelu bulk kernels.

Every supported width has contiguous and scalar-broadcast bulk paths; comparisons emit canonical byte BOOL and no priority group regresses.

PR01

Implemented; predicate ISA paths remain

Binary Perf PR04

Prepared broadcast specializations.

Priority rank 2-4 strided patterns avoid per-element type-erased calls and repeated index arithmetic while the arbitrary-rank fallback remains correct.

PR02, PR03

Implemented

Binary Perf PR05

Processor calibration and persistence.

Correctness-gated callbacks jointly tune all five exposed parameters; cache lifecycle, overrides, inspection, and immutable-session behavior pass onnx-light integration tests.

PR01, PR04

Implemented

Binary Perf PR06

Final parity and runtime-overhead gate.

Every priority group reaches median 1.0x ONNX Runtime, no case is below 0.9x, small p90 stays within 2%, and any residual runtime cost is assigned to a measured component rather than hidden by kernel timing.

PR05

Complete through #599; dedicated-machine reruns remain optional cross-machine validation

Binary Perf PR06 completes this implementation roadmap. #599 adds the final integer Pow and dispatch corrections exposed by the expanded benchmark. Remaining AVX2-specific gaps are tracked by the AVX2 performance follow-up.