Dequant

Experimental C++20 API, package version 0.1.0. Link SimaKernels::a65_dequant. The caller contract below describes supported behavior; implementation notes describe this release, not a stable dispatch or binary ABI promise.

Header: simaai/kernels/a65/dequant.h (A65/dequant/backend.h). Typed request: ops::dequant::Request.

Contract

The experimental C++20 interface is simaai/kernels/a65/dequant.h (A65/dequant/backend.h). Link SimaKernels::a65_dequant from the installed CMake package.

• INT8 input to FP32 or BF16 output. INT16, INT32 and other tensor input types are rejected by check_inference().
• One input and one explicit output, with matching logical shapes of zero to six dimensions. Axes are not folded or padded into a different logical rank.
• Contiguous or independently strided input/output. Strides are nonnegative byte offsets, aligned to the port's scalar width, and positive on axes with extent greater than one. Zero strides are permitted on singleton or zero-sized axes. The same bounded sorted-span proof as Cast rejects overlapping or unproven interleaved views. Singleton strides are normalized; broadcast and negative strides are not supported. Scalars contain one element; any zero extent makes execution a no-op with zero required bytes. Negative extents are rejected.
• Params{scale, zero_point}: scale is required, finite and strictly positive; zero_point defaults to zero. Its INT32 parameter type is not an INT32 tensor input. The full parameter range preserves the existing numerical behavior.
• The result is equivalent to the staged FP32 expression (float(input) - float(zero_point)) * scale. Exact integer subtraction is used when equivalent; wider zero points retain staged FP32 semantics. Finite parameters can still produce infinity through output overflow.
• Nonempty execution requires nearest rounding, gradual FP32 underflow and masked FP traps. Unsupported rounding, flush-to-zero or enabled FP trap modes return UnsupportedEnvironment before computation. Execution never changes the caller's FP control mode. Scale validation itself is bit-based and independent of those modes.

check_inference() returns an owning plan or a structured error. It validates operation ID, arity, dtype, shape, layout and parameters, and checks element/byte arithmetic for overflow. Input scalars occupy one byte; output scalars occupy four bytes for FP32 or two for BF16, with scalar-width alignment. Required storage includes strided gaps: allocate from the plan's requirements, not merely the logical element count. Output holes and capacity beyond the accessed span remain untouched. Tessellated layouts belong to separate fused operations.

The backend-neutral request and metadata plan are declared in ops/dequant.h (common/dequant/dequant.h). a65::dequant::prepare(shared_plan) selects A65 computation without repeating admission. The A65 check_inference(request) convenience performs both steps. Neither shared metadata nor parameters contain hardware choices or function pointers; A65's backend plan privately stores its selected numerical function.

execute() accepts borrowed CPU-accessible views and checks their actual extents, null pointers, alignment, address overflow and overlapping accessed ranges on every invocation, before any output writes. Views must describe real storage; raw-pointer validation cannot prove allocation ownership, accessibility or lifetime. The caller retains storage until the synchronous call returns. Runtime owns mapping, coherency, allocation permissions and ordering.

The plan can be reused with new buffers or shared by concurrent calls with independent storage. Changed shapes or parameters require a new plan. There is no workspace, hidden worker pool or scheduler in this kernel.

Parameters

Params{scale, zero_point} requires a finite positive binary32 scale. The zero point defaults to zero and accepts the full INT32 parameter range; this does not admit INT32 input tensors.

Numerical behavior

The reference operation order is:

const float zp = static_cast<float>(zero_point);
const float difference = static_cast<float>(input_int8) - zp;
const float result = difference * scale;

Each step rounds to binary32 nearest-even. Subtraction and multiplication are not contracted into an FMA; exact integer subtraction may replace the subtraction only where equivalent. Large zero points must retain INT32-to-FP32 conversion rounding. A finite scale may still produce infinity or a signed subnormal/zero. Those output values are not request errors. Empty calls perform no arithmetic.

For BF16 output, narrow the rounded FP32 result to nearest-even using its representation bits. The upper word is incremented when the discarded word is above half, or exactly half with an odd retained least-significant bit. This preserves signed zero and overflow to infinity; no NaN can arise from admitted finite parameters. BF16 subnormals are not flushed. Do not narrow the scale or intermediate difference to BF16.

This staged result is not always direct rounding from the exact real product. For input=127, zero_point=-123, and scale bits 0x4037ced9, the FP32 product is 718; narrowing that midpoint yields BF16 720 (0x4434). Directly rounding the exact product would yield 716 (0x4433). Fusion removes memory passes, not these rounding boundaries.

Quant's multiplier is not this scale: do not assume either API silently takes a reciprocal.

C++ call

Place the includes and namespace aliases at file scope; the remaining statements belong in a simaai::kernels::Status-returning caller function. input_views and output_views are caller-supplied arrays of ConstHostBuffer and HostBuffer in port order, with accessible capacities and alignment from the checked plan. Keep their storage alive until synchronous execution returns.

#include <array>
#include <simaai/kernels/a65/dequant.h>

namespace k = simaai::kernels;
namespace dequant = k::a65::dequant;
const std::array inputs{k::TensorSpec{k::Shape{2, 3}, k::DType::Int8}};
// Same logical shape, two-byte BF16 words, one untouched word after each row.
const std::array outputs{
    k::TensorSpec{k::Shape{2, 3}, k::DType::BFloat16, k::Strided{{8, 2}}}};
const dequant::Request request{inputs, outputs,
                               dequant::Params{0.25f, -3}};
// This request is also accepted by ops::dequant::check_inference for EV74;
// only backend preparation, memory binding and submission differ.
auto checked = dequant::check_inference(request);
if (const auto *error = std::get_if<k::Error>(&checked)) return *error;
const auto &plan = std::get<dequant::Plan>(checked);

return dequant::execute(plan, input_views, output_views);

See the correctness tests (A65/dequant/tests/correctness.cpp) for concrete storage, independent expected results and failure cases.

See calling kernels for the shared lifecycle.

Memory and lifecycle

Allocate from the checked storage/alignment requirements and retain accessible input/output buffers until synchronous return. Nonempty execution checks FP state before writing. Plans own metadata and may be shared by independent calls; buffers may not overlap.

Implementation notes

Actual dispatch in this release

The shared plan owns tensor metadata and parameters. A65 preparation adds a private output-typed function pointer, classifying scale representation bits and integer zero-point ranges without floating-point arithmetic. Numerical execution checks the calling thread's FP state before any zero-point conversion or writes. The existing FP32 generic dense body retains its once-per-call classification; proven power-of-two routes and short spans are selected ahead of execution. DetessDequant also supports FP32 and BF16 outputs and uses separate private composition functions.

That private composition hook also selects the general routes ahead of execution: INT8-range zero points use widening integer subtraction, FP32 conversion and multiplication for arbitrary scales. Wider zero points use separately rounded FP32 conversion and subtraction, with distinct unit-scale and scaled functions. Selection uses integer zero-point comparisons and scale representation bits; it performs no FP arithmetic. The numerical function therefore does not repeat parameter classification for every tile run. The standalone dispatcher and prepared functions retain the same numerical operations and scalar tail, but keep separate generic vector bodies: their different call patterns benefit from different compiler scheduling. The standalone serial function has an explicit non-inlining boundary; the prepared functions avoid classification within each tile. These are private implementation choices, not additional API requirements.

For standalone A65 calls, preparation selects the exact power-of-two numerical function when the zero point is in the INT8 range and the exponent is from -4 through 4. Wider zero points in [-16777089,16777088] also have a standalone power-of-two specialization in this exponent range. Other calls retain the existing standalone serial function. Selection uses integer metadata and scale bits only. The fixed-size plan stores one additional function pointer, which is copied with the plan; no allocation is added. Execution still checks bindings and the FP environment before invoking the selected function with the original vector/scalar partitions.

Common contiguous spans below 16 elements select the original scalar expression directly, with the runtime call count retained; no vector parameter dispatch is needed. Narrow-zero-point vector routes subtract directly from the high input lanes as well as the low lanes, preserving the same widened integer differences.

For the bounded wider-zero-point specialization, both the zero point and every INT8-minus-zero-point difference are exactly representable in FP32. Scaling by 2^-4 through 2^4 stays exact and normal, except for exact zero. Complete 16-element blocks therefore scale the input and zero point before subtraction, with bit-identical results and no per-vector multiplication. Zero-point scaling occurs during execution, after FP-environment validation, never at preparation. The original scalar remainder is unchanged. This specialization does not change the private numerical functions used by tiled composition.

The general-scale INT8-range-zero-point loops process two independent 16-element blocks per iteration. A remaining complete 16-element block and the original scalar tail follow separately. This scheduling change does not promote tail elements into vector arithmetic or alter conversion/multiplication order.

Build and parameters	Complete 16-element blocks	Why this preserves the result
AArch64 NEON; -128 <= zero_point <= 127; scale = 2^e, -4 <= e < 0	Widening INT8 subtraction → INT32 widening → fixed-point signed-integer-to-FP32 conversion with -e fractional bits	The difference is an exact integer in [-255,255]; the resulting binary fractions are exactly representable.
AArch64 NEON; INT8-range zero point; scale = 2^e, 0 <= e <= 4	Widening subtraction → widening left shift by e → FP32 conversion	The bounded shifted integer is exact. At e=0, this also removes the FP multiply.
AArch64 NEON; INT8-range zero point; any other admitted scale	Widening subtraction → INT32 → FP32 → scale multiplication	Exact integer subtraction is equivalent here; only the final multiplication needs FP rounding.
Standalone AArch64 NEON; wider zero point in [-16777089,16777088]; scale = 2^e, -4 <= e <= 4	Exact scaled-input conversion → subtraction of the exactly scaled zero point	All differences fit FP32 exactly; bounded power-of-two scaling preserves every bit.
Other AArch64 NEON calls with zero point outside INT8 range	INT8 → INT16 → INT32 → FP32 → FP32 subtraction of the converted zero point → scale multiplication	Retains the separately rounded conversion/subtraction required for arbitrary INT32 parameters. Multiplication is omitted only when scale == 1.0f.
Portable host build	Scalar staged FP32 expression for every element	Development implementation of the same numerical contract, not a NEON performance claim.

Power-of-two detection inspects the scale's IEEE binary32 representation. The bounded specialization covers exactly 1/16, 1/8, 1/4, 1/2, 1, 2, 4, 8, 16; other positive powers of two remain valid and take the general route. The bound limits specialization code size; it is not an input restriction.

Each vector path processes 16 input values into four groups of four FP32 values. BF16 functions immediately reinterpret these register values as integer bits, apply nearest-even narrowing with NEON, and store two groups of eight BF16 words. Their general and bounded power-of-two routes use the same eligibility domains and staged results as FP32. No native BF16 ISA extension is required. Portable builds and short spans use a fused scalar expression and integer narrowing.

Dense execution bypasses strided carry state. A65 separately prepares compatible physical fastest-axis prefixes without changing shared logical shape, strides, suffix span or dense flag. Their outer carry pairs the same logical coordinates on both ports. Only a longer span replaces the existing logical traversal; packed compatible permutations execute one numerical call.

For short physical fragments, sufficiently long affine FP32 rows gather whole live fragments into NEON registers and publish only their exact output regions. BF16 short-span panels gather at most 16 live INT8 values and retain staged FP32 subtract/multiply before bit narrowing. The 128-element setup cutoff keeps tiny calls on the original leaf. Exact-sized accesses preserve holes and protected page ends. Scratch is bounded and invocation-local; there is no full-tensor packing, parameter interpreter or implicit tensor copy. Dense FP32 numerical bodies used by tiled composition remain unchanged.

Any remainder, including a complete call shorter than 16 elements, uses the staged scalar expression. The remainder need not use the vector path's instruction sequence to produce the same admitted result. There is no tensor-sized temporary, implicit worker pool or per-element parameter dispatch.

Keep wider zero-point semantics intact

Do not generalize the INT8-range integer subtraction to arbitrary INT32 zero points without proving equivalence. Converting a large zero point to FP32 may round before subtraction. For example, parameter 16777217 becomes 16777216 under nearest-even conversion. With input 1 and scale 1, the staged result is -16777215; subtracting in integer arithmetic first and then converting instead produces -16777216. The large zero point is admitted, but only the staged expression is the contract.

Execution validates the FP environment before zero-point conversion or arithmetic. The FP state is not changed to make a fast path eligible, and valid output subnormals/overflow are not silently flushed or saturated. The production compile policy disables FP contraction for this operation; subtraction and multiplication must not become a differently rounded FMA.

Source anchors: kernel.cpp (A65/dequant/kernel.cpp), int8_kernels.h (A65/dequant/detail/int8_kernels.h), bf16_kernels.h (A65/dequant/detail/bf16_kernels.h), and the per-operation build policy (cmake/A65Library.cmake). The independent correctness suite (A65/dequant/tests/correctness.cpp) checks both output types, independent layouts, exhaustive INT8 values, parameters, FP state, protected tails, immutable preparation and rejected bindings. Implementation route details are version-specific, not additional public parameters.

Tests and qualification

The default build includes a65_dequant_contract and a65_dequant_benchmark. See build and test for host and DevKit execution.

The independent correctness suite is in A65/dequant/tests/correctness.cpp (A65/dequant/tests/correctness.cpp). See build and test for commands and the distinction between host development checks, A65 measurements and baseline qualification.

The latency executable preserves the existing dense FP32 cases and adds matching BF16 and independent-stride workloads. Both correctness and latency remain registered by default. New cases require reviewed same-cohort baselines before regression qualification; measurement-only reports are not performance guarantees. BF16 writes fewer bytes, but narrowing and small-span overhead still require measurement on A65. Median gates are unchanged.