Quant

Experimental C++20 API, package version 0.1.0. Link SimaKernels::a65_quant. The caller contract below describes supported behavior; implementation notes describe this release, not a stable dispatch or binary ABI promise.

Header: simaai/kernels/a65/quant.h (A65/quant/backend.h). Typed request: ops::quant::Request.

Contract

• One BF16, IEEE FP16 or FP32 input and one INT8 output. Each port independently accepts contiguous or admitted strided storage. BF16 and FP16 are distinct dtypes, not interchangeable 16-bit storage labels.
• Explicit matching input/output shapes, with zero to six logical dimensions. Scalars contain one element; any zero extent makes the call a no-op.
• Required positive finite multiplier, including positive subnormals. It multiplies the input; it is not a dequantization scale or divisor.
• zero_point defaults to zero and must be in [-128,127]. The C++ parameter is INT32, but this contract admits only INT8 zero points.
• rounding defaults to NearestEven. TowardZero, TowardPositive (ceil) and TowardNegative (floor) are also supported.

Parameters

Changing multiplier, zero_point or rounding requires a new plan. Their ranges and defaults are listed above. Integer rounding mode is distinct from the FP control mode required for the product.

Numerical behavior

Keep the rounding boundaries explicit

multiplier is a binary32 parameter. First round the product to binary32; then apply the selected integer rounding; then add the integer zero point; then saturate to [-128,127]. Algebraic rearrangement is not generally safe. For input 1.25, multiplier 2 and zero point -3, nearest-even gives round_even(2.5) - 3 = -1. Rounding after adding the zero point would instead round -0.5 to zero. That is a different operation.

Do not clamp the product to the INT8 interval before adding the zero point. For example, product 200 and zero point -100 produce 100, not 27. Internal safety clamps may be wider and must preserve the final result.

The result rule is:

product = round_to_binary32_nearest_even(exact(input) * multiplier)
output  = saturate_int8(round_to_selected_integer_mode(product) + zero_point)

This specifies the result, not a requirement to execute FP32 instructions. The integer and mixed-precision specializations reproduce it. An exact-product integer rounding rule would differ near some ties and is not silently substituted. NaN inputs map to the zero point; positive/negative infinities saturate to 127/-128. Finite products that overflow also saturate. AArch64 FP32-arithmetic paths use saturating FP-to-INT32 conversion: NaNs convert to zero and out-of-range values to the corresponding INT32 endpoint. The zero-point addition cannot wrap: vectors use saturating addition; scalar tails widen to INT64 before adding. The final INT8 saturation therefore preserves the result for huge and nonfinite inputs without a separate floating-point clamp. Host scalar paths retain the explicit clamp before integer conversion.

C++ call

Place the includes and namespace aliases at file scope; the remaining statements belong in a simaai::kernels::Status-returning caller function. input_views and output_views are caller-supplied arrays of ConstHostBuffer and HostBuffer in port order, with accessible capacities and alignment from the checked plan. Keep their storage alive until synchronous execution returns.

#include <array>
#include <simaai/kernels/a65/quant.h>

namespace k = simaai::kernels;
namespace quant = k::a65::quant;
const std::array inputs{
    k::TensorSpec{k::Shape{2, 3}, k::DType::BFloat16, k::Strided{{8, 2}}}};
const std::array outputs{
    k::TensorSpec{k::Shape{2, 3}, k::DType::Int8, k::Strided{{1, 2}}}};
// Shared admission is backend-neutral; A65 preparation selects computation.
auto checked = k::ops::quant::check_inference(
    {inputs, outputs, quant::Params{2.0f}});
if (auto *error = std::get_if<k::Error>(&checked)) return *error;
const auto plan = quant::prepare(std::get<k::ops::quant::Plan>(checked));

return quant::execute(plan, input_views, output_views);

See the correctness tests (A65/quant/tests/correctness.cpp) for concrete storage, independent expected results and failure cases.

See calling kernels for the shared lifecycle.

Memory and lifecycle

ops::quant::check_inference() validates the operation, port count, dtypes, shape, layout, parameters and overflow-safe storage requirements. It owns copied metadata shared by A65 and EV74 host integration; link SimaKernels::quant when only admission is needed. a65::quant::prepare() selects A65 numerical code from that validated plan. The convenience a65::quant::check_inference() performs both stages. Preparation decodes multiplier bits, independently of the caller's FP environment, and selects the function once. A65 also caches the applicable FPCR rejection mask with that selection, so execution reads and checks the calling thread's FPCR without repeating invariant dtype/half-route decisions. No tensor-content scan or allocation occurs during preparation. execute() rechecks borrowed CPU views for capacity, null pointers, scalar alignment, address overflow and overlapping accessed ranges before computation. No workspace, full intermediate tensor, implicit worker pool or per-element dtype dispatch is required by vector loops.

Strides are byte offsets, nonnegative and scalar-aligned, with positive strides on axes of extent greater than one. Singleton strides normalize to zero. A bounded sorted-span proof rejects overlapping and unproven exotic interleaved views. Input and output can have different physical axis orders or row padding; logical shapes still match. Requirements include the entire accessed address span, not just logical elements times scalar width. Output gaps are left unchanged. Empty tensors require zero storage, but their descriptors are still validated.

Nonempty calls require nearest FP control rounding, gradual FP32 underflow and masked FP traps. Paths using FP16 additionally require IEEE half behavior and no half flush-to-zero. Unsupported control modes fail before output writes; the kernel never changes the caller's control mode. Arithmetic may set FP status flags. Empty calls do not require a particular FP mode.

Buffers must describe real, accessible storage and stay alive until this synchronous call returns. Runtime remains responsible for ownership, mapping, coherency and ordering. Plans can be reused or shared by independent calls; changing metadata or parameters requires preparation again.

Implementation notes

Open the dispatch diagram at full size.

Independent strided traversal

Preparation retains the shared logical suffix and separately coalesces compatible physical fastest-axis prefixes for A65 execution. Outer axes remain logical coordinate identifiers even when visited in a different physical order. Public shape, strides, span_elements() and contiguous() retain their shared meaning. A packed compatible permutation can therefore execute as one numerical call.

The execution pointer is selected during preparation: numerical code directly for a full physical span, a bounded carry for longer fragments, or a short-span panel for sufficiently large scalar-FP32 workloads. Panels gather only live scalar bytes into eight lanes and scatter only corresponding output bytes; compatible affine rows additionally amortize their carry once per row. Their 128-element setup cutoff retains small-call behavior. FP32, BF16 and FP16 panels use the selected staged FP32 expression and rounding mode, never switch to native-half arithmetic merely because scratch becomes contiguous, and preserve row tails and physical holes. The scratch is bounded and invocation-local.

QuantTess retains the separate numerical pointer for its own tile traversal. QuantTess separately admits contiguous or proven nonoverlapping strided input and explicit MlaTiled output. Successful execution introduces no heap allocation or full-tensor temporary.

Automatic dispatch in this release

The decision is made during A65 preparation, not for each tensor element. Let n be the private coalesced physical span on both ports (the whole tensor for a packed compatible traversal), and let k satisfy multiplier = 2^k.

Input	Additional predicates	Automatic route
BF16	AArch64/Linux reports HWCAP_ASIMDHP; n >= 8; multiplier is an exact power of two; -110 <= k <= 110	Packed BF16 bits → bounded FP16 encoding → integer conversion
BF16	Any other admitted configuration	Fused in-register FP32 conversion/multiply/round/pack
FP16	CPU has native half SIMD; n >= 8; exact power of two; -24 <= k <= 15; additionally k >= 0 or rounding is nearest-even/toward-zero	Native FP16, with multiplication omitted at k == 0
FP16	Any other admitted configuration	Fused in-register FP32 fallback
FP32	Any admitted configuration	FP32 route; NEON where built for AArch64

For negative FP16 shifts, underflow may change directed rounding even when a nearest-even or truncating INT8 result is unchanged. That is why eligibility includes the rounding mode, not just dtype and CPU support.

The BF16 half route does not reinterpret arbitrary BF16 as IEEE FP16. It constructs bounded encodings using exponent/significand operations, preserving the final INT8 rule despite the formats' different exponent ranges. No complete FP32 input tensor is created by the fallback either: widening occurs in registers. All FP16-input calls, and selected BF16 half routes, enforce the additional IEEE half/no-half-flush environment requirement.

Pure-integer BF16 power-of-two and general-multiplier implementations are private qualification candidates. They are not automatically selected and are not caller-selectable API parameters. The route name exposed by implementation() is useful diagnostic information, not a stable identifier to dispatch on. The authoritative selection predicates are in dispatch.h (A65/quant/detail/dispatch.h).

Preparation also selects loop structure from the private coalesced span's element count. Floating-point routes below eight elements use only the original scalar path. Spans below 16 elements omit the paired-vector loop; larger spans retain it. The eight-element loop and scalar remainder still use each numerical call's runtime count, including calls from tiled compositions. Native FP16 tails reuse the qualified half arithmetic one value at a time; floating fallback and BF16 tails retain their scalar arithmetic. Implementation names and FP-environment checks are unchanged. The same paired-loop choice applies to native FP16 and packed BF16 half routes without changing their eligibility or eight-lane operations.

BF16 is not implemented as a separate cast followed by quantization. The private qualification candidates are:

• BF16 packed integer power-of-two: eight 16-bit lanes; exponent/significand manipulation, explicit rounding and saturation, with no FP arithmetic.
• BF16 general integer: exact 8-bit × 24-bit significand multiplication in 32-bit lanes, followed by explicit binary32 and integer rounding. Handles arbitrary supported multipliers, including subnormals.
• BF16 bit manipulation plus FP16 conversion: for power-of-two multipliers 2^k, -110 <= k <= 110, constructs bounded FP16 encodings directly from BF16 bits and converts eight lanes to integers. Raw BF16 magnitude comparisons select the small-value and saturation cases without extracting an exponent in every vector. The checked plan caches the threshold and bit-offset constants; rebinding buffers does not recompute them. No FP32 widening or FP multiply occurs in its vector body. Small sentinels and a magnitude-256 clamp preserve the INT8 result, rather than pretending BF16 has FP16's range.
• BF16 fused FP32: integer shift/widen directly into FP32 registers, then multiply, round and pack. No FP32 tensor is allocated or stored.
• FP16 native: eight-lane FP16 arithmetic/conversion for exactly representable power-of-two multipliers 2^k, -24 <= k <= 15. Negative k is admitted only for nearest-even/truncation, because FP16 underflow can change directed rounding. The identity specialization omits multiplication. Each remaining tail value is loaded as exactly two bytes, duplicated in a register, converted with the same half operations, and stored as one INT8 byte. Signed INT16 conversion and saturating zero-point addition preserve the final INT8 result; no extra input or output lane is accessed.
• FP16 fused FP32 fallback and FP32 NEON for the remaining cases.

The floating-point implementations process two independent eight-element groups per full sixteen-element iteration and combine their results into one store. A remaining eight-element group uses the same eight-lane arithmetic. Native FP16 tails reuse that arithmetic with an exact one-element load and store; other routes retain their scalar tails. There is no read or write beyond the declared tensor storage.

FP16 arithmetic instructions are isolated in ISA-specific functions, selected only after Linux reports the required CPU feature. Host builds use scalar paths. Candidate overrides are private test hooks, not public parameters. Preparation selects an eligible implementation; callers do not choose SIMD routines.

Tests and qualification

The default build includes a65_quant_contract and a65_quant_benchmark. See build and test for host and DevKit execution.

The independent correctness suite is in A65/quant/tests/correctness.cpp (A65/quant/tests/correctness.cpp). See build and test for commands and the distinction between host development checks, A65 measurements and baseline qualification.

The existing exhaustive BF16/FP16 encoding sweeps, multiplier and rounding domains, FP32 boundary cases and candidate comparisons remain. The generic executable also checks 300 shapes per input dtype with independent port layouts, common-span boundaries, empty/invalid strides, binding rejection, and unchanged output gaps. The latency catalogue retains all 432 dense IDs and adds 216 strided workloads; new coverage is measurement, not an automatic baseline or a speedup claim.