DetessDequant

Experimental C++20 API, package version 0.1.0. Link SimaKernels::a65_detessdequant. The caller contract below describes supported behavior; implementation notes describe this release, not a stable dispatch or binary ABI promise.

Header: simaai/kernels/a65/detessdequant.h (A65/detessdequant/backend.h). Typed request: ops::detessdequant::Request.

Contract

simaai/kernels/a65/detessdequant.h (A65/detessdequant/backend.h) provides checked, synchronous MLA-tiled INT8 → FP32 or BF16 execution. It fuses layout decoding and dequantization without an intermediate INT8 tensor or full FP32 tensor for BF16 output.

Property	Supported contract
Ports	Exactly one input and one output; both specs are explicit.
Shapes	Equal logical shape and rank, 1–6 dimensions. Empty tensors are valid; scalars are rejected. No implicit axis folding.
Input	Int8, MlaTiled or MlaCBlocked, 16-byte-aligned host pointer.
Output	Float32 or BFloat16, Contiguous or admitted Strided; pointer aligned to four or two bytes respectively. Strides are nonnegative, scalar-aligned, positive for nonunit axes, and pass a sorted-span nonoverlap proof.
Mapping	Explicit D/H/W/C axis mapping, with C required; unmapped logical axes remain independent outer dimensions. Compact or padded-to-16 channels.
Parameters	Params follows Dequant's typed schema: required float scale, optional int32_t zero_point defaulting to zero.
Scale	Finite and strictly positive, including positive binary32 subnormals. This is a dequantization scale, not Quant's multiplier.
Zero point	Full INT32 parameter domain, preserving the Dequant contract. This does not add INT32 tensor support.
Unsupported	UINT8/INT16/INT32/FP16/BF16 tensor inputs, FP16/integer outputs, per-axis scales and overlapping, negative or non-scalar-aligned output strides.
Aliasing	Accessed ranges must not overlap. Adjacent views and unused extra capacity are allowed.
Ownership	Borrowed, accessible CPU buffers through synchronous return. Runtime owns allocations, mappings, coherency and lifetime.

The input follows the explicit MlaTiled (include/simaai/kernels/tensor.h) contract used by native QuantTess. Each pixel stores its logical INT8 channels, then any padded channels. Tiles use actual boundary extents and 16-byte-aligned spans. Channel tiles precede D/H/W tiles; outer slices retain logical row-major order.

Channel padding and alignment gaps are ignored, even if they contain nonzero bytes. The output contains only logical elements. Input capacity is derived from physical tile geometry; output capacity is the full addressed span, including stride holes. For contiguous output this equals logical element count × scalar width. Stride holes and unused output capacity remain untouched. The caller's supplied capacity remains an independently checked fact.

Compiler channel-block input

MlaCBlocked{{tile_h, tile_w, tile_c}} admits ranks 3–6 with trailing H/W/C axes. Earlier axes are independent HWC volumes. Each channel slice is padded independently to 16; output contains only logical elements, with caller-selected strides. This representation is not interchangeable with pixel-major MlaTiled.

For a tile with P = actual_h * actual_w pixels, local channel c and local pixel p, the input-byte offset is:

16 * ((c / 16) * P + p) + c % 16

Integer division is used. The fused reader traverses channel blocks and converts only valid lanes directly into the final output. It uses the same numerical core as the pixel-major path; no full-size repacking tensor or extra numerical pass is introduced. Existing MlaTiled routes retain their optimized dispatch. Depth tiling or nontrailing H/W/C mappings are not implied by this layout.

Parameters

ops::detessdequant::Params requires positive finite scale; its optional full-range INT32 zero_point defaults to zero. It has the same fields as Dequant but remains a distinct request type for typed executor dispatch. No per-axis quantization is admitted.

Numerical behavior

DetessDequant preserves Dequant's separately rounded binary32 operations:

const float zp = static_cast<float>(zero_point);
const float difference = static_cast<float>(q) - zp;
const float output = difference * scale;

Conversion, subtraction and multiplication use round-to-nearest-even; the subtract and multiply are not contracted into an FMA. Large zero points retain the specified INT32-to-FP32 rounding behavior. Valid finite scales may produce signed subnormal/zero or infinity outputs. Such results are not input errors.

For BF16 output, narrow the separately rounded FP32 result to nearest-even:

const uint32_t bits = std::bit_cast<uint32_t>(output);
const uint16_t bf16 = uint16_t((bits + 0x7fffU + ((bits >> 16) & 1U)) >> 16);

This is not BF16 arithmetic or direct rounding of the unrounded real product. Finite FP32 results can round to BF16 infinity; small signed results can round to signed zero. Valid INT8 values, finite zero points and positive finite scales do not produce NaNs. Invalid scales, including NaNs and infinities, are rejected.

Nonempty execution requires nearest rounding, gradual FP32 underflow and masked FP exception traps. Both Dequant and DetessDequant use the same private checker. It checks A65 FPCR and, on x86, SSE rounding/flush/mask controls plus x87 rounding/trap state. The kernel rejects unsupported state before output writes; it never changes the thread's mode. Arithmetic exception flags may be raised. Empty calls perform no arithmetic and do not require a particular FP mode.

Preparation validates scale bits, not a floating-point comparison, so admission is independent of the caller's flush/rounding mode. It never converts the zero point to FP32 or accesses tensor contents.

C++ call

The call below validates a shared request, prepares A65 execution and writes FP32 output into caller-supplied storage. BF16 output uses the same lifecycle with a BF16 output specification and appropriately sized storage.

Place the includes and namespace aliases at file scope; the remaining statements belong in a simaai::kernels::Status-returning caller function. input_views and output_views are caller-supplied arrays of ConstHostBuffer and HostBuffer in port order, with accessible capacities and alignment from the checked plan. Keep their storage alive until synchronous execution returns.

#include <array>
#include <simaai/kernels/a65/detessdequant.h>

namespace k = simaai::kernels;
namespace dd = k::a65::detessdequant;
namespace op = k::ops::detessdequant;
k::MlaTiled layout;
layout.axes = {-1, -1, 0, 1};
layout.tile_shape = {1, 1, 2, 3};
const std::array inputs{k::TensorSpec{{2, 3}, k::DType::Int8, layout}};
const std::array outputs{
    k::TensorSpec{{2, 3}, k::DType::Float32, k::Contiguous{}}};
auto admitted = op::check_inference({inputs,
                                     outputs, op::Params{0.25f, -3}});
if (const auto *error = std::get_if<k::Error>(&admitted)) return *error;
const auto plan = dd::prepare(std::get<op::Plan>(admitted));

return dd::execute(plan, input_views, output_views);

For a pitched [2, 3] FP32 result, replace the output layout with k::Strided{{16, 4}}. The required addressed span is 28 bytes, not 24; the four-byte hole between rows stays untouched. Scale, zero point and ordered arithmetic are unchanged.

See the correctness tests (A65/detessdequant/tests/correctness.cpp) for concrete storage, independent expected results and failure cases.

See calling kernels for the shared lifecycle.

Memory and lifecycle

ops::detessdequant::check_inference() reuses Dequant's parameter and logical-shape checks, then validates tiled input and admitted linear output with the shared MLA planner. Its immutable plan owns specs, parameters and tile geometry; it contains no backend function pointers or tensor storage. a65::detessdequant::prepare() selects numerical functions and records the largest legal contiguous channel/row/plane/tile runs. The A65 convenience check_inference() performs both steps.

Execution checks actual port counts, capacities, null pointers, alignment, address overflow, overlap and current FP state before writing. Invalid calls cannot partially update the output. Pointer accessibility and buffer lifetime remain the caller's responsibility. Plans are reusable and may be shared by concurrent invocations with separate output storage.

Implementation notes

Contiguous runs reuse A65 serial/NEON numerical helpers. FP32 retains its existing prepared traversal and numerical functions. BF16 uses the same staged arithmetic with vector integer narrowing and direct packed 16-bit stores; it does not call a separate Cast operation. For an INT8-range zero point and power-of-two scale from 1/16 through 16, preparation selects an integer-widening/shift or fixed-point conversion path. These specializations remove vector FP subtraction/multiplication where exact. For other scales, preparation selects a direct INT8-range-zero-point function or a wider-zero-point function. The former widens the integer difference before FP32 conversion and multiplication; the latter retains FP32 conversion followed by separately rounded subtraction, then multiplication unless the scale is one. Unit scale is selected from the scale bits, not a floating-point comparison. BF16 additionally reuses the proven wider-zero-point power-of-two specialization for zero points from -16777089 through 16777088, where the scaled operands and differences remain exact. Other wide zero points retain the general staged route. No parameter classification is repeated per tile, and preparation performs no FP arithmetic. Every route retains the original scalar expression for its tail, including the rounding of wider zero points before subtraction.

General tiles with contiguous channel runs of at least 16 elements use a channel-run loop with cached extents and strides. It makes the same numerical call for each pixel and advances past stored-channel padding, without joining pixels or moving scalar tails into vector blocks. Short, noncontiguous and already-collapsed tile/row/plane runs retain their existing traversal.

Short or noncontiguous channel runs batch up to 16 elements across each tile row. Compact input bypasses gathering; contiguous output bypasses scattering. This uses at most 16 input bytes plus 64 FP32 output bytes, or 16 input bytes plus 32 BF16 output bytes, on the stack. Scratch is fixed-size, never a tensor-sized intermediate. FP32 retains its direct scalar fallback; BF16 uses the bounded row conversion for noncontiguous short runs. No heap allocation, worker threads or caller workspace is needed.

Compiler CBlock16 input retains one conversion call per original physical channel block, at most 16 channels. Contiguous output channels still write directly. Nonunit output channel strides convert into a fixed 16-element buffer and scatter only valid lanes; this preserves each block's vector/scalar partition and leaves holes untouched. Row, pixel and outer pitches are applied directly by the existing tile traversal. No tensor-sized staging or extra arithmetic pass is introduced.

Short-channel Pad16 gathering

For C2 or C3 physical rows stored as 16 channels per pixel, A65 gathers complete groups of 16 pixels with NEON table selection. Four full Pad16 pixels form each bounded input table; only live channel bytes are selected. Required physical ghost bytes may be loaded but never participate in arithmetic.

Preparation selects this family from the full physical row, not the potentially shorter last W tile. Rows of at least eight pixels qualify when channels and pixels are contiguous in the requested output. Full 16-pixel groups reuse the prepared Dequant function. A remaining group of eight to fifteen pixels is staged separately: absent table vectors are initialized locally, never loaded from the next row, and the function receives exactly the live channel count. Smaller tails retain the established exact walker. C1 retains its established SIMD gather; other channel pitches retain their qualified fallback.

When the actual tile order is also one packed output pixel stream, preparation coalesces adjoining rows and tiles, including packed outer frames through rank six. The proof requires one C tile, exactly sixteen input bytes per pixel, exactly one output scalar per logical element, packed mapped/outer strides, and spatial tile order matching the output order. Multi-row tiles cannot cross finer H/W tiles, and multi-H tiles cannot cross finer W tiles. Holes and physical permutations retain the row/tile walker. This avoids repeating scalar tails at row boundaries without relabeling or changing the tensor contract.

The optional executor and its partial-panel staging are isolated from the established fallback loop. Gathering changes data movement, not subtraction, multiplication or BF16 narrowing. It uses bounded channel-panel scratch, not a full intermediate tensor or public selector.

Contiguous physical output-axis panels

For pixel-major MlaTiled input, a spatial D/H/W tile axis may be physically contiguous in an admitted strided output even when C is not. Preparation chooses this route only when the full tile has at least sixteen logical channels, at least four positions along that axis, and its output stride equals the output scalar width. CBlock16 input retains its separate block reader.

A bounded panel loads four actual pixels and sixteen valid channels per pixel. NEON byte and halfword ZIP operations transpose those sixty-four INT8 values to channel-major groups of four. One existing Dequant call converts exactly those sixty-four values; each four-value group is published contiguously along the selected output axis. Output scratch is at most 256 bytes for FP32 or 128 bytes for BF16. There are no padded numerical lanes or tensor-sized intermediates.

Partial axis positions retain the original sixteen-channel numerical call, and partial C blocks use only their actual channel count. Checked output strides place each live value and leave holes untouched. The ordered numerical core, alias checks, FP-state checks and caller-owned buffer lifetime remain unchanged.

Joining padding-free channel tiles

Preparation caches the eligible prefix length and the channel-only traversal flag in the fixed-size plan. These metadata fields are copied with the plan; they require no per-tile allocation or floating-point arithmetic. Execution checks current bindings and the FP environment before using either route.

Execution joins full C tiles into one call to the prepared Dequant numerical function when there are at least three C tiles, D/H/W each have extent one and one tile, the outer-slice count is one, and the dense output channel stride is the output scalar width (four bytes for FP32, two for BF16). A full C tile must contain a multiple of 16 channels, with stored channel count, INT8 payload bytes and aligned storage bytes all equal. These conditions prove that the full-tile prefix has no padding gaps and follows the dense logical output order.

The final tile remains a separate call, even when it is full. Keeping each full-tile boundary on a 16-element vector boundary preserves the numerical function's vector/tail partition. Scale/zero-point dispatch, subtraction and multiplication order remain unchanged. The final call reads only logical channels: channel padding and alignment gaps remain unread, and the input is never modified.

For 97 channels and C tiles of 32, execution processes a 96-element prefix and a one-element final tile. C tiles of 17 or other padding gaps cannot use this coalescing route; a separate channel-only traversal can still avoid the general D/H/W tile walker. See the shared predicate in mla_tiled_plan.h (common/mla_tiled_plan.h).

Traversing channel-only tiles with padding

After trying the padding-free prefix route, execution checks for a single logical channel stream: one outer slice, one tile along each of D/H/W, D/H/W extents all one, and dense output channel stride the output scalar width (four bytes for FP32, two for BF16). These conditions also cover short tensors with only one C tile. The C tile width is not restricted to 17 or to a multiple of 16; compact and padded-to-16 channel storage are both supported.

For this case, a private channel-only loop calls the same prepared numerical function once for each full C tile. It advances the input by the full tile's physical storage bytes, skipping channel padding and alignment gaps, and advances the output by the full tile's logical channel count. A separate final call uses the final tile's actual logical channel count, even when that tile is full. With only one C tile, FP32 execution invokes that same final-tile numerical function directly, without the traversal-helper call. No tile boundaries are merged here: each tile retains the original vector/scalar tail partition and subtraction/multiplication order. Padding is never read as logical input and the input remains unchanged.

This removes general spatial-coordinate traversal for a layout already proven to be a single channel stream; it does not introduce a new numerical algorithm. Layouts with nonunit spatial extents or multiple outer slices retain the general traversal and its gather/scatter rules. Both fast paths run only after binding and floating-point environment validation, and empty execution returns before either path. See kernel.cpp (A65/detessdequant/kernel.cpp).

Fusion alone does not establish a universally fastest route.

Tests and qualification

The default build includes a65_detessdequant_contract and a65_detessdequant_benchmark. See build and test for host and DevKit execution.

Every numerical/layout correctness case checks both output dtypes against an independent staged arithmetic and tiled-layout oracle. Protected-page tails and FP-environment rejection cover both outputs. Latency cases retain existing FP32 identifiers and add BF16 variants; host measurements are not A65 qualification. Physical-family tests positively exercise strided outputs, including row pitches, scalar stepping, physical permutations and outer dimensions. CBlock16 coverage includes nonunit channel strides, partial channel blocks, exact-capacity no-write rejection, poisoned holes and protected-page bounds.

The independent correctness suite is in A65/detessdequant/tests/correctness.cpp (A65/detessdequant/tests/correctness.cpp). See build and test for commands and the distinction between host development checks, A65 measurements and baseline qualification.