QuantTess

Experimental C++20 API, package version 0.1.0. Link SimaKernels::a65_quanttess. The caller contract below describes supported behavior; implementation notes describe this release, not a stable dispatch or binary ABI promise.

Header: simaai/kernels/a65/quanttess.h (A65/quanttess/backend.h). Typed request: ops::quanttess::Request.

Contract

Contiguous or admitted strided input	Output
FP32	MLA-tiled INT8
IEEE FP16	MLA-tiled INT8
BF16	MLA-tiled INT8

Input and output have the same explicit logical shape, with one through six dimensions. Scalars, UINT8/INT16/INT32 input or output, and per-channel quantization parameters are not supported by this operation. Input byte strides must be nonnegative, scalar-aligned and conservatively provable as nonoverlapping. Active axes require positive strides; singleton/empty axes may have zero strides. Input holes are not logical values and remain untouched. Empty tensors are valid and require no storage. Negative dimensions are rejected, even when another dimension is zero.

Parameters

Params has the same numerical fields as native Quant, but is a distinct type so Runtime can distinguish typed requests. Rounding is shared:

• multiplier: required, finite and strictly positive, including subnormal values. This is a multiplier, not a divisor.
• zero_point: optional, default 0, in [-128, 127].
• rounding: optional, default NearestEven; also TowardZero, TowardPositive and TowardNegative.

Numerical behavior

Numerical results follow:

product = round_to_nearest_even_binary32(exact_input × multiplier)
output  = saturate_int8(round_selected(product) + zero_point)

NaNs map to zero_point; positive/negative infinity saturates to 127/-128. This defines the result, not an instruction requirement. Eligible BF16 paths use integer bit manipulation plus native FP16 conversion; eligible FP16 paths use native FP16. General cases use Quant's fused in-register widening path. There is no tensor-sized dense INT8 intermediate and no BF16/FP16 reinterpretation as each other.

C++ call

Place the includes and namespace aliases at file scope; the remaining statements belong in a simaai::kernels::Status-returning caller function. input_views and output_views are caller-supplied arrays of ConstHostBuffer and HostBuffer in port order, with accessible capacities and alignment from the checked plan. Keep their storage alive until synchronous execution returns.

#include <simaai/kernels/a65/quanttess.h>
#include <array>

namespace k = simaai::kernels;
namespace op = k::a65::quanttess;
const k::Shape shape{1, 1, 1, 1, 2, 3};
k::MlaTiled layout;
layout.axes = {2, 3, 4, 5};
layout.tile_shape = {1, 1, 2, 3};
const std::array inputs{k::TensorSpec{shape, k::DType::Float32}};
const std::array outputs{k::TensorSpec{shape, k::DType::Int8, layout}};
// Shared admission is identical for A65 and EV74 preparation.
auto checked = k::ops::quanttess::check_inference({
    inputs, outputs,
    op::Params{0.5f, -3, op::Rounding::NearestEven}});
if (const auto* error = std::get_if<k::Error>(&checked)) return *error;
const auto plan = op::prepare(std::get<k::ops::quanttess::Plan>(checked));

return op::execute(plan, input_views, output_views);

See the correctness tests (A65/quanttess/tests/correctness.cpp) for concrete storage, independent expected results and failure cases.

See calling kernels for the shared lifecycle.

Memory and lifecycle

QuantTess uses the same MlaTiled descriptor and shared validation/traversal as CastTess:

• Explicit logical-axis indices and tile extents in D/H/W/C order. C is mandatory; missing spatial axes are unit dimensions with mapping -1 and tile extent one.
• Unmapped logical axes are independent outer dimensions, in logical order.
• Tile order is C → D → H → W; within each tile, payload order is D → H → W → C.
• Boundary tiles use actual extents. Channels are compact or rounded up to 16.
• Each tile's total byte span is rounded up to 16 bytes.

INT8 stores one byte per channel, not BF16 low/high byte planes. The two kinds of padding have deliberately different values:

Region	Written value
Actual tensor elements	Quantized result
Padded channel elements	zero_point, representing quantized numerical zero
Alignment-only gap after a tile	Zero bytes; not logical tensor elements

Every execution writes both payload and padding, so reused buffers need no prezeroing. Raw-zero padding is not equivalent when the zero point is nonzero. The producer and consumer must agree on physical layout and padding semantics.

Call check_inference() once per specification/parameter configuration. It validates dtype, shapes, layouts, parameters and overflow-safe storage sizes, then returns an owning fixed-size plan. No request references are retained.

The shared entry point is ops::quanttess::check_inference() in simaai/kernels/ops/quanttess.h. Use a65::quanttess::prepare(shared_plan) to select A65 code, or the A65 check_inference() convenience wrapper. Shared preparation owns the original input strides, tiled output geometry and numerical parameters. Its dense Quant metadata describes the contiguous numerical calls inside the fused traversal, not an intermediate tensor or a separate execution.

Preparation reuses Quant's implementation selection and the shared MLA geometry planner. For each of at most 16 boundary/full tile classes, it records the largest contiguous input/output run: tile, plane, row or channels. Short/noncontiguous channel runs use eight-element gather blocks when a complete vector is available, otherwise a scalar-addressing fallback. Padding and channel subtiles prevent invalid run coalescing. Execution does not copy or re-prepare a Quant plan for each run.

Preparation also caches the channel-only route predicates. For eligible small plans it stores up to eight direct-call records containing offsets, counts and tile-class indices, never buffer addresses. These records use fixed-size plan storage rather than heap allocation; copying a plan copies that bounded storage as well. On 64-bit builds the record array occupies 256 bytes, with additional cached-route metadata. Per-call binding and FP-state validation is unchanged.

The caller supplies one borrowed host-accessible input view and one output view:

• Input alignment: 4 bytes for FP32; 2 bytes for FP16/BF16.
• Output alignment: 16 bytes. Physical capacity comes from the plan, including padding—not merely logical element count.
• Accessible ranges must not overlap. Adjacent/disjoint subviews of one allocation are allowed, and extra capacity outside required storage is untouched.

execute() checks arity, capacity, pointers, alignment and address ranges on every call. For nonempty tensors it also checks the calling thread's current FP state before writing data or padding: nearest FP rounding, gradual FP32 underflow and masked FP traps; paths using FP16 also require IEEE half and no half flush. These are the same requirements as Quant. The explicit rounding parameter controls the later integer rounding step; it does not relax the binary32 product rule. The kernel does not change the FP control mode. Arithmetic may set exception flags.

Execution is synchronous, heap-allocation-free and single-threaded. Gather paths use a fixed stack block of at most 32 input bytes plus 8 output bytes; contiguous runs write directly to the final output. There is no tensor-sized intermediate or caller-provided scratch allocation. The Runtime/caller owns allocation, mapping, lifetime and cache/DMA synchronization. A plan can be shared across independent calls with disjoint outputs. The A65 numerical interface does not implement a scheduler. The EV74 host codec consumes the same shared admission result.

Implementation notes

Open diagram at full size.

Fusion changes data movement, not the equation

The semantic sequence is Quant's binary32 product rounding, selected integer rounding, zero-point addition and saturation, followed by placement at the final MLA byte address. There is no dense quantized tensor between these stages. QuantTess reuses Quant's prepared numerical selection; it does not invent a separate multiplier convention or relax rounding for tile interiors.

For each full/boundary tile class, preparation chooses the largest valid contiguous run: whole tile, plane, row or channels. Channel padding and channel subtiles can prevent joining adjacent rows. An eight-element gather block covers suitable noncontiguous runs; smaller tails use exact scalar addressing. These choices alter reads/writes, not output values or padding semantics. See kernel.cpp (A65/quanttess/kernel.cpp).

Per-pixel channel runs

When preparation selects a channel run, execution uses a dedicated channel-tile function. It selects padded or unpadded output handling once per tile and caches the tile extents and admitted input strides. Each D/H/W pixel still receives one call to the prepared Quant function with the original channel count. Adjacent pixels are not joined, and each call retains its vector and scalar-tail partition.

For padded output, channel padding is filled with zero_point after each pixel's quantized values. The shared traversal clears the tile's alignment gap after the payload. Gather, scalar-addressing, row, plane and whole-tile runs retain their separate handling.

Bounded direct-call schedules

Outside the channel-only routes below, preparation can cache a schedule when there are at most eight physical tiles and every tile needs exactly one existing whole-tile call or one single-pixel channel call. Execution uses those original input/output offsets and counts directly. Calls are not joined, and numerical vector and scalar-tail partitions are unchanged.

For a single-pixel padded channel call, channel padding is filled with zero_point after its quantized values. Each tile's alignment gap is then cleared with raw zeros. Plans with more than eight tiles, or a tile requiring multiple row, channel, scalar or gather calls, retain the general traversal. The bound is an implementation choice, not a limit on supported tensor sizes.

Joining padding-free channel tiles

Execution joins the full-tile prefix into one numerical call only when all of these conditions hold:

• There are at least three C tiles: at least two full tiles followed by a final tile, which may itself be full.
• D, H and W each have logical extent one and one tile; the outer-slice count is one. The dense channel stride equals the input scalar width.
• Each full C tile has a channel count divisible by 16. Its stored channels, INT8 payload bytes and aligned storage bytes all equal that count: no channel padding or alignment gap separates the full tiles.

The final tile is always a separate call to the same prepared Quant numerical function. Its channel padding is filled with zero_point; its alignment gap is filled with raw zeros. Multiples of 16 preserve the numerical function's eight-element vector partition, so joining full tiles does not turn scalar tails into vector work or change rounding. FP32, FP16 and BF16 inputs use their existing prepared numerical selection unchanged.

For example, 97 channels with C tiles of 32 process a 96-element prefix and a one-element final tile. C tiles of 17 cannot use this joined-prefix route. This is a bounded data-movement specialization, not a new layout or parameter contract. See the shared predicate in mla_tiled_plan.h (common/mla_tiled_plan.h).

Walking separate channel-only tiles

If the joined-prefix route does not apply, execution can walk physical C tiles directly when there are at least three C tiles, D/H/W each have logical extent one and one tile, the outer-slice count is one, and the dense channel stride equals the input scalar width. Unlike prefix joining, this route permits padding between full tiles, including compact or padded C17 tiles.

Every full tile and the final tile receive separate calls to the same prepared Quant function with their original channel counts. Input advances by logical channel bytes; output advances by each tile's physical storage span. Channel padding is filled with zero_point, and alignment-only gaps are filled with raw zeros. The walker does not join numerical calls or change their counts; tail arithmetic follows the selected Quant implementation.

Preparation gives the joined-prefix route priority and preserves its call partitions. Layouts with fewer than three C tiles, nonunit spatial dimensions, multiple outer slices or a different dense channel stride instead use an eligible bounded direct-call schedule or the general tile traversal, including the per-pixel channel-run handling above.

Worked physical-size example

For logical shape {1,1,1,2,5,3}, axes {2,3,4,5}, compact channels and tile_shape={1,2,4,3}, INT8 has 30 logical bytes. The first W tile contains 1*2*4*3 = 24 bytes and occupies a 32-byte aligned span. The boundary tile contains 1*2*1*3 = 6 bytes and occupies 16 bytes. Total required storage is 48 bytes, not 30. The 8-byte and 10-byte alignment gaps are zero-filled. This example has no padded channels; choosing PadTo16 changes the required capacity and introduces channel padding filled with zero_point.

Preparation chooses the largest legal contiguous run for each tile class; execution writes directly into final storage. A short gather/scatter block handles noncontiguous mappings without a tensor-sized intermediate. Padding can prevent row coalescing even when logical shapes match. See the detailed fusion diagram below.

Tests and qualification

The default build includes a65_quanttess_contract and a65_quanttess_benchmark. See build and test for host and DevKit execution.

The independent correctness suite is in A65/quanttess/tests/correctness.cpp (A65/quanttess/tests/correctness.cpp). Its channel-layout coverage includes compact and padded tile boundaries, channel-axis permutations, and nonunit spatial and outer dimensions that must not use the channel-only routes. The multidimensional channel-run check covers {13,17,33} with C17/C32 tiles, all three input types, both padding modes and all four rounding modes. Bounded-schedule tests cover seven, eight and nine outer-slice calls, copied plans, and padded or multi-channel-tile layouts that require general traversal. See build and test for commands and the distinction between host development checks, A65 measurements and baseline qualification.