Skip to main content

Native Dequant boundary

Use simaai/kernels/ops/dequant.h and the same request for A65 or EV74. Link SimaKernels::dequant; the EV host codec is SimaKernels::ev74_dequant. The host target does not initialize MetaWare or depend on ConfigManager, JSON, GStreamer or private Core headers.

Shared tensor contract​

Dequant defines the canonical parameters and numerical contract: INT8 input, explicit FP32 or BF16 output, matching logical shapes from scalar through rank six, and independently contiguous or admitted strided views. The finite, strictly positive FP32 scale is required; the INT32 zero point defaults to zero. Subnormal scales are supported. No public image or separate batch fields are needed. BF16 output is plain BF16 storage, not an implicit MLA-tiled layout.

Both backends preserve output gaps and excess capacity. Empty tensors do not submit numerical work. Input/output overlap is rejected. Device-visible address and storage limits remain EV binding restrictions, not different tensor semantics.

Preparation and asynchronous submission​

  1. ops::dequant::check_inference() owns shared metadata admission.
  2. ev74::dequant::check_inference() checks device representability.
  3. encode_relocatable() prepares an immutable packet with zero address slots. For subnormal scales it also prepares the bounded exact-output table below.
  4. Runtime retains the prepared packet and actual tensor owners. It either supplies DMA-BUF relocations or calls bind() with resolved device views. bind() validates that the packet and shared plan match; it never recomputes the table. encode() is the combined convenience operation.
  5. Runtime submits asynchronously and signals completion only after confirmed device retirement and completed coherency. A synchronous call submits and waits.

The kernel library provides no general allocator, hidden packing buffer or worker pool. Packet upload, DMA-BUF mapping, coherency and failure recovery belong to Runtime.

Native descriptor​

simaai_kernels_ev74_dequant_v1 consumes a versioned, little-endian descriptor:

Field groupRepresentation
HeaderMagic, version, byte length, output conversion, rank and flags
Input/output addresses64-bit relocation slots at byte offsets 24 and 32; actual EV domain is 32-bit
Tensor metadataElement count, accessed storage spans, six-capacity extent and byte-stride arrays
ParametersFP32 scale bits at byte 208; signed INT32 zero point at byte 212
Optional exact tableRaw-byte-indexed result bits, beginning at byte 216

Normal-scale packets are 216 bytes. A subnormal-scale BF16 packet is 728 bytes; a subnormal-scale FP32 packet is 1,240 bytes. The table is inline and needs no third relocation. Scalars lower privately to one element. Unused axes are zero and singleton strides are normalized for execution without changing accessed addresses. The device rejects unresolved pointers, invalid sizes, parameters, flags, spans, strides and overlaps before numerical dispatch. Extent validation bounds each non-singleton factor to the 32-bit device address domain before an exact 64-bit multiply; oversized spans are rejected without a software 64-bit division. Singleton strides remain semantically irrelevant, including strides wider than 32 bits. Final address-plus-span and overlap checks remain mandatory.

The private C++-to-OpenCL dispatcher control is a second ABI boundary. Its 32-bit pointer layout uses explicit padding and 8-byte alignment, with every field offset and the total size checked by both compilers. Identical field declarations alone are insufficient because ARC C++ and OpenCL can align 64-bit members differently. Host dispatcher doubles do not establish this ABI.

Platform firmware must compose and identify the native entry explicitly; missing capability is an error, not a reason to send a packet to an unmatched entry or silently execute on A65.

Ordered numerical implementation​

Normal-scale execution preserves these stages:

  1. Convert the zero point to FP32 with round-to-nearest-even.
  2. Construct the exact FP32 value of each INT8 input.
  3. Round the FP32 subtraction of the converted zero point.
  4. Round the FP32 multiplication by scale.
  5. For BF16 only, round the resulting FP32 bits to BF16 using Cast's integer retained-bit/discarded-bit rule.

The EV vector path constructs the byte value through integer widening and a floating magic-bias subtraction. This avoids scalarized byte-to-float conversion without narrowing the zero point: only the input byte is converted through the bounded trick, and the full zero-point subtraction stays a separate FP32 stage. The compiler must preserve subtraction and multiplication boundaries; operation fusion removes a buffer/pass, not these rounding steps.

BF16 uses vector integer rounding and packed 16-bit stores inside Dequant. There is no intermediate FP32 tensor or second Cast submission. Overflow to infinity and BF16 rounding to signed zero follow the shared contract.

Exact subnormal-scale path​

A positive normal scale cannot produce a nonzero FP32 subnormal here: the rounded INT8-minus-rounded-INT32 difference is integer-valued and has magnitude at least one when nonzero. Positive subnormal scales need additional care because EV floating flush behavior must not change admitted semantics.

For those scales, preparation computes every one of the 256 possible results using integer arithmetic. A subnormal scale is an integer significand times 2^-149; the rounded integer difference times that significand fits in a bounded integer product. Explicit ties-to-even rounding produces exact FP32 result bits, then the same BF16 narrowing rule when requested. This computation does not depend on the preparation thread's floating-point rounding or flush mode.

The immutable table costs 1 KiB for FP32 or 512 bytes for BF16. Device execution copies it into local memory, indexes it by the raw input byte, and stores integer result bits. It performs no floating arithmetic on table entries. This is a parameter-derived constant table, not tensor-sized host fallback. It is not the default normal-scale path; gather and upload costs must be measured separately.

Data movement and implementation selection​

Output dtype and normal/table mode select one of four native numerical entries once per invocation. Dispatcher handles are cached, with platform serialization across native entry calls; up to four workgroups process independent regions.

  • Dense and long common contiguous spans use 8,192-element blocks and ping-pong input/output local buffers. Input prefetch overlaps computation; an output slot is not overwritten until its previous DMA completes.
  • Short contiguous spans use direct vector global access where a full vector fits, avoiding DMA startup overhead.
  • Remaining scatter uses an increment/carry coordinate walker rather than recomputing rank-wise division/modulo for each element.
  • Output widths specialize local storage, DMA byte counts and stores. BF16 writes two bytes per element; FP32 writes four. Tails and gaps are never precleared.

Each full block reads 8 KiB and writes 32 KiB for FP32 or 16 KiB for BF16. Both the arithmetic and table paths use this geometry. The two input/output slots occupy 80 KiB for FP32 or 48 KiB for BF16, plus their existing guard padding; table paths additionally retain their small parameter-derived table. This bounds expanding transfers without adding host staging or submissions.

Block size is an implementation choice, not a guarantee of uniform latency. DMA address phase, output width and workgroup count affect transfer cost. Qualify changes with retained input/output allocations and independent offset sweeps, alongside tails, strided cases and normal/subnormal scale profiles. Do not assume a smaller block is faster for every dtype or address.

Build and verification​

Ordinary host/A65 builds compile ev74_dequant_contract with the real native entry and a mock dispatcher. It checks lowering, rejection, parameter/plan binding, malformed metadata and device-address extent boundaries, and exact table bits against an independent staged-arithmetic oracle; it does not execute EV SIMD or DMA.

The private sima_dequantize_int8 target builds the numerical OpenCL kernels and native entry. When Clang-based CPU models are enabled, ev74_dequant_contract also compiles the gather worker's OpenCL expressions as C++ with checked immediate and deferred DMA models. These translation units use signed char to preserve OpenCL INT8 semantics on both x86 and AArch64. This checks numerical expressions, addressing and software ordering, not EV SIMD execution or hardware DMA behavior; those still require private compilation and real-device checks.

Runtime owns the real-device fixture tests/dequant_cvu_e2e.cpp, built as dequant_cvu_e2e with BUILD_TESTING=ON and SIMA_RUNTIME_LINUX_CVU=ON. Build it in the configured Runtime cross-build with matching installed Kernel packages:

cmake --build <runtime-build> --target dequant_cvu_e2e
file <runtime-build>/dequant_cvu_e2e

Run this AArch64 binary only on the approved DevKit, inside the exclusive guarded firmware-test procedure, with a matching composed firmware image. Do not add the firmware-switching procedure to ordinary host CTest.

  • dequant_cvu_e2e --probe-only checks the advertised Dequant capability; it is not numerical or performance acceptance.
  • dequant_cvu_e2e performs the default correctness suite and all latency cases. It has no correctness-only option. The fixture also exercises Dequant-to-Cast dependencies, so its full run needs both native entries in the composed image.

The fixture checks both output dtypes across 600 generated rank/layout cases, normal/subnormal scale and full-INT32 zero-point profiles, dense/padded/transposed views, vector/DMA boundaries, guarded subviews, repeated invocation and empty requests. Independent staged arithmetic supplies expected IEEE results. Latency runs include retained-allocation input/output DMA phase sweeps and verify output, input preservation and untouched output gaps outside the timed region.

The 68 default latency cases report Runtime submission-to-confirmed-retirement wall time and process CPU cost. These are not device-entry cycle counts or simulator cycles. Only medians gate the reviewed latency policy; p95/mean/max remain diagnostic. A successful host contract test, capability probe or private compilation does not establish real-device acceptance. See private builds for the licensed toolchain boundary and qualification for the distinction between measurements and acceptance.

C++ call​

Place the includes and namespace aliases at file scope; the remaining statements belong in a simaai::kernels::Status-returning caller function. It checks metadata and encodes a relocatable descriptor, not EV74 computation. Runtime must map actual buffers, bind addresses and retain inputs, outputs and any workspace until confirmed device completion.

#include <array>
#include <simaai/kernels/ev74/dequant.h>
#include <simaai/kernels/ops/dequant.h>

namespace k = simaai::kernels;
namespace op = k::ops::dequant;
namespace ev = k::ev74::dequant;
const std::array inputs{k::TensorSpec{{2, 3}, k::DType::Int8}};
const std::array outputs{k::TensorSpec{{2, 3}, k::DType::BFloat16}};
const op::Request request{inputs, outputs, op::Params{1.0F}};
auto checked = op::check_inference(request);
if (const auto *error = std::get_if<k::Error>(&checked)) return *error;
const auto &plan = std::get<op::Plan>(checked);
auto encoded = ev::encode_relocatable(plan);
if (const auto *error = std::get_if<k::Error>(&encoded)) return *error;
// Encoding leaves device addresses unbound; no device work has run.
return {};

See the host contract tests (EV74/dequant/tests/correctness.cpp) and device fixture (EV74/dequant/tests/runtime_e2e.cpp) for binding and execution coverage.

Physical traversal and exact finite-domain tables​

A private compatible physical-axis order exposes large SIMD spans while public logical coordinates remain unchanged. True port-order differences use bounded local gather panels and exact output rectangles. An INT8 input has a finite 256-value domain; exceptional-scale routes use the exact prepared domain table with bounded local vector gathers, preserving separately rounded subtraction and multiplication and the FP32/BF16 output contract.