Native Preproc

Preproc fuses image sampling, color conversion, channel affine transformation, output conversion and optional MLA tessellation in one EV74 operation. Use ops::preproc::Request to describe the call; Runtime submits and completes the device work asynchronously.

Preproc is EV74-only. Selecting A65 returns an unsupported-backend error; there is no A65 implementation or automatic CPU fallback. A host numerical reference in the tests is not an alternative execution backend.

Inputs and outputs

Every port has an explicit TensorSpec. Inputs are positive UInt8 planes with trailing H,W,C axes and rank 3–6. Leading axes form an independent image batch; their shapes must match across planes and, for ordinary requests, output. ROI-list requests use the output outer shape as capacity. Six-dimensional capacity does not require six axes. Source/output width and height are at most 4096. Empty images and scalars are rejected.

Input format	Ordered ports per image	Size constraints
Gray	[H,W,1]	Positive width and height
RGB / BGR	[H,W,3]	Interleaved bytes in the named order
NV12	Y [H,W,1], UV [H/2,W/2,2]	Even width and height
I420	Y [H,W,1], U [H/2,W/2,1], V [H/2,W/2,1]	Even width and height
NV16	Y [H,W,1], UV [H,W/2,2]	Even width; odd height is supported

Inputs accept Contiguous or Strided storage with packed channels and pixels, positive row pitch, and a constant flattened frame pitch. Strided uses the canonical byte-stride vector, not new Preproc-specific public pitch parameters: for [N,H,W,C], the strides are [frame_pitch,row_pitch,C,1]. Row pitch must cover W*C active bytes; frame pitch must cover the last active row. Higher outer axes must encode the same flattened frame stepping. Arbitrary pixel or channel strides, negative strides and overlapping frames are rejected. Output storage is unchanged. No packing copy or full-frame intermediate is introduced.

For batch size N, height H, width W and channels C, the required plane span is:

$$(N-1)\,s_{frame} + (H-1)\,s_{row} + W C.$$

Trailing padding after the last row/frame is not accessed or required. Each plane carries its own pitch: NV12/NV16 Y and UV pitches can differ, as can I420 U and V. Plane base offsets belong in Runtime BufferView bindings; ROI coordinates remain relative to the logical image. A frame-interleaved NV12 allocation can bind Y and UV with their respective offsets and a common frame pitch. Read-only plane bounding spans may overlap; writes may not overlap any input span. Runtime must retain the authoritative allocation/mapping and group its access hooks correctly rather than fabricate unrelated owners for aliases. NV61, NV21, YV12 and unlisted formats are not admitted. A mislabeled image cannot be detected by inspecting metadata.

One explicit output contains Gray or RGB/BGR pixels. Gray output requires Gray input; Gray input can expand to RGB/BGR. Color input is not silently converted to luminance or red-channel-only extraction.

Output dtype	Contiguous	MlaTiled	Transformation
UInt8	Yes	Yes	Raw pixels; identity affine, no quantization
INT8	Yes	Yes	Channel affine and required quantization
BF16	Yes	Yes	Channel affine, then BF16 truncation
FP16	Yes	No	Channel affine, then IEEE binary16 nearest-even

MLA tiles map the trailing H/W/C axes, omit D, and include all output channels in one channel slice. Tile height and width cannot exceed the output height and width. Compact and PadTo16 channels are supported. Split channel tiles, MlaCBlocked, strided output, FP16 tiled output and in-place execution are rejected. Layout is separate from scalar dtype. UInt8 tiled output preserves unsigned 0–255 byte values with identity affine and no quantization. The tile address map is the existing one-byte layout; padded channels contain zero. This does not reinterpret UInt8 as signed INT8 or claim that an MLA consumer accepts unsigned arithmetic: the consumer must agree on the numerical dtype.

Parameters and preparation

Params describes image meaning, not transport:

• input_format and output_format: plane/channel interpretation.
• source_roi: optional integer rectangle in full-source coordinates. Omit it to use the full source. Origins must be nonnegative, extents positive, and the whole rectangle inside the source; out-of-frame virtual ROIs are not supported.
• destination: required integer rectangle inside the output canvas. Its width and height determine resize geometry; the output shape determines the canvas.
• interpolation: Identity, Nearest, Bilinear, Bicubic or BoxAverage; defaults to Bilinear.
• color_matrix: explicit Bt601Full or Bt709Full fixed-point conversion. Output resolution never selects a matrix. Bt601Full is the default; limited-range conversion is unsupported.
• affine.offset and affine.multiplier: three coefficients in semantic R/G/B order, independent of BGR storage order. Gray output uses the first coefficient; Gray-to-RGB expansion applies each output channel's coefficient.
• quantization: required for INT8 and rejected for other output dtypes. Its positive multiplier is a multiplier, not a divisor; zero point is in −128…127.
• padding: raw semantic RGB pixels for the canvas outside destination; defaults to {0, 0, 0}.

Input and output formats default to RGB. Affine defaults are offset zero and multiplier one. Set destination explicitly: its zero-initialized default is not a valid rectangle. All three affine coefficient slots are validated, even for Gray output. UInt8 requires identity affine in all three slots.

Preparation has two admission boundaries:

1. ops::preproc::check_inference() validates planes, shapes, rectangles, dtype/layout combinations, coefficients and overflow-safe storage sizes, then returns an owned plan.
2. ev74::preproc::check_inference() checks device representability: source and output dimensions at most 4096, BoxAverage footprints at most eight cells per axis, and batch/plane/output spans representable in 32 bits. encode_relocatable() also performs these EV checks; a common plan alone does not establish EV admission.

Binding checks actual capacity, alignment, nonzero 32-bit device addresses, address-span arithmetic and aliasing. Preparation cannot validate future buffers.

Use ops::preproc::prepare_executable(request, Target::EV74) when publication needs the accepted ROI geometry. Its Prepared result contains the executable and a shared_ptr<const Plan> alias to the exact checked plan used by binding. Both handles share the existing native-payload owner; no second plan allocation, repeated admission or tensor copy is needed. Either handle keeps that metadata alive. Neither owns future input/output buffers: Runtime still retains their mappings until confirmed completion. make_executable() remains the executable-only convenience call, and existing ROI preparation facts are unchanged.

Coefficients must be finite. INT8's half-precision coefficients must be zero or have magnitude from 2^-24 through 65504; the quantization multiplier is nonzero. INT8 affine offsets also cannot be less than −65248, ensuring that the pixel-minus-offset half intermediate stays finite before multiplication. This avoids undefined infinity-times-zero behavior. These restrictions are admission rules, not permission to reinterpret invalid coefficients.

Neat/application code translates mean, standard deviation, aspect-ratio policy and camera metadata into these explicit parameters. The kernel neither parses configuration nor silently changes a requested rectangle to preserve aspect ratio.

C++ call

This call checks an odd-height NV16 input and a BF16 letterboxed output, then encodes a relocatable descriptor. It does not submit a device job or provide a CPU numerical backend.

Place the includes and namespace aliases at file scope; the remaining statements belong in a simaai::kernels::Status-returning caller function.

Runtime must map actual buffers, bind addresses and retain inputs, outputs and any workspace until confirmed device completion.

#include <array>
#include <simaai/kernels/ev74/preproc.h>

namespace k = simaai::kernels;
namespace op = k::ops::preproc;
namespace ev = k::ev74::preproc;
// One packed NV16 image: UV has the SAME height as Y, unlike NV12.
const std::array inputs{k::TensorSpec{{481, 640, 1}, k::DType::UInt8},
                        k::TensorSpec{{481, 320, 2}, k::DType::UInt8}};
const std::array outputs{k::TensorSpec{{224, 224, 3}, k::DType::BFloat16}};
op::Params params;
params.input_format = op::InputFormat::NV16;
params.output_format = op::OutputFormat::RGB;
params.interpolation = op::Interpolation::Bilinear;
params.color_matrix = op::ColorMatrix::Bt709Full;
params.destination = {0, 28, 224, 168};
params.affine.offset = {127.5f, 127.5f, 127.5f};
params.affine.multiplier = {1.f / 127.5f, 1.f / 127.5f, 1.f / 127.5f};
params.padding = {114, 114, 114};
// BF16 output has no quantization parameters.
auto checked =
    op::check_inference({inputs, outputs, params});
if (const auto *error = std::get_if<k::Error>(&checked)) return *error;
const auto &plan = std::get<op::Plan>(checked);
if (auto error = ev::check_inference(plan)) return error;
auto encoded = ev::encode_relocatable(plan);
if (const auto *error = std::get_if<k::Error>(&encoded)) return *error;
// Encoding leaves device addresses unbound; no device work has run.
return {};

See the host contract tests (EV74/preproc/tests/correctness.cpp) and device fixture (EV74/preproc/tests/runtime_e2e.cpp) for binding and execution coverage.

Ordered computation

Preproc keeps sampling, arithmetic and storage in one operation, but they still have a defined order:

1. Sample the ROI using the selected interpolation mode and ROI-border rules.
2. For YUV input, convert the sampled values with the selected full-range matrix.
3. Apply the channel affine transformation and the dtype-specific output conversion.
4. Store channels in the requested order and initialize required physical padding.

Canvas padding skips sampling and color conversion, but still goes through output conversion. Physical MLA padding is different: it is storage outside the logical image. See numerical order for equations, interpolation rules, intermediate precision, rounding boundaries and the ROI chroma diagram.

ROI-list batches

Set Params::source_rois to prepare a list of crops, including crops that extend beyond the source image. Omit it for ordinary image mode; an explicitly empty list produces zero accepted crops and clears every output slot. Each accepted ROI uses the same output dtype, layout and affine/quantization contract.

Preparation filters records in order and retains the accepted geometry as owned plan facts. Output leading dimensions define capacity, from one to 50 crops. Unused slots are binary zero, including physical padding. A single Runtime submission processes the accepted list; it is not a plugin loop over crops.

Read ROI-list batches for exact filtering, per-ROI destinations, virtual-image sampling and published preparation facts.

Memory, invocation and completion

Prepare metadata once, then bind input, output and required workspace for each invocation. Runtime owns allocation, mapping, cache coherency, asynchronous submission and confirmed completion. Keep all bound buffers and their owners alive until completion, not only until submission returns.

The EV implementation uses bounded local storage and retires its DMA events before returning. This does not replace Runtime's host-side completion and coherency responsibilities. See memory and execution for wire compatibility, alias checks, local storage and implementation selection.

Validation and measurement

Start with the existing correctness test (EV74/preproc/tests/correctness.cpp) and real-device fixture (EV74/preproc/tests/runtime_e2e.cpp). The latter uses installed Kernels and Runtime packages, not a private arithmetic-loop benchmark.

Validate and measure Preproc covers build commands, host-model limits, device modes, camera workloads and the median-only acceptance policy. A codec build is not device qualification; an increased latency budget is not a speed improvement.