SIMOR depth map

Convert one Metoak SIMOR camera frame into RGB, metric depth and camera-space XYZ in a single EV74 invocation. The operation consumes separated I420 planes and disparity; camera capture and raw-camera unpacking remain outside Kernels.

Contract

All tensors are dense and contiguous, with batch one. Width W must be even and in 8–2048; height H must be even and in 8–1536. Shapes use the standard six-axis capacity but retain the ranks below, without implicit singleton padding.

Port	Scalar type	Shape	Meaning
Input 0	UINT8	H × W	I420 Y
Input 1	UINT8	H/2 × W/2	I420 U
Input 2	UINT8	H/2 × W/2	I420 V
Input 3	UINT16	H × W	Disparity, 32 subpixel units per pixel
Input 4	FP32	1	Baseline × focal calibration, BF
Input 5	FP32	3	Focal length in pixels, principal point cx and cy
Output 0	UINT8	H × W × 3	Interleaved RGB
Output 1	UINT16	H × W	Depth in millimetres; zero means invalid
Output 2	FP32	H × W × 3	Interleaved XYZ in metres; NaN means invalid

UINT16 is unsigned storage, not INT16 or BF16. Other operations do not gain UINT16 support merely because the shared dtype enum can represent it. A65 selection fails explicitly; this operation has no CPU fallback.

Parameters

There are no named numerical parameters. Geometry comes from the tensor specifications. BF and projection are input tensors, so calibration can change on every frame without rebuilding the prepared executable. Disparity scale is fixed at 32. One focal length is used for both image axes.

Metadata admission checks every port before preparation. The native entry checks finite positive BF and focal length, finite principal point coordinates and a finite BF-times-32 product before launching workers. Invalid calibration returns an error without writing outputs; host preparation does not map calibration data.

Numerical behavior

For pixel coordinates x and y, sample chroma at floor(x/2), floor(y/2). Subtract 16 from luma and 128 from each chroma sample, using signed 32-bit intermediates. Preserve these integer coefficients and the rounding bias:

$$\begin{aligned} Y' &= Y-16, & U' &= U-128, & V' &= V-128,\\ R &= \operatorname{clip}_{[0,255]}((298Y'+409V'+128)\gg8),\\ G &= \operatorname{clip}_{[0,255]}((298Y'-100U'-208V'+128)\gg8),\\ B &= \operatorname{clip}_{[0,255]}((298Y'+516U'+128)\gg8). \end{aligned}$$

The right shift is arithmetic. Output order is RGB, not BGR. This preserves the released conversion; replacing it with a differently quantized colour matrix can change output bytes.

For nonzero unsigned disparity d, evaluate in FP32 in the following order. Depth rounds half-up and saturates, whereas XYZ uses the unrounded depth:

$$\begin{aligned} s &= \operatorname{FP32}(BF\times32),\\ z_{mm} &= \operatorname{FP32}(s/d),\\ D &= \operatorname{UInt16}(\operatorname{FP32}(\min(65535,z_{mm})+0.5)),\\ z &= \operatorname{FP32}(z_{mm}/1000),\quad r=\operatorname{FP32}(1/f),\\ X &= \operatorname{FP32}(\operatorname{FP32}(\operatorname{FP32}(x-c_x)\times z)\times r),\\ Y &= \operatorname{FP32}(\operatorname{FP32}(\operatorname{FP32}(y-c_y)\times z)\times r),\\ Z &= z. \end{aligned}$$

Zero disparity writes zero depth and three NaNs. Saturating depth does not clip XYZ. There is no finite-output guarantee for extreme but admitted calibration: reciprocal or projection arithmetic can overflow. Do not add FMA contraction, fast-math reassociation or a rounded-depth intermediate to the projection.

C++ call

#include <array>
#include <simaai/kernels/ops/simor_depth_map.h>

namespace k = simaai::kernels;
namespace simor = k::ops::simor_depth_map;

// Inside the caller's preparation function:
const std::array<k::TensorSpec, 6> inputs{{
    {{360, 640}, k::DType::UInt8},
    {{180, 320}, k::DType::UInt8},
    {{180, 320}, k::DType::UInt8},
    {{360, 640}, k::DType::UInt16},
    {{1}, k::DType::Float32},
    {{3}, k::DType::Float32}}};
const std::array<k::TensorSpec, 3> outputs{{
    {{360, 640, 3}, k::DType::UInt8},
    {{360, 640}, k::DType::UInt16},
    {{360, 640, 3}, k::DType::Float32}}};
auto executable = simor::make_executable({inputs, outputs, {}}, k::Target::EV74);

Handle the returned error before passing the executable to Runtime. Link SimaKernels::simor_depth_map. The named-request facade uses simor_depth_map with the same ordered ports and no parameters. This snippet prepares metadata; Runtime supplies memory, submission and completion. See the device fixture (EV74/simor_depth_map/tests/runtime_e2e.cpp) for the complete invocation.

Memory and lifecycle

Input storage is W×H bytes for Y, W×H/4 for each chroma plane, 2×W×H for disparity, four for BF and twelve for projection. Outputs require 3×W×H, 2×W×H and 12×W×H bytes respectively. Disparity requires two-byte alignment; calibration requires four-byte alignment; output bases require 64-byte alignment.

Read-only inputs may share storage. Outputs must not overlap any input, another output or the device packet. No external workspace or full-frame intermediate is required. Runtime owns DMA-BUF mapping, coherency and retained owners until confirmed completion. A timeout is not permission to release storage.

Implementation notes

The released 0.5.0 SIMD arithmetic is retained in a 16-lane loop with a scalar row tail. Two local row slots overlap four input transfers, computation and three output transfers. All DMA events are retained and drained; completion of one plane is not completion of every plane. Row arrays reserve 84 KiB per worker before compiler stack, spills and alignment overhead.

Workers partition complete row bands whose output boundaries align to 64 bytes; the last band owns the image tail. They use the actual dispatched group count, not the operating-system CPU count. Small frames use fewer useful workers. The dispatcher handle is retained between invocations.

The operation declares developer graph ID 504, mapped to native transport 84. This is a new versioned 96-byte packet, not the legacy graph-20 100-byte packet. Existing released native routes remain unchanged. Rebuild the generic CVU adapter against the generated catalogue and compose matching firmware; no operation-specific Runner or GStreamer switch is added.

Tests and qualification

Host contracts and source safety (EV74/simor_depth_map/tests/correctness.cpp) check port rejection, unsigned storage, wire fields, binding aliases, calibration, dispatch failures, SIMD tails, DMA completion, guards and worker output ownership against an independent scalar oracle. Routine CPU coverage uses small mechanism witnesses; full mode expands geometry. Host models do not establish EV instruction behavior.

Device tests (EV74/simor_depth_map/tests/runtime_e2e.cpp) cover minimum, tail, native sensor, 720p, 1080p and maximum geometry, saturation, rounding and changing calibration on a reused executable. RGB and depth compare exactly; XYZ uses the released absolute-plus-relative tolerance of 1e-5 with matching invalid-value masks. Correctness and submit-to-completion latency run by default. No native latency baseline is claimed before a guarded device qualification.