Geometry, arithmetic order and selection

This page explains the current A65 recipe implementations. It supplements the admitted recipes and common contract; it does not define a generic decoder for arbitrary model exports. Model-space boxes are represented internally by center x,y and size width,height. The published output is original-image integer x,y,width,height after selection, resize inversion and clipping.

Scalar interpretation comes first

For an INT8 port, subtract its INT8-range zero point in integer arithmetic, convert that difference to FP32, then multiply by that port's binary32 dequant_scale. The bounded integer difference is exact. For BF16, expand the 16 stored bits into the upper half of an FP32 encoding; do not interpret them as an INT16 value or apply integer quantization parameters.

Prepared readers locate the same logical channels in dense/padded or admitted channel-blocked storage. They do not change the scalar meaning. Coordinate vectors are read when a candidate's geometry is needed, rather than converting an entire prediction tensor in advance.

For packed BF16 spans, prepared readers interleave the low- and high-byte planes into little-endian 16-bit words. This is a bit reconstruction, not a floating conversion: signed zeros, subnormals and NaN payloads remain unchanged. Partial spans copy only the requested logical lanes; physical padding never becomes a class or coordinate. Reads use the current frame binding, while the plan caches only address metadata. See PreparedReader (A65/boxdecode/detail/execution.h).

BF16 raw maximum and first-index selection are separate bounded operations. If any nonnegative raw word is present, the signed 16-bit maximum identifies the greatest sortable key; otherwise, the unsigned minimum identifies the greatest complemented negative key. An index search returns the first exact raw match, using full in-bounds vectors and a scalar suffix. A caller that only needs to reject a score can omit the index search.

This integer reduction preserves every raw encoding, including NaN payloads and the distinct keys for positive and negative zero. Callers retain their existing finite-score checks and any numerical-zero tie handling; the helper does not impose a different floating-point ordering. See bf16_reduction.h (A65/boxdecode/detail/bf16_reduction.h) and the exhaustive encoding/guarded-tail tests (A65/boxdecode/tests/bf16_reduction_test.inc).

The arithmetic below describes source ordering. It is not a blanket promise that every BoxDecode intermediate is separately rounded binary32: this operation has its own compile policy, and some helpers contain double literals or mixed precision. Preserve the actual types, expression grouping, math overloads and qualified build settings when modifying a path. Do not transplant Dequant's no-contraction contract onto BoxDecode. See family_geometry.h (A65/boxdecode/detail/family_geometry.h), geometry.h (A65/boxdecode/detail/geometry.h), numeric.h (A65/boxdecode/detail/numeric.h) and the build target (A65/boxdecode/CMakeLists.txt).

Anchor-free and decoded geometry

Let the cell be (x,y), the head width/height be W,H, and model dimensions be MW,MH. The code forms FP32 ratios sx = float(MW)/W, sy = float(MH)/H where that recipe uses grid projection.

Recipe	Source-ordered geometry
YOLO26 / YOLOv6 RawLTRB	Expand/dequantize left, top, right, bottom. X center is (float(x) + 0.5f + ((right-left) * 0.5f)) * sx; width is (left+right) * sx. Y uses top/bottom and sy. Do not reinterpret these inputs as centers or log sizes.
YOLOX	X center is (expanded_x + float(x)) * sx; width is std::exp(expanded_width) * sx. Y uses its own offset, exponential and sy. Objectness is a separate score input, not a fifth coordinate.
Decoded CxCyWH	Expand/dequantize x/y and width/height directly. They are already model-pixel centers and sizes: no sigmoid, exponential, cell addition or stride multiplication is inserted. Split extents use their own explicitly mapped input port.
DETR	Expand/dequantize each of four raw box logits, apply sigmoid independently, then multiply x/width by model width and y/height by model height. These are not already-normalized box values. Class softmax is a separate score operation; the final background class is excluded from selection.
CenterNet	Add cell coordinates to expanded x/y offsets first; read expanded width/height from the separate extent port. Then multiply x/width by sx and y/height by sy. Inputs are heatmap-cell quantities, not already-model-pixel boxes.

CenterNet scans each logical class for a strict eight-neighbor peak, excluding the three-cell image border. DETR and CenterNet do not run NMS. Their general selection path validates geometry before retaining a candidate in a bounded streaming top-K, so an invalid high score cannot evict a valid survivor.

DFL16: reduce distributions before geometry

For each of four sides, consume its 16 consecutive regression channels in bin order 0..15:

1. BF16: expand all logits, find the maximum, then compute std::exp(logit - maximum) using the float overload. Do not introduce a double exp through an unqualified math call.
2. INT8: find the largest raw value. Use the prepared 256-entry table at maximum_raw - raw; entry delta was prepared as std::exp(-float(delta) * coordinate_scale). The common zero point cancels from this difference; replacing it with independently dequantized differences can change floating-point rounding.
3. Accumulate sum += e and weighted += e * bin in ascending bin order. If sum > 0, divide weighted by sum. The resulting side distance is in grid-cell units.

With head factor s = 1 << head_index, the x expression is (right-left) * (s*4) + x*s*8 + s*4, while width is (left+right) * s * 8. Y is analogous. Preserve this ordering rather than silently replacing the implementation with a normalized generic softmax or a reassociated distance expression. Nonfinite or invalid resulting geometry is handled by the shared geometry checks.

The table removes repeated INT8 exponentials from execution without storing an FP32 regression tensor. It is an implementation choice, not a promise that BF16 and INT8 recipes share every intermediate rounding step.

Combined YOLO and EfficientDet

YOLOv5 and YOLOv7 use the fixed anchors and channel order described in YOLO heads and the catalogue. Their size formulas are mathematically related but their source grouping is not identical:

• YOLOv5 forms dw = 2.f * sigmoid(width_logit), then width = dw * dw * anchor_width. Its center uses (2.f * sigmoid(x_logit) - .5f + x) * sx.
• YOLOv7 stores four-times anchor sizes, squares the sigmoid in a float helper, then multiplies that square by the stored anchor. Its center helper uses ((sigmoid(x_logit) * 2.0) + x - 0.5) * stride, including the double literals. Algebraically rewriting one helper into the other is not automatically a bit-preserving optimization.

EfficientDet reads nine anchor groups per cell in dy,dx,dh,dw order. The implementation swaps x/y and width/height regression fields before using the shared exponential decoder. At level h, factor s=2^h, the anchor center is ((8*x+4)*s, (8*y+4)*s). The fixed base width/height pair from the catalogue is multiplied by s. The decoder then performs:

• width: anchor width multiplied by std::exp(expanded_dw);
• x center: anchor x plus (expanded_dx * anchor_width);
• height and y center: the corresponding h/y operations.

There is no SSD-style 0.1/0.2 coder scaling in this recipe. Explicit ordered levels and fixed anchors prevent a model-name heuristic from choosing an incompatible geometry convention.

SSD priors and channel order

See SSD recipes for exact feature sizes, class counts and activations. Priors are constructed once during preparation, in the channel order required by the selected recipe. They are not generated from a model-family name at runtime.

SSD300

For its six levels, preparation uses:

Feature size	Step	Scale	Next scale	Aspect ratios, in order
38	8	21	45	2
19	16	45	99	2, 3
10	32	99	153	2, 3
5	64	153	207	2, 3
3	100	207	261	2
1	300	261	315	2

First form binary32 sk1 = scale/300.f, sk2 = next_scale/300.f, then sk3 = std::sqrt(sk1 * sk2). Emit square priors (sk1,sk1) and (sk3,sk3); for each aspect a, append (sk1*sqrt(a), sk1/sqrt(a)) followed by its swapped pair. Each stored width and height is capped at 1.f. The prepared center increment is float(step)/300.f, not 1.f/feature_size. For example, the 38-wide feature uses 8/300, not 1/38.

MobileNet and TorchVision Lite

Both use level scales 0.2,0.35,0.5,0.65,0.8,0.95 and a terminal next scale of 1.0, with float square roots computed at preparation. Their center increment is 1.0f/feature_size, but their anchor sequences differ:

• MobileNet first level: (0.1,0.1), then the scale-0.2 aspect-2 pair and its swapped pair. Later levels: scale square, aspect-2 pair/swapped pair, aspect-3 pair/swapped pair, then the interpolated square sqrt(scale * next_scale). These sizes are not capped at one.
• TorchVision Lite every level: scale square, interpolated square, aspect-2 pair/swapped pair, then aspect-3 pair/swapped pair. Each stored size is capped at one.

These differences affect anchor ordering as well as values. Treating the recipes as interchangeable because both have six feature levels gives incorrect boxes. The source of truth is ssd.h (A65/boxdecode/detail/ssd.h).

Ordered SSD decode

For anchor a, SSD300 loc channels are coordinate-major: coordinate*A + a. The other recipes are anchor-major: a*4 + coordinate. MobileNet orders those coordinates as ty/tx/th/tw, whereas SSD300 and TorchVision Lite use x/y/w/h deltas. The decoder remaps these channels before applying its coder scales.

The current source creates float dx = expanded_x * .1f, dy = expanded_y * .1f, dw = expanded_w * .2f, dh = expanded_h * .2f. For prior (pw,ph) and prepared increment inv_fk:

• x center: (dx * pw + min((float(x)+.5f) * inv_fk, 1.f)) * model_width;
• y center: the corresponding y/ph/model-height expression;
• width: (std::exp(dw) * pw) * model_width;
• height: (std::exp(dh) * ph) * model_height.

The clamp applies to the prior center term, not the entire regressed center. Preserve the scale-before-exponential and left-associated size multiplications; do not fold model size into the prior and assume identical rounding.

Geometry validation, NMS and final coordinates

Before integer conversion or suppression, reject nonpositive sizes, nonfinite area, float area below 0.5, area above INT32_MAX-1, and nonfinite model-space corners or corners beyond ±10,000,000. These checks occur on decoded content, not merely on tensor metadata.

For NMS, the implementation rounds model-space corners with std::lround. It separately computes area as lround(width * height): this is not the product of the rounded corner differences. The suppression threshold is lround(nms_iou_threshold * 256). For same-class boxes with positive intersection and union, suppression implements integer IoU×256 strictly greater than that threshold. The division-free form tests 256 * intersection >= (threshold + 1) * union using widened integer arithmetic. A nonpositive union never suppresses. Do not replace this with floating-point IoU or >= threshold, which changes boundary decisions.

Only accepted boxes undergo original-image projection. Centered-letterbox uses upscale = max(original_width/model_width, original_height/model_height) with float ratios and centered margins in original-image units. Stretch uses separate x/y ratios and is selected only for SSD/DETR. Top-left and size are formed before clipping; clip both endpoints to image bounds, then recompute nonnegative size. Publication rounds top-left and the clipped sizes independently to INT32 using std::lround. It does not publish the integer NMS rectangle or derive output size by subtracting separately rounded endpoints.

These stages are deliberately distinct: changing when clipping or rounding occurs can change suppression and output coordinates. See selection.h (A65/boxdecode/detail/selection.h), execution.h (A65/boxdecode/detail/execution.h), and kernel.cpp (A65/boxdecode/kernel.cpp). For RCNN's anchor rounding and fifth-output level assignment, see RCNN proposals.

Exact prepared exponential reuse

INT8 RCNN and YOLOX coordinates have only 256 raw encodings per port. Preparation evaluates std::exp(float(raw - zero_point) * dequant_scale) for each encoding under the required floating-point environment. The selected coordinate's entry then replaces only the exponential call: RCNN still multiplies anchor width on the left; YOLOX still multiplies its spatial stride on the right. BF16 keeps the direct float exponential. Invalid infinite, zero-sized or out-of-range geometry continues through the same rejection rules.

The private 256-entry storage is operation-specific: DFL uses its existing nonnegative raw-delta table, whereas RCNN/YOLOX use raw-payload indexing. These roles are mutually exclusive. No full-tensor FP32 conversion, frame allocation, or mutable cross-frame memoization is introduced. The independent prepared-coordinate tests (A65/boxdecode/tests/prepared_geometry_test.inc) exercise every raw INT8 value, multiple scales/zero points and physical layouts, including exponential/dequantization overflow and final clipped integer outputs.