Pose, segmentation and proposal outputs
Set Params::task explicitly; detection is the default. For pose and segmentation,
each head adds a unique auxiliary_input port matching its H/W and dtype. These
ports do not participate in class argmax or change the four common detection
outputs. RCNN proposals instead use the detection task and derive their fifth
output from the selected boxes; they do not require an auxiliary prediction port.
- Pose: DFL or YOLO26, 1–17 keypoints (default 17), auxiliary C=
3*keypointsordered raw x/y/visibility. Fifth output is contiguous FP32[K,keypoints,3]. Coordinates are rounded/clipped original-image pixels represented as floats; visibility is sigmoid. DFL uses(cell+2*offset)*stride; YOLO26 uses(cell+0.5+offset)*stride, followed by centered-letterbox inversion. Nonfinite coordinates/visibility become zero, not an unchecked integer cast. - Segmentation: DFL, YOLO26, YOLOv5 or YOLOv7. Auxiliary C=32 coefficients per
anchor (96 for combined three-anchor heads).
mask_basis_inputidentifies a separate[160,160,32]basis; the model must be 640×640. Fifth output is UInt8[K,160,160], withsigmoid(dot(coefficients,basis))*255truncated inside the rounded model-space box crop and zero outside. Masks remain in prototype/model space, not resized original-image masks. Basis dtype may independently be INT8 or BF16 with its own quantization. INT8×INT8 uses widened integer NEON accumulation and applies scales to the sum. Nonfinite mask probabilities give zero. No full-tensor FP32 staging is needed. - RCNN proposals: detection task, fifth output INT32
[K]is level 0–3. K is global capacity, not K per level. Proposals are score-ordered; callers that require level-grouped ROI inputs must regroup the valid prefix using the fifth output.
All output tails, including the fifth tensor, are zeroed on every successful call.
YOLO26 auxiliary outputs support one to three heads. Task/port/shape errors are rejected in check_inference.
These limits are admission bounds, not a large-memory or latency qualification.
DFL pose preparation fragment
The operation selects the tensor recipe; task adds keypoints without introducing
another runner. This builds a BF16, one-class, 17-keypoint plan with compact HWC
inputs. The plan owns copies of these local specifications and parameters.
#include <simaai/kernels/ops/boxdecode.h>
#include <array>
#include <vector>
simaai::kernels::Result<simaai::kernels::ops::boxdecode::Plan>
prepare_dfl_pose() {
namespace k = simaai::kernels;
namespace b = k::ops::boxdecode;
constexpr int K = 16;
b::Params params;
params.model_input_size = {.width = 640, .height = 640};
params.original_image_size = {.width = 640, .height = 640};
params.num_classes = 1;
params.max_detections = K;
params.score_activation = b::ScoreActivation::Sigmoid;
params.detection_threshold = 0.5f;
params.nms_iou_threshold = 0.5f;
params.task = b::Task::Pose;
params.keypoints = 17;
std::vector<k::TensorSpec> inputs;
for (int hw : {80,40,20}) {
const auto first = inputs.size();
for (int channels : {64,1,51}) {
inputs.push_back({{1,1,1,hw,hw,channels}, k::DType::BFloat16});
}
params.heads.emplace_back(first, first+1, std::nullopt, first+2);
}
std::array outputs{
k::TensorSpec{{K,4}, k::DType::Int32},
k::TensorSpec{{K}, k::DType::Float32},
k::TensorSpec{{K}, k::DType::Int32},
k::TensorSpec{{1}, k::DType::Int32},
k::TensorSpec{{K,17,3}, k::DType::Float32}};
params.profile = b::Profile::YoloDflV1;
return b::check_inference({inputs, outputs, params});
}
See the common BoxDecode contract for errors, memory, outputs and qualification.
Mask initialization work
Every successful mask output still defines the entire requested output,
including pixels outside each crop and unused detection slots. A full-width
crop has no left/right padding, so publication skips the corresponding
zero-length per-row memset calls. Nonempty padding is still cleared, and crop
pixels are still computed by the selected mask dot-product path. This is not an
option to leave tails undefined or to rely on a previously cleared allocation.
See outputs.h (A65/boxdecode/detail/outputs.h).
Packed BF16 reconstruction
For complete C16 stripes, mask evaluation interleaves the low and high byte
planes directly into BF16 words before expanding them to FP32. The interleave
preserves every bit pattern, including signed zeros, subnormals and NaNs.
Neighboring pixels have independent accumulators: each retains the same eight
four-lane multiply-adds, contraction behavior and horizontal reduction order.
Partial spatial runs and odd channel tiles keep their bounded tail/fallback
handling. This changes data reconstruction, not the mask formula or output
rounding. See masks.h (A65/boxdecode/detail/masks.h) and the
scalar-oracle tests (A65/boxdecode/tests/mask_specialized_test.inc).
Contiguous mask-complement gaps
For a nonempty crop, initialization writes the prefix before its first pixel, the contiguous gap between each pair of crop rows, and the suffix after its last pixel. Each inter-row gap combines the preceding row's right padding with the following row's left padding. No crop pixel is cleared or written twice by this step. An empty crop clears the entire mask slot. Unused slots remain the responsibility of output publication; no previous-frame output is reused.
See mask_clear.h (A65/boxdecode/detail/mask_clear.h) and the
partition tests (A65/boxdecode/tests/mask_clear_test.inc).
For an all-INT8 basis and selector, the existing exact zero-selector shortcut
writes probability byte 127 inside the crop. Crop widths 1–4 use bounded stores;
width 3 writes exactly three bytes, including at the output edge. Other widths
retain row fills. This policy is selected only after all 32 selector values match
the selector zero point. It does not apply to BF16/mixed masks and does not
change complement initialization or read selector values twice. See
mask_zero.h (A65/boxdecode/detail/mask_zero.h) and the
guarded zero-selector tests (A65/boxdecode/tests/mask_zero_test.inc).