Accessing Result Data with FlatBuffers

This page is for ctrlX OS (ctrlX CORE or ctrlX IPC): it describes how a ctrlX PLC program reads the result.

LumiScan AI Object Matching publishes the result of each inference on the ctrlX Data Layer as a binary FlatBuffers message. A PLC program can only interpret this message with the schema that describes its layout. The application therefore provides the schema file object_detection.fbs.

This chapter describes how to obtain the schema, how to import it into ctrlX PLC Engineering, and how to read the result, including the position and score of a detected object.

Result Node

Table 1. Result Node Properties
Property	Value
Node address	`hdvisionsystems/lumiscan-ai-object-matching/<network name>/output/result`
Permission	Read-Only
Data type	FlatBuffers
Registered type address	`types/hdvisionsystems/lumiscan-ai-object-matching/object_detection`
Schema file	`object_detection.fbs`
Schema namespace	`hdv.object_detection2d`
Root type	`Result`

The node holds the result of the latest inference. Read it only after the inference has finished and succeeded: with the handshake, output/package-identifier has grown by one (status/code is 3 at the end of every inference requested through the handshake); with control/inference/execute, status/code is 0. See Outcome of an Inference and Status Code Values.

Prerequisites

ctrlX PLC Engineering 2.6 or later.
The libraries CXA_Datalayer and CXA_Flatbuffers are referenced in the Library Manager of the PLC project. CXA_Datalayer reads nodes from the Data Layer. CXA_Flatbuffers provides the base function blocks of the FlatBuffers function blocks that the import generates. After the import, check the Library Manager. If a library is missing, add it there (see also Install the ctrlX Data Layer Library).
LumiScan AI Object Matching is installed and has been started at least once with a valid licence (see Requirements).

Step 1: Obtain the Schema File

The application copies the schema to its application data folder on the ctrlX CORE each time it starts. To download it:

Log in to the ctrlX OS web interface and open Manage app data (see LumiScan AI Object Matching Folder Structure).
Open the folder lumiscan-ai-object-matching/schemas/flatbuffers.
Click the download icon in the Actions column of object_detection.fbs and save the file on the engineering PC.

Alternatively, download the file over WebDAV (see Uploading Data).

The schemas folder is created while the application starts, after the licence check. If the folder does not exist, the application has not started successfully. Check that a valid licence is installed and read the system log. See The schemas folder does not exist.

The application deletes and recreates the schemas folder on every start. Do not edit files in this folder; changes are lost. The schema always matches the installed application version.

Step 2: Import the Schema into the PLC Project

In ctrlX PLC Engineering, open the PLC project.
In the device tree, right-click Application.
Select Import from FlatBuffers file….
Select the downloaded object_detection.fbs.

ctrlX PLC Engineering device tree with the Application context menu open on Import from FlatBuffers file

Figure 1. Import from FlatBuffers file

PLC Engineering adds the following function blocks to the application. Their names consist of the schema namespace and the type name.

Table 2. Function Blocks Created by the Import
Function block	Purpose
`hdv_object_detection2d_Result`	Root of the message. Provides the best score and class, and access to the array of instances.
`hdv_object_detection2d_Instance`	One detected object: class index, score and bounding box.
`hdv_object_detection2d_OrientedBoundingBox`	Center, size and angle of the bounding box of an instance.
`hdv_object_detection2d_InstanceTVec`	Not needed for reading results (see Known Message: Identifier 'VectorT' Not Defined).

Known Message: Identifier 'VectorT' Not Defined

Depending on the version of ctrlX PLC Engineering, the editor of the function block hdv_object_detection2d_InstanceTVec shows the message Identifier 'VectorT' not defined. The message does not affect reading results. Causes and steps are described in Troubleshooting:

The message "Identifier 'VectorT' not defined" appears after the import

Install the ctrlX Data Layer Library

If Import from FlatBuffers file… is not available in the context menu, the ctrlX Data Layer library is missing. Install it as follows, then repeat Step 2:

In the PLC Engineering menu bar, select Tools > Library Repository….

ctrlX PLC Engineering Tools menu with the Library Repository command highlighted

Figure 2. Library Repository command

Search for CXA_Datalayer, select the library and click Open.

Find Library dialog with the search term CXA_Datalayer and the library CXA_Datalayer selected

Figure 3. Searching for the CXA_Datalayer library

Step 3: Read the Result in the PLC Program

Run an inference with the handshake described in Initializing the Inference.
Wait until the inference has finished and check that it succeeded: inference/ready is True again and output/package-identifier has grown by one. With control/inference/execute instead, check that status/code is 0.
Read the node hdvisionsystems/lumiscan-ai-object-matching/<network name>/output/result with the function block DL_ReadNodeValue of the CXA_Datalayer library. The function block returns the value in a DL_NodeValue.
Pass the address and size of the buffer (GetData and GetSize of the DL_NodeValue) to getRootAsResult of hdv_object_detection2d_Result.
Read the fields of the result with the get methods of the generated function blocks.

The generated function blocks read the received buffer in place; they do not copy it. Use the values or copy them before the node is read again.

Example: Reading Position and Score of an Instance

The instances array of the result is sorted by score in descending order. Instance 0 is therefore the instance with the highest score. The example reads the position (x, y), the score and the class index of this instance.

Declaration:

PROGRAM PRG_ReadObjectMatchingResult
VAR
    // Data Layer access (CXA_Datalayer)
    fbReadResult : DL_ReadNodeValue;
    nvResult     : DL_NodeValue;    // receives the FlatBuffers buffer
    bRead        : BOOL;            // set to TRUE to read the node once
    sNode        : STRING(255) := 'hdvisionsystems/lumiscan-ai-object-matching/<network name>/output/result';

    // FlatBuffers readers, generated by "Import from FlatBuffers file..."
    fbResult     : hdv_object_detection2d_Result;
    fbInstance   : hdv_object_detection2d_Instance;
    pBox         : POINTER TO hdv_object_detection2d_OrientedBoundingBox;

    // Values of the instance with the highest score
    bFound       : BOOL;
    rX           : REAL;            // x of the bounding box center [pixel]
    rY           : REAL;            // y of the bounding box center [pixel]
    rScore       : REAL;            // score [0.0 - 1.0]
    uClassIndex  : ULINT;           // index into class-names
END_VAR

Implementation:

// 1. Read the node. A rising edge on bRead starts one read.
fbReadResult(Execute := bRead, NodeName := sNode, NodeValue := nvResult);

IF fbReadResult.Done THEN
    bFound := FALSE;

    IF nvResult.GetSize() > 0 THEN
        // 2. Point the generated reader at the received buffer.
        fbResult.getRootAsResult(data := nvResult.GetData(), size := nvResult.GetSize());

        // 3. The instances are sorted by score, so instance 0 has the highest score.
        IF fbResult.getInstancesLength() > 0 THEN
            IF fbResult.getInstances(j := 0, result := ADR(fbInstance)) THEN
                rScore      := fbInstance.GetScore();
                uClassIndex := fbInstance.GetClassIndex();

                // 4. The position is part of the bounding box.
                pBox := fbInstance.getOrientedBoundingBox();
                IF pBox <> 0 THEN
                    rX     := pBox^.GetCenterX();
                    rY     := pBox^.GetCenterY();
                    bFound := TRUE;
                END_IF
            END_IF
        END_IF
    END_IF
END_IF

// 5. End the access.
IF fbReadResult.Done OR fbReadResult.Error THEN
    bRead := FALSE;
    fbReadResult(Execute := bRead, NodeName := sNode, NodeValue := nvResult);
END_IF

Replace <network name> in sNode with the name of the network. Set bRead to TRUE only after the inference has finished and succeeded (see Step 3). If fbReadResult.Error is TRUE, evaluate fbReadResult.ErrorID and fbReadResult.ErrorIdent.

Table 3. Methods Used in the Example
Value	Method	Type	Unit or range
Number of instances	`fbResult.getInstancesLength()`	`UDINT`
Instance by index	`fbResult.getInstances(j, result)`	`BOOL`	`TRUE` if the instance exists
Score	`fbInstance.GetScore()`	`REAL`	0.0 to 1.0
Class index	`fbInstance.GetClassIndex()`	`ULINT`	index into `class-names`
Bounding box	`fbInstance.getOrientedBoundingBox()`	`POINTER TO hdv_object_detection2d_OrientedBoundingBox`
x of the center	`pBox^.GetCenterX()`	`REAL`	pixel
y of the center	`pBox^.GetCenterY()`	`REAL`	pixel

The score order is the behaviour of the current version. The schema allows the sort order to become configurable in a future version. getBestIndex() of hdv_object_detection2d_Result always returns the index of the instance with the highest score; it is 0 while the instances are sorted by score. Use it instead of the fixed index 0 if the program has to remain independent of the sort order.

Reading All Instances

To evaluate all instances, loop over the array. Because the instances are sorted by score, the loop can stop at the first instance below a minimum score. The following fragment replaces step 3 of the implementation. It needs the additional declarations i : UDINT; and rMinScore : REAL := 0.5;.

i := 0;
WHILE i < fbResult.getInstancesLength() DO
    IF fbResult.getInstances(j := i, result := ADR(fbInstance)) THEN
        rScore := fbInstance.GetScore();

        IF rScore < rMinScore THEN
            EXIT;   // sorted by score: no instance with a higher score follows
        END_IF

        pBox := fbInstance.getOrientedBoundingBox();
        IF pBox <> 0 THEN
            rX := pBox^.GetCenterX();
            rY := pBox^.GetCenterY();
            // Use or store the values of instance i here.
        END_IF
    END_IF
    i := i + 1;
END_WHILE

Do not use a FOR loop with the unsigned count (FOR i := 0 TO count - 1): if the count is 0, the upper limit wraps around to the largest UDINT value.

Result Structure

The following tables describe the content of object_detection.fbs. The schema file is the authoritative definition.

Table 4. Result (table, root type)
Field	Type	Description
`instances`	`[Instance]`	Detected objects, sorted by score in descending order (highest score first). The array can contain objects of different classes; the PLC program has to select the required class (see Class Names and Class Filters).
`best_index`	`ulong`	Index of the instance with the highest score in `instances`. Always 0 while the instances are sorted by score.
`best_score`	`float`	Score of the best instance, in the range 0.0 to 1.0.
`best_class_index`	`ulong`	Class index of the best instance.
`image_id`	`ulong`	Package identifier of the input image the result belongs to. In versions before October 2026 it was a 16-bit `short`.
`image_name`	`string`	Not used.
`requested_timestamp`	`ulong`	Time at which the application accepted the inference request that the PLC had set to True.
`timestamp`	`ulong`	Timestamp of the input image. Type and resolution depend on the timestamp source of the `lumiscan-cam-driver` application that delivered the image.
`status`	`ulong`	Not used. Use `status/code` to check the outcome (see Status Code Values).

Table 5. Instance (struct)
Field	Type	Description
`oriented_bounding_box`	`OrientedBoundingBox`	Position, size and orientation of the detected object in the image.
`class_index`	`ulong`	Index of the detected class (see Class Names and Class Filters).
`score`	`float`	Detection score, in the range 0.0 to 1.0.

Table 6. OrientedBoundingBox (struct)
Field	Type	Unit	Description
`center_x`	`float`	pixel	Horizontal position of the box center.
`center_y`	`float`	pixel	Vertical position of the box center.
`width`	`float`	pixel	Width of the box.
`height`	`float`	pixel	Height of the box.
`angle`	`float`	degree	Orientation of the box. The current implementation can return negative angles; do not assume that the value is normalized to the range 0.0 to 360.0.

Class Names and Class Filters

With the default class mapping, class_index identifies a class by its position in the array of class names configured for the network. Use the following nodes to resolve and restrict classes:

If custom class-index remapping is configured, the published class index follows that mapping and may differ from the position in setting/class-names. Interpret the result using the configured mapping.

Table 7. Class-Related Nodes
Address	Permission	Description
`hdvisionsystems/lumiscan-ai-object-matching/<network name>/setting/class-names`	Read-Only	Class names of the network. With the default mapping, the position of a name in the array is its class index.
`hdvisionsystems/lumiscan-ai-object-matching/<network name>/input/class-filters`	Writable	Class names to include in the result. Leave the array empty to disable filtering.

Updating the Application

After an update of the application, download the schema again and re-import it into the PLC project, so that the PLC program uses the schema of the installed version.