Streams (Experimental)

Streams are experimental: in the Cam Driver from v2.3.0, in AI Object Matching from v2.2.0 and in Calibration from v2.1.0. Their nodes may still change. Verify a stream with your camera and network before you rely on it.

A stream processes every item a source produces, as it arrives: camera frames into AI Object Matching, a finder or a code reader; detections or points into Calibration. There is no request per item. The producer pushes each item, a consumer processes it and pushes its result. Producer and consumer work at the same time, which is why a stream is faster than one execute call per item.

Producers and consumers

App Role

Cam Driver

produces frames on <camera>/control/publish/output/stream/frame

AI Object Matching

consumes frames, produces detections on <module>/control/subscribe/output/stream/result

Template Finder, Object Finder, Area Detector, Text Reader

consume frames, produce their results on <module>/control/subscribe/output/stream/result

DMC, QR, Aztec and Barcode Reader

consume frames, produce codes on <app>/control/subscribe/output/stream/result

Calibration

consumes points or another app’s result, produces 3D points on <module>/control/subscribe/output/stream/points and plain-points

A consumer has one source. To process two cameras, use two modules.

The producer: the Cam Driver

All addresses start with hdvisionsystems/lumiscan-cam-driver/<camera>/.

Address Type Meaning

control/publish/execute

method

applies the inputs below together

control/publish/input/enable

bool

publish while true

control/publish/input/keep-latest-output

bool

also write each frame to the normal output

control/publish/input/tag

string

the tag on its frames; saved in the module file

control/publish/output/stream/frame

FlatBuffers

each frame, pushed

control/publish/output/stream/active

bool

the stream runs

control/publish/output/stream/statistics/…​

uint64

counters

The Cam Driver never triggers the camera by itself. A frame comes when the camera is triggered by hardware or by control/trigger/execute, or when its trigger mode is off and it runs freely. The stream needs a connected camera; connect-camera, disconnect-camera and reset stop a running stream. See the Cam Driver manual.

A consumer

The same nodes in every consumer, under <module>/ (a code reader: under <app>/):

Address Type Meaning

control/subscribe/execute

method

checks, saves and applies the inputs below

control/subscribe/input/enable

bool

stream mode

control/subscribe/input/address

string

the one source: the producer’s stream folder

control/subscribe/input/tag

string

the tag on this module’s output; empty: the module id

control/subscribe/input/queue-size

uint32

how many items may wait

control/subscribe/input/keep-latest-output

bool

also write each result to the normal output

control/unsubscribe/execute

method

drops the source named by …​

control/unsubscribe/input/address

string

…​ this address; another address is refused

setting/stream/…​

read-only

what is applied and saved: enable, address, tag, queue-size, keep-latest-output

control/subscribe/output/stream/active

bool

the stream runs

control/subscribe/output/stream/<result>

FlatBuffers

each result, pushed

control/subscribe/output/stream/statistics/…​

uint64

counters, published every 250 ms, reset at each start

The finders and code readers also have control/subscribe/output/stream/last-failure: why the last item failed.

What a consumer’s source must hold is on hdvisionsystems/<app>/system/meta/subscription-schema, as a JSON Schema.

Starting and stopping a stream

The inputs take effect only on execute. Writing enable alone starts nothing.

  1. Start the producer: on the camera write control/publish/input/enable = true, then call control/publish/execute. Wait until control/publish/output/stream/active is true.

  2. On the consumer write control/subscribe/input/address, for example hdvisionsystems/lumiscan-cam-driver/camera1/control/publish/output/stream.

  3. Optionally write tag, queue-size and keep-latest-output.

  4. Write control/subscribe/input/enable = true, then call control/subscribe/execute.

  5. Wait until control/subscribe/output/stream/active is true. Results arrive from then on.

  6. To stop, write enable = false and call execute again.

Good to know:

  • Stream mode is a level, not a command: the stream runs for as long as the applied enable is true.

  • The inputs start with what is applied, so change only what you need and call execute.

  • A consumer can be started before its producer. Results arrive once items do. A subscription that fails is tried again every 5 s while the stream runs.

  • Another address replaces the source.

  • enable = true without an address is refused (Rejected, ctrlX DL_INVALID_VALUE), and nothing changes.

Calibration as a consumer

Calibration is given a stream folder too, and its module file says which node in it to read and how to turn it into points: an onEntry step in the stream section. To read the objects AI Object Matching finds, the step names the node result, the list in it and the fields that are the point. Without such a step Calibration reads its source as a plain array of numbers, as a PLC variable delivers them. See the Calibration manual.

Keep latest output

With keep-latest-output on, every item - or every result - is also written to the module’s normal output, as an on-request call would write it, and output/package-identifier is counted. A PLC that reads the normal output then sees the latest streamed result. With it off, the stream publishes only under …​/stream/…​, and the normal output keeps the result of the last call.

Stream results are FlatBuffers records. For a PLC, switch on keep-latest-output in the module whose result you need and read its plain output nodes. Calibration also publishes plain-points: all points of one result in one float32 array x0, y0, z0, x1, …​, without lineage.

What is kept across a restart

What Kept How

A consumer’s stream (enable, address, tag, queue-size, keep-latest-output)

yes

control/subscribe/execute and control/unsubscribe/execute save them in the stream section of the module’s settings file before they apply them. A consumer whose saved enable is true subscribes by itself at start

The Cam Driver’s stream

no

only its tag is saved. After every start call control/publish/execute with enable = true again

The camera connection

no

run the connect-camera handshake after every start

A file that cannot be written makes execute answer CreationFailed (ctrlX DL_CREATION_FAILED), and nothing changes.

The stream section of a consumer’s settings file:

"stream": {
    "enable": false,
    "address": "hdvisionsystems/lumiscan-cam-driver/camera1/control/publish/output/stream",
    "tag": "",
    "queueSize": 1,
    "keepLatestOutput": false,
    "onEntry": []
}

Queue size

App Range Default When the queue is full

AI Object Matching, the finders, the code readers

1 to 64

1

a new item pushes out the oldest

Calibration

1 to 4096

64

a new item pushes out the oldest waiting one

An item pushed out of a full queue counts as skipped.

Streams and single operations

While a stream runs, operations it would collide with are refused:

  • Cam Driver: capture/ready is false; capture/execute answers InvalidOperationMode (ctrlX DL_INVALID_OPERATION_MODE). Do not retry in a loop. To take single images while streaming, read the camera’s …​/stream/frame, or switch on keep-latest-output and read output/image when output/package-identifier changes.

  • AI Object Matching: inference/ready is false; a handshake request ends with status 100002.

  • Finders: a method called while a frame is processed is refused as busy.

  • Code readers: execute and the stream do not hold each other up; each frame is read by a detector of its own.

Stopping the Cam Driver’s stream makes it reconnect the camera so that capture works again; this takes about two seconds. Wait until capture/ready is true. Switch between stream and capture as rarely as you can.

Following the chain

Each app counts what it receives under …​/stream/statistics/. The first counter that does not grow shows where a chain stops:

App Counters

Cam Driver

frames, published-frames, lost-frames, incomplete-frames

AI Object Matching

received-frames, inferred-frames, skipped-frames, failed-frames, lost-frames

Finders, code readers

received-frames, processed-frames, skipped-frames, failed-frames, lost-frames

Calibration

received-updates, transformed-updates, skipped-updates, failed-updates, lost-updates

lost- counts items that never arrived, found from gaps in the sequence numbers; skipped- items pushed out of a full queue; failed-* items that could not be processed.

The result: an envelope with its lineage

Every stream result is an envelope: the item’s lineage, then the payload - the same result as the app’s non-stream output, with a few differences described in each app’s manual.

The lineage lists every module the item passed. Each step has:

Field Meaning

producer

the Data Layer address of the module, for example hdvisionsystems/lumiscan-cam-driver/camera1

tag

the module’s tag, or its module id when the tag is empty

run

when the producer’s stream started (µs since the epoch); a new run numbers from 1 again

sequence

the item’s number in the run: 1, 2, …​ without gaps, so a gap is a lost item

time_us

when the step was made

settings_hash

the hash of the settings the step was made with, as on setting/settings-hash

elapsed_us

how long the step took

external_id

the producer’s own number for the item, for example the camera’s block id

The lineage keeps at most 16 steps: the origin and the newest. A changed settings_hash means a setting changed. Each hash is saved once in settings-history/ in the app’s settings folder, so the settings behind a result can be looked up.

To follow one image through the chain on a PC, use LumiScan Keryx Utilities, see Tools.