Reading Codes on an Image Stream (Experimental)

The image stream is experimental. Its nodes and settings may still change. Use it for evaluation, and verify it with your cameras and your Data Layer before relying on it.

execute reads one image per call: your program sets input/image-address and writes to execute. An image stream works the other way round. The Cam Driver publishes every image it takes on <camera name>/control/publish/output/stream/frame, with its size, its pixel format and its lineage (see the Cam Driver manual, Streaming Images). The reader subscribes to the stream of one camera, reads every QR code in each frame as it arrives, and publishes one result per frame on control/subscribe/output/stream/result.

Each frame is read exactly as execute reads an image. execute takes no input but the image address, and the frame takes its place. input/parameter is read by neither: see Data Layer Reference.

Each result carries the lineage of its frame - which camera took it, in which run of the camera’s stream, as which frame of the run - with the reader’s own step added. See The lineage of a result.

By default the reader never falls behind its camera. When frames arrive faster than the reader reads them, it takes the newest frame each time the previous read is finished. The frames in between are counted as skipped. How many frames may wait is set with the queue size, see The queue size.

How streams work in all LumiScan apps - producers and consumers, starting and stopping, the lineage - is described in Streams of the LumiScan Keryx manual. This page describes the reader’s stream.

The stream’s nodes

The reader has the stream’s nodes whether it subscribes to a stream or not. No restart is needed to start using them. All addresses below start with hdvisionsystems/lumiscan-qr-code-reader/. There are no modules: one stream per installation.

Table 1. Methods
Address	Meaning
`control/subscribe/execute`	Checks the five inputs of `control/subscribe` together, saves them in the reader’s settings file, then applies them: a running stream follows on its next cycle, with no restart. Writing an input alone changes nothing. See Changing the source.
`control/unsubscribe/execute`	Drops the source named by `control/unsubscribe/input/address`, which ends stream mode too. The tag stays. Saved in the reader’s settings file like a subscribe.

Table 2. Inputs of control/subscribe
Address	Type	Meaning
`control/subscribe/input/enable`	bool	Stream mode: `true` runs the stream, `false` stops it.
`control/subscribe/input/address`	string	The one source: a camera’s stream folder, for example `hdvisionsystems/lumiscan-cam-driver/camera1/control/publish/output/stream`. The reader reads its `frame` node. Another address replaces the source.
`control/subscribe/input/tag`	string	The tag the reader puts on its results: the reader’s step in the lineage of each result carries it, so whoever reads a result downstream sees which module read it. Empty: the reader’s name, `lumiscan-qr-code-reader`. A new tag marks the next result; the source stays subscribed.
`control/subscribe/input/queue-size`	uint32	How many frames may wait while a frame is read: `1` to `64`. See The queue size.
`control/subscribe/input/keep-latest-output`	bool	`true` to write each stream result to `output/results` as well. See Keeping the latest output.

The inputs start with what is applied at the start of the application, so a client can change one of them and execute.

What the source must hold: read system/meta/subscription-schema. It is a JSON Schema of the frame node of the source, a stream_image with its lineage and its image, in its JSON form. See Data Layer Reference.

Table 3. Input of control/unsubscribe
Address	Type	Meaning
`control/unsubscribe/input/address`	string	The source to drop: the address `setting/stream/address` shows. A `/` at either end does not count.

Table 4. Settings
Address	Type	Meaning
`setting/stream/enable`, `setting/stream/address`, `setting/stream/tag`, `setting/stream/queue-size`, `setting/stream/keep-latest-output`	as the inputs	What `control/subscribe/execute` or `control/unsubscribe/execute` applied last, as the settings file keeps it. Read-only.
`setting/settings-hash`	string	The hash of what the results are made with, as 16 hex digits. Read-only. See The settings hash.

Table 5. Outputs
Address	Type	Meaning
`control/subscribe/output/stream/active`	bool	`true` while the stream runs.
`control/subscribe/output/stream/result`	flatbuffers `stream_codes`	The codes read from each frame, and the frame’s lineage. Subscribe to it to receive each one. See The result of a frame.
`control/subscribe/output/stream/last-failure`	string	Why the last frame that could not be read failed. It names the source by its address, for example `a frame of 'hdvisionsystems/lumiscan-cam-driver/camera1/control/publish/output/stream' failed: …`. Emptied when the stream starts.
`control/subscribe/output/stream/statistics/…`	uint64	Counters of the current or the last stream, see Reading the statistics.

execute writes to output/results, the stream to control/subscribe/output/stream/result - singular. The stream writes output/results too only with keep-latest-output on.

Setting it up

Make sure the Cam Driver’s stream runs: its hdvisionsystems/lumiscan-cam-driver/<camera name>/control/publish/output/stream/active is true.
Write the camera’s stream folder, for example hdvisionsystems/lumiscan-cam-driver/camera1/control/publish/output/stream, to control/subscribe/input/address. Leave control/subscribe/input/tag empty to mark the results with the reader’s name, or write a tag of your own to it.
Subscribe to control/subscribe/output/stream/result.
Set control/subscribe/input/enable to true, then write to control/subscribe/execute. Wait until control/subscribe/output/stream/active is true. The results arrive from then on.
To stop, set control/subscribe/input/enable to false and write to control/subscribe/execute again. The frame being read is finished first.

On ctrlX OS, make the same writes in the Data Layer view of the ctrlX OS web interface or from your PLC program.

On a PC, the same steps with krxclient, the command line client of LumiScan Keryx that stands in for a PLC (see krxclient). Add the --keryx-backend and --keryx-url options your broker needs:

krxclient write hdvisionsystems/lumiscan-qr-code-reader/control/subscribe/input/address hdvisionsystems/lumiscan-cam-driver/camera1/control/publish/output/stream
krxclient write hdvisionsystems/lumiscan-qr-code-reader/control/subscribe/input/enable true
krxclient execute hdvisionsystems/lumiscan-qr-code-reader/control/subscribe/execute
krxclient subscribe hdvisionsystems/lumiscan-qr-code-reader/control/subscribe/output/stream/result --count 5

krxclient subscribe shows a flatbuffer by its size only. It tells you that results arrive, not what they hold.

Good to know:

Stream mode is a level, not a command: the stream runs for as long as the applied enable is true.
The stream can be started before the Cam Driver’s stream. Results arrive once frames do.
What control/subscribe/execute applies is saved in the reader’s settings file. After a restart of the application the reader starts as it was: one that streamed streams again at once, without your program executing anything. See Kept in the settings file.

Changing the source

A reader takes one source. A PLC or another client changes it while the application runs, with or without a running stream:

Write the new values to the inputs of control/subscribe - only those you change: the others hold what is applied.
Write to control/subscribe/execute. The inputs are checked together, saved in the reader’s settings file, and only then applied: setting/stream/ shows them at once, and a running stream follows on its next cycle. Another address replaces the source; the frames the old one had waiting are counted as skipped. A new tag, queue size or keep-latest-output needs no new subscription: a new tag marks the next result.
To drop the source, write its address to control/unsubscribe/input/address, then write to control/unsubscribe/execute. Stream mode ends with it. The inputs of control/subscribe are set to what it applied: stream mode off, no address. The tag stays: it marks the reader’s results, not the source.

When the settings file cannot be written, nothing changes. See Kept in the settings file.

The methods answer:

ctrlX Data Layer LumiScan Keryx When

ctrlX Data Layer	LumiScan Keryx	When
`DL_OK`	`Ok`	Done.
`DL_INVALID_VALUE`	`Rejected`	`control/subscribe/execute`: stream mode without an address, or a queue size outside `1` to `64`. `control/unsubscribe/execute`: the address is not the source, or there is no source.
`DL_CREATION_FAILED`	`CreationFailed`	The settings file could not be written, for example because it is not valid JSON. Nothing changed.

DL_OK

Ok

Done.

DL_INVALID_VALUE

Rejected

control/subscribe/execute: stream mode without an address, or a queue size outside 1 to 64. control/unsubscribe/execute: the address is not the source, or there is no source.

DL_CREATION_FAILED

CreationFailed

The settings file could not be written, for example because it is not valid JSON. Nothing changed.

The result of a frame

control/subscribe/output/stream/result holds a FlatBuffers table of type stream_codes (file identifier HSCD), registered on the Data Layer as types/hdvisionsystems/lumiscan-qr-code-reader/stream_codes. Its compiled schema is installed as bfbs/stream_codes.bfbs.

Field Meaning

Field	Meaning
`lineage`	Where the result came from, see The lineage of a result.
`result`	The codes, as a `CodeDetectResultArray`: the same table `execute` writes to `output/results` for the same picture, described in The Result. A frame with no QR code gives an empty `results` array.

lineage

Where the result came from, see The lineage of a result.

result

The codes, as a CodeDetectResultArray: the same table execute writes to output/results for the same picture, described in The Result. A frame with no QR code gives an empty results array.

A read of control/subscribe/output/stream/result returns the latest result; before the first frame, an empty one. The last result stays after the stream stops.

A frame is decoded as execute decodes the image node, so the same pixel formats are accepted, see Requirements. The whole frame is searched. A frame and the same picture read through input/image-address give the same codes.

A frame that cannot be read - for example one in a pixel format the reader does not support, or one whose pixels do not fill the size it gives - gives no result. It is counted in statistics/failed-frames, and control/subscribe/output/stream/last-failure says why.

The lineage of a result

lineage.steps lists where the result came from, origin first:

The camera’s step, as the Cam Driver made it: producer is the camera’s address, hdvisionsystems/lumiscan-cam-driver/<camera name>; run when the camera’s stream started; sequence the frame’s number in that run; external_id the camera’s own frame number; settings_hash the camera’s settings; tag the Cam Driver’s tag for the camera, by default its module id, for example camera1.
The reader’s step: producer is the reader’s address, hdvisionsystems/lumiscan-qr-code-reader; run when the reader’s stream started (each start is a new run); sequence the result’s number in the run, from 1; settings_hash the reader’s settings, see The settings hash; elapsed_us the microseconds from the frame’s arrival at the reader to its result, waiting included; time_us when the result was made, in microseconds since the epoch; tag the reader’s tag, control/subscribe/input/tag, or lumiscan-qr-code-reader when it is empty.

So a result names its frame exactly: the camera by producer, the frame by run and sequence. To know which camera a result came from, read the producer of its first step. A gap in the camera step’s sequence between two results of the same camera is frames that were skipped, failed or lost.

A lineage holds at most 16 steps. A longer one keeps the origin and the newest steps, and dropped counts the steps left out.

A frame that carries no lineage gives a result with the reader’s step alone. A value on a stream’s frame node that is not a stream_image is ignored, and logged once.

The settings hash

setting/settings-hash, and the settings_hash of the reader’s step in each result, identify what the results are made with. The node shows the hash as 16 hex digits; the step holds the same value as a number.

The reader has no setting that changes a read, so the hash covers only the code type and the application’s name and version: it changes when the application is updated. It does not cover stream mode, the source, the queue size or keep-latest-output, which choose and pace the frames.

The first time a hash appears, the reader writes the settings it stands for to settings-history/qr-code-reader-<hash>.json in the application’s configuration directory, so a hash found in a result can be looked up later. The application does not delete these files. A file looks like this:

{
    "format": 1,
    "settings": {
        "code-type": "qr-code",
        "software.name": "lumiscan-qr-code-reader",
        "software.version": "<version>"
    },
    "settingsHash": "<16 hex digits>"
}

Reading the statistics

The counters start from zero each time the stream starts, are updated about four times a second while it runs, and keep their last values after it stops. All addresses start with control/subscribe/output/stream/statistics/.

Counter Meaning

Counter	Meaning
`elapsed-ms`	Milliseconds since the stream started. Divide a counter by it for a rate.
`received-frames`	Frames the subscription delivered.
`processed-frames`	Frames read, each with a result published on `control/subscribe/output/stream/result`.
`skipped-frames`	Frames never read: pushed out of the queue by newer ones while a frame was read (see The queue size), or waiting when the source was replaced or dropped.
`failed-frames`	Frames that could not be read. `control/subscribe/output/stream/last-failure` says why the last one failed, and the log names the failures, at most one line every 5 seconds.
`lost-frames`	Frames dropped before they arrived, counted from gaps in the camera’s `sequence` within one run of its stream. A new source, or a new run of the camera’s stream, starts the count again.
`processing-us`	Microseconds all reads took, summed. Divide by `processed-frames` for the mean.

elapsed-ms

Milliseconds since the stream started. Divide a counter by it for a rate.

received-frames

Frames the subscription delivered.

processed-frames

Frames read, each with a result published on control/subscribe/output/stream/result.

skipped-frames

Frames never read: pushed out of the queue by newer ones while a frame was read (see The queue size), or waiting when the source was replaced or dropped.

failed-frames

Frames that could not be read. control/subscribe/output/stream/last-failure says why the last one failed, and the log names the failures, at most one line every 5 seconds.

lost-frames

Frames dropped before they arrived, counted from gaps in the camera’s sequence within one run of its stream. A new source, or a new run of the camera’s stream, starts the count again.

processing-us

Microseconds all reads took, summed. Divide by processed-frames for the mean.

Frames still waiting when the stream stops are dropped without being counted.

Kept in the settings file

Stream mode and the source are settings of the reader. They are kept in the stream member of the reader’s settings file, qr-code-reader.json in the application’s configuration directory.

control/subscribe/execute and control/unsubscribe/execute save what they apply there themselves, all five values in one write, before they apply them. When the file cannot be written, the method answers CreationFailed (ctrlX DL_CREATION_FAILED) and nothing changes. After a restart the reader subscribes to the same source and, when stream.enable is true, starts its stream at once: nobody needs to execute anything.

Saving changes only the stream member; the rest of the file stays as it was, comments included. The reader makes the file when there is none.

The configuration directory is:

On ctrlX OS: /var/snap/rexroth-solutions/common/solutions/activeConfiguration/lumiscan-qr-code-reader.

On a PC (Windows): %APP_RESOURCE_DIR%\etc\lumiscan-qr-code-reader; without APP_RESOURCE_DIR, C:\ProgramData\HD Vision Systems GmbH\LumiScan Vision Suite\lumiscan-qr-code\etc\lumiscan-qr-code-reader.

The file holds one member, stream. An example - on a PC, this is the example the application writes with --generate-settings, see Starting from an Example:

{
    // The settings of the QR Code Reader, read once, at the start: the image stream it reads codes in.
    // control/subscribe/execute and control/unsubscribe/execute save what they change here before
    // they apply it, the rest of the file left as it is, comments included.
    "$schema": "schemas/qr-code-reader.schema.json",
    "stream": {
        // Stream mode: read every frame of the source as it arrives. Off if left out.
        "enable": false,
        // The one image stream to read: its folder.
        "address": "hdvisionsystems/lumiscan-cam-driver/camera1/control/publish/output/stream",
        // "tag" (left out here): what marks the codes it reads, in their lineage - the reader's name,
        // lumiscan-qr-code-reader, if left out.
        // How many frames may wait while a frame is read: 1 to 64, 1 if left out.
        "queueSize": 1,
        // Each stream result also on output/results, as execute writes it. Off if left out.
        "keepLatestOutput": false
    }
}

Key Required Meaning

Key	Required	Meaning
`enable`	No	Stream mode: `true` starts the stream at the start of the application. Needs an `address`. `false` if left out.
`address`	No	The one source: a camera’s stream folder.
`tag`	No	What marks the reader’s results: the `tag` of the reader’s step in their lineage. The reader’s name, `lumiscan-qr-code-reader`, if left out.
`queueSize`	No	How many frames may wait: `1` to `64`, `1` if left out.
`keepLatestOutput`	No	`true` to write each stream result to `output/results` as well. `false` if left out.
`onEntry`	No	The steps applied to each frame as it arrives, each an object with a `kind`, as every LumiScan application’s `stream` has them. This application has none yet: leave it out, or empty (`[]`). A step starts the reader out of stream mode, with a warning in the log; saving keeps it.

enable

Stream mode: true starts the stream at the start of the application. Needs an address. false if left out.

address

The one source: a camera’s stream folder.

tag

What marks the reader’s results: the tag of the reader’s step in their lineage. The reader’s name, lumiscan-qr-code-reader, if left out.

queueSize

How many frames may wait: 1 to 64, 1 if left out.

keepLatestOutput

true to write each stream result to output/results as well. false if left out.

onEntry

The steps applied to each frame as it arrives, each an object with a kind, as every LumiScan application’s stream has them. This application has none yet: leave it out, or empty ([]). A step starts the reader out of stream mode, with a warning in the log; saving keeps it.

control/subscribe/execute and control/unsubscribe/execute write all five.

Good to know:

The file is read once, at the start of the application. Changes made to the file by hand take effect at the next start.
A value that cannot be used is ignored, and logged: an enable or keepLatestOutput that is not true or false, an address or tag that is not a string, a queueSize outside 1 to 64 (1 is used). Stream mode without an address is ignored: the reader starts out of stream mode.
The subscriptions member of earlier versions is no longer read, and the reader logs that it moved: give the source as address.
A file that is not valid JSON is ignored, and logged.

The queue size

control/subscribe/input/queue-size sets how many frames may wait while a frame is read: 1 to 64, 1 by default. It is saved as stream.queueSize, and setting/stream/queue-size shows the value in use. A frame that arrives when that many wait pushes out the oldest of them, which is counted in statistics/skipped-frames. A new queue size applies from the next frame that arrives, with no new subscription.

Queue size	Effect
1 (default)	The reader reads the newest frame. It never falls behind; when reading is slower than the camera, it skips frames.
More than 1	Short bursts of frames are all read, and fewer are skipped. When reading is slower than the camera for longer, the reader falls behind: it reads frames up to that many reads old. Each waiting frame holds a whole image in memory.

Queue size

Effect

1 (default)

The reader reads the newest frame. It never falls behind; when reading is slower than the camera, it skips frames.

More than 1

Short bursts of frames are all read, and fewer are skipped. When reading is slower than the camera for longer, the reader falls behind: it reads frames up to that many reads old. Each waiting frame holds a whole image in memory.

Keeping the latest output

With control/subscribe/input/keep-latest-output true - applied by control/subscribe/execute, saved as stream.keepLatestOutput - the reader also writes each stream result to output/results, as execute writes it, once the result is published on control/subscribe/output/stream/result. A client that reads output/results then has the newest frame’s result without subscribing to the stream. It costs a copy of each result. Off by default.

execute while the stream runs

execute and the stream do not hold each other up. Each frame is read by a detector of its own, as each call of execute is, and nothing is shared between the two: execute can be called at any time while the stream runs, and a frame never waits for it.

Limitations

input/parameter is read neither by execute nor by the stream.
(ctrlX OS, Ubuntu) The stream has not been verified on Linux, on the ctrlX Data Layer, or with a real camera.
(PC) The stream has been tested on Windows with LumiScan Keryx and simulated images only. It has not been verified with a real camera.
LumiScan Cockpit does not draw the stream’s results yet.
Changes made to the settings file by hand take effect at the next start of the application.

Starting from an Example (PC)

On a PC, the QR Code Reader writes an example of its settings into a folder of your choice, and ends without starting. For example (Windows):

lumiscan-qr-code --generate-settings C:\work\qr-code-reader-settings

Without a folder (-g alone) it writes into the current folder. The example is the settings file qr-code-reader.json: stream mode off, the stream of the Cam Driver’s camera1 as its source, the queue size it has by default and keep-latest-output off. Beside it, in schemas/, are the schema it names in "$schema", qr-code-reader.schema.json, and stream.schema.json, which describes the stream section; an editor that understands JSON Schema can check the file against them. Copy the schemas folder along with it. Existing files are never overwritten: when one of them is in the folder already, nothing is written and the QR Code Reader says which.

The QR Code Reader then names the folder it reads its settings from, on Windows by default C:\ProgramData\HD Vision Systems GmbH\LumiScan Vision Suite\lumiscan-qr-code\etc\lumiscan-qr-code-reader. Edit the example, copy it there, and start the QR Code Reader.