---
title: "Streaming Images (Experimental)"
product: "LumiScan Cam Driver"
component: "lumiscan-cam-driver"
version: "2.2.1"
display_version: "2.2.1.10"
latest_version: true
language: "en"
url: "https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html"
---

# Streaming Images (Experimental) – LumiScan Cam Driver

> [!IMPORTANT]
> The image stream is **experimental** (from v2.3.0). Its nodes may still change. Use it for evaluation, and verify it with your camera before relying on it.

`capture` delivers one image per request, and each request waits until its image has arrived. On the camera we tested this gives about 34 images per second, well below what the camera itself can deliver: the next exposure cannot start before the previous image has been read out and sent.

The image stream works the other way round. While it runs, the Cam Driver publishes **every** image the camera delivers on `<camera name>/control/publish/output/stream/frame`. Your program subscribes to that node and is sent each image as it arrives - with its size, its pixel format and where it came from, see [What each frame carries](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html#frames).

The stream is switched on and off by one function, `control/publish`: set its inputs, then call `control/publish/execute` to apply them. See [Running a stream](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html#running-a-stream).

The general stream concept - producers, consumers and the lineage of their results - is described in [Streams](https://docs.hdvisionsystems.com/en/lumiscan-keryx/1.0.0/streams.html). This page describes the Cam Driver’s stream.

The Cam Driver does not trigger the camera by itself. An image comes when the camera is triggered - by a signal on its trigger input, or by `control/trigger/execute` - or, with the camera’s trigger mode off, as fast as the camera runs. See [Triggering the camera](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html#triggering).

**Table 1. What a subscriber receives**

| Data Layer | Images delivered to a subscriber of `control/publish/output/stream/frame` |
| --- | --- |
| ctrlX Data Layer (ctrlX OS) | Every image, in order, as soon as it is published, whatever the subscription’s publish interval. Images are dropped only when the Data Layer falls far behind. |
| LumiScan Keryx (PC) | Every image, in order. The Data Layer drops images only when a subscriber falls far behind. |

Each frame carries its number in the stream, so a subscriber sees a dropped image as a gap. A read of `control/publish/output/stream/frame` always returns the latest image.

## Addresses {#addresses}

All addresses below start with `hdvisionsystems/lumiscan-cam-driver/<camera name>/`. They follow the rule of every `execute` method: the method owns its nodes, `control/publish/execute`, `control/publish/input/…` and `control/publish/output/…`.

**Table 2. The method and its inputs**

| Address | Type | Default | Meaning |
| --- | --- | --- | --- |
| `control/publish/execute` | method |   | Applies the inputs below together. Writing an input alone changes nothing. Returns at once: the stream starts or stops within the Cam Driver’s next cycle, and `control/publish/output/stream/active` says when it runs. Refused with `DL_RESOURCE_UNAVAILABLE` while the Cam Driver shuts down, or after its settings were changed on the Data Layer (`system/restart-required`). Refused with `DL_CREATION_FAILED` when a changed `tag` cannot be saved: nothing changes then. |
| `control/publish/input/enable` | bool | `false` | `true` runs the stream while the camera is connected; `false` stops it. |
| `control/publish/input/keep-latest-output` | bool | `false` | `true` copies every streamed image to `output/image` as well: `data`, `width`, `height`, `pixel-type` and `timestamp` (the time the image arrived), and then increases `output/package-identifier` by one, as `capture` does. Each copy takes time, which slows a fast stream down. Applied to a running stream without restarting it. See [Capture while the stream runs](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html#capture-while-streaming). |
| `control/publish/input/tag` | string | `stream.tag` of the module configuration file, or empty | The tag on the module’s images: the camera’s step in their lineage carries it, and so do the results of every application that processes them. Empty: the camera name, for example `camera1`. Give it another, for example the station the camera looks at, to see that in every result. Applied to a running stream without restarting it. See [The tag](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html#tag). |

What `execute` applied is not saved, except the tag: after the Cam Driver starts, the stream is off until `control/publish/execute` is called with `enable` `true`.

**Table 3. The method’s outputs, under `control/publish/`**

| Address | Type | Meaning |
| --- | --- | --- |
| `output/stream/active` | bool | `true` while the stream runs. |
| `output/stream/frame` | flatbuffers (`stream_image`) | Each streamed image: the pixels, their size and format, and the image’s lineage. Subscribe to it to receive the images. See [What each frame carries](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html#frames). |
| `output/stream/width`, `output/stream/height` | uint64 | Size of the streamed images in pixels, for browsing. Each frame carries its own. |
| `output/stream/pixel-type` | string | Pixel format of the streamed images, for browsing. Each frame carries its own. |
| `output/stream/statistics/…` | uint64 | Counters of the current or last stream, see [Reading the statistics](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html#statistics). |

**Table 4. Beside the method**

| Address | Type | Meaning |
| --- | --- | --- |
| `setting/settings-hash` | string | The hash of the settings the images are taken with, as 16 hex digits. See [The settings hash](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html#settings-hash). |
| `setting/stream/tag` | string | The tag `control/publish/execute` applied last, as the images carry it. Empty: the images carry the camera name. |

### The tag {#tag}

A changed `control/publish/input/tag` is saved in the module configuration file, as `stream.tag`, before `control/publish/execute` applies it, the rest of the file as it was. So the images carry it after a restart too, and `control/publish/input/tag` starts with it. If the file cannot be written, `control/publish/execute` answers `DL_CREATION_FAILED`, and neither the tag nor the other inputs change. An unchanged tag is not saved, so a module whose file cannot be written still streams.

The tag is not part of the settings hash: it names the images, it does not change them.

## Triggering the camera {#triggering}

Each trigger the camera receives gives one image on `control/publish/output/stream/frame`. Set the camera’s trigger mode before you start the stream, through its feature nodes under `features` or in a feature file.

| Images come | Camera settings | What to do |
| --- | --- | --- |
| On a hardware trigger | `TriggerMode` `On`, `TriggerSource` a trigger input, for example `Line0` | Send a pulse to the trigger input for each image. |
| On request | `TriggerMode` `On`, `TriggerSource` `Software` | Call `control/trigger/execute` once for each image. The Cam Driver sends the camera’s software trigger, as hdv’s feature mapping names it. |
| Continuously | `TriggerMode` `Off` | Nothing. The camera delivers as fast as it runs, or at its `AcquisitionFrameRate` where that is enabled. |

A camera without a software trigger mode - a simulated camera playing a folder of images, or a plugin camera without `TriggerSoftware` - takes one image for each call to `control/trigger/execute` as well.

`control/trigger/execute` answers when the trigger has been sent, not when its image has arrived. A camera that is still busy with the previous image can ignore a trigger without reporting it, see [For the highest frame rate: TriggerOverlap](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html#trigger-overlap).

### For the highest frame rate: TriggerOverlap {#trigger-overlap}

With the GenICam feature `TriggerOverlap` set to `Off`, a camera takes the next image only once the previous one has been read out, and it ignores a trigger that comes earlier. An overlap lets the next exposure start while the previous image is still read out or sent, which raises the frame rate the camera reaches.

**To stream at the highest frame rate, `TriggerOverlap` must not be `Off`.** Set it to an overlap your camera supports - `PreviousFrame` or `ReadOut`, see its documentation - in a feature file of the camera module, one feature per line:

```none
TriggerOverlap PreviousFrame
```

List the file in the module’s `cameras[].featureFiles`. The Cam Driver applies it each time it connects the camera, before acquisition starts, so it holds for `capture` as well. A feature node under `features` works too, but some cameras refuse trigger features while they acquire, and the value is lost when the camera is connected again.

## What each frame carries {#frames}

`control/publish/output/stream/frame` holds a FlatBuffers table of type `stream_image`, registered on the Data Layer as `types/hdvisionsystems/lumiscan-cam-driver/stream_image`. Its compiled schema is installed as `bfbs/stream_image.bfbs`.

| Field | Meaning |
| --- | --- |
| `image.data` | The pixels, row after row, without padding. |
| `image.width`, `image.height` | The image size in pixels. |
| `image.pixel_type` | The pixel format, for example `Mono8`. |
| `lineage.steps` | Where the image came from. For a camera image this is one step, the camera’s own:<br><br>\* `producer`: the camera’s address, `hdvisionsystems/lumiscan-cam-driver/<camera name>`. \* `tag`: the module’s tag, `setting/stream/tag`, or the camera name when it is empty. See [The tag](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html#tag). \* `run`: when the stream started, in microseconds since 1970. Each start of the stream is a new run. \* `sequence`: the image’s number in the run, from 1. A gap means images were lost on the way. \* `external_id`: the camera’s own frame number (the GigE Vision block id). A gap here without a gap in `sequence` means images were lost between the camera and the Cam Driver. \* `time_us`: when the image was published, in microseconds since 1970. \* `settings_hash`: the settings the image was taken with, see [The settings hash](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html#settings-hash). |

The applications that process the stream - AI Object Matching, Calibration - copy the lineage of what they receive and add a step of their own, with their own tag. Their results therefore still name the camera image they came from, by `producer`, `run` and `sequence`, and each step says by its `tag` which module made it.

## The settings hash {#settings-hash}

`setting/settings-hash` and the `settings_hash` of each frame identify the settings the camera works with. The hash changes when one of them changes:

- the module settings and the feature files, as the camera was connected with;
- every camera feature written while the camera is connected, for example `ExposureTime` through `input/exposure-time` or a feature node under `features`, with the value the camera accepted;
- the Cam Driver’s version.

Settings that only affect how images are transported, such as the timeout or the packet size, do not change the hash. The first time a hash appears, the Cam Driver writes the settings it stands for to `settings-history/<camera name>-<hash>.json` in its configuration folder, so a hash found in a result can be looked up later.

> [!NOTE]
> A camera takes the images that are already on their way with the old settings. The images that arrive right after a feature was written can carry the new hash although they were taken before the change.

### What the hash does not tell {#settings-hash-limits}

- **That, not what.** A frame carries the hash only. A different hash says that something the images are taken with changed, not what changed. To find out, compare the two files in `settings-history` that the hashes name, by hand. The files are on the device the Cam Driver ran on.
- **Only what goes through the Cam Driver.** The hash follows a feature node under `features`, `input/exposure-time`, and the settings and feature files the camera was connected with. It does not follow values the camera changes by itself: with `ExposureAuto` on, the hash follows the `ExposureAuto` setting, not the exposure times the camera picks.

> [!IMPORTANT]
> That the hash follows a camera feature change has so far been tested only in unit tests, with a stand-in for the camera, not with a real camera. Until it has been, do not rely on the camera’s settings hash alone to show that a camera feature changed.

## Running a stream {#running-a-stream}

1. Connect the camera with the `connect-camera` handshake. See [Connection Steps](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/initializing-the-camera.html#connection-steps).
2. Set the camera features you need, such as the exposure time, the trigger mode and `TriggerOverlap`, **before** starting the stream. See [Triggering the camera](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html#triggering).
3. Subscribe to `control/publish/output/stream/frame`.
4. Set `control/publish/input/enable` to `true`, and `control/publish/input/keep-latest-output` to `true` if you also want each image on `output/image`.
5. Call `control/publish/execute`, and wait until `control/publish/output/stream/active` is `true`.
6. Trigger the camera. Each image arrives on `control/publish/output/stream/frame`.
7. To stop, set `control/publish/input/enable` to `false` and call `control/publish/execute` again. `control/publish/output/stream/active` turns `false` within the next cycle.
8. Before capturing again, wait until `control/capture/ready` is `true`. See [After the stream](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html#after-the-stream).

Good to know:

- The inputs take effect only on `execute`, all together. A program can therefore set them in any order, and nothing changes half-way.
- What `execute` applied stays until the next `execute`: the stream runs for as long as `enable` was applied `true` and the camera is connected. Applied while the camera is disconnected, the stream starts as soon as the camera is connected.
- `connect-camera`, `disconnect-camera` and `reset` stop a running stream. If `enable` was applied `true`, the stream starts again once the camera is connected.
- If the stream cannot start, `control/publish/output/stream/active` stays `false` and the reason is written to the log. It is not retried until `control/publish/execute` is called again.
- A camera ignores triggers that come faster than it can take images, without reporting it. If `statistics/frames` grows more slowly than the number of triggers you sent, trigger less often or set an overlap, see [For the highest frame rate: TriggerOverlap](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/streaming-images.html#trigger-overlap).
- The exposure time on `input/exposure-time` and the inter-packet delay are not applied while the stream runs. Change camera features only while the stream is stopped.

## Capture while the stream runs {#capture-while-streaming}

The stream and `capture` use the camera in different ways, so they never run at the same time:

- `control/capture/ready` is `false` while the stream runs, so a program using the handshake waits.
- `control/capture/execute` answers `DL_INVALID_OPERATION_MODE` while the stream runs, and `DL_TOO_MANY_OPERATIONS` while the stream is starting or stopping. Do not retry `DL_INVALID_OPERATION_MODE` in a loop. Stop the stream and wait for `capture/ready` instead. See [Execute a Method](https://docs.hdvisionsystems.com/en/lumiscan-cam-driver/2.2.1/lumiscan-camdriver/execute-a-method.html).
- A capture request that reaches the Cam Driver while the stream runs is refused with status `-3` (`invalidInput`), with the text "capture refused: the image stream (experimental) is active". It does **not** latch a fault, so no `reset` is needed.
- A stream that is switched on while a capture runs starts once that capture has finished. It never cuts a capture short.

Streamed images are published on `control/publish/output/stream/frame`. They reach `output/image` only while `keep-latest-output` is applied `true`: then each one replaces the image there, and `output/package-identifier` counts it as it counts a capture, continuing from the last capture’s number. `save` keeps saving the last image taken with `capture` or `load`, also then.

## After the stream {#after-the-stream}

While streaming, the camera delivers its images in a different acquisition mode from the one `capture` uses. To switch back, the Cam Driver disconnects the camera and connects it again when the stream stops, applying every connection setting as `connect-camera` does. `capture/ready` turns `true` once that has finished. Until then, `capture/execute` answers `DL_TOO_MANY_OPERATIONS`.

Starting and stopping therefore take different times. On the camera we tested:

| Step | Took | Mostly spent on |
| --- | --- | --- |
| Starting the stream | about 0.35 s | Switching the camera’s acquisition mode. |
| Stopping the stream, until `capture/ready` is `true` again | about 2.2 s | Connecting the camera again: opening the device (about 0.8 s) and reading all of its GenICam features for the Data Layer (about 0.8 s). |

So switch between streaming and capturing as rarely as you can. A program that needs single images while streaming can take them from `control/publish/output/stream/frame` instead of stopping the stream, or apply `keep-latest-output` and read `output/image` when `output/package-identifier` changes.

## Measured frame rates {#measured-frame-rates}

These figures were measured with a LUCID PHX016S (1440 x 1080, Mono8, exposure 1 ms) and LumiScan Keryx on one PC. They show the order of magnitude only: your camera, exposure time, network and Data Layer decide what you reach.

They were measured while the Cam Driver still triggered the camera itself, at a fixed rate. A hardware trigger or a continuously running camera is expected to give the same. Triggering with `control/trigger/execute` adds one method call per image; that has not been measured yet.

| Method | Images per second delivered |
| --- | --- |
| `capture/execute` in a loop | about 34 |
| Stream, `TriggerOverlap` `Off` | up to 63, every image delivered |
| Stream, `TriggerOverlap` `PreviousFrame` | up to 66, every image delivered |
| What the camera itself can deliver (`AcquisitionFrameRate`) | 71.7 |

## Reading the statistics {#statistics}

The counters, under `control/publish/output/stream/statistics/`, start from zero each time the stream starts, and are updated about four times a second while it runs.

| Counter | Meaning |
| --- | --- |
| `elapsed-ms` | Milliseconds since the stream started, measured by the Cam Driver. Divide a counter by it for a rate. |
| `frames` | Complete images the camera delivered. |
| `incomplete-frames` | Images the camera delivered incomplete. They are not published. |
| `lost-frames` | Images that never reached the Cam Driver, counted from gaps in the camera’s frame numbers. |
| `published-frames` | Images written to `control/publish/output/stream/frame`. |