How To: Find a Part from a Camera Image

Start to finish: teach a part from a live camera, find it, and make the settings survive a restart.

Before you start

  • LumiScan Cam Driver is installed and publishing an image, for example at hdvisionsystems/lumiscan-cam-driver/camera1/output/image.

  • LumiScan Object Finder is installed and licensed, and system/modules lists module-1.

  • The part is in front of the camera, in the lighting the installation will actually use.

Step 1: teach the first view

Look at the live image and note the pixel rectangle the part occupies - the ctrlX web interface shows the image, or read width and height from the image node.

Set the inputs and execute:

system/create-object/input/input-image-address = hdvisionsystems/lumiscan-cam-driver/camera1/output/image
system/create-object/input/object-id           = screw-m6
system/create-object/input/crop/startX         = 820
system/create-object/input/crop/startY         = 400
system/create-object/input/crop/width          = 160
system/create-object/input/crop/height         = 120
write system/create-object/execute

Check system/create-object/output/status-code is 1, and output/object-path names the file. system/objects should now include screw-m6.

Crop tightly. Background inside the template is background the detector will expect to see.

Step 2: teach the other orientations

The detector does not rotate a picture, so each orientation the part arrives in needs its own object. Repeat step 1 with a different object-id - screw-m6-tilted, say - and search for it in its own run.

If the part only ever arrives one way up, one object is enough.

Step 3: find it

module-1/find-object/input/input-image-address = hdvisionsystems/lumiscan-cam-driver/camera1/output/image
module-1/find-object/input/object-id           = screw-m6
module-1/find-object/input/nms-min-dist        = 40
write module-1/find-object/execute

Read module-1/find-object/output/status-code. 1 means the run completed; then read output/result.

nms-min-dist at roughly half the part’s size in pixels stops one part being reported as a cluster of matches.

Step 4: set the threshold from real parts

Run step 3 several times with real parts and note the score of the genuine matches. Then set

module-1/find-object/input/confidence-threshold = <a little below the worst genuine score>

If genuine scores hover near the threshold, turn on blurring - set module-1/find-object/input/preprocess/blur-sigma-x to 1.0 - and measure again. See Tuning the Search.

Step 5: make it permanent

Settings take effect immediately but are not saved until you ask:

system/save-module-inputs/input/module-id = module-1
write system/save-module-inputs/execute
write system/save-current-modules/execute

Both should report status 1. The module and its settings now come back after a restart.

Step 6: run it from a control program

Per cycle:

  1. Write to module-1/find-object/execute.

  2. Read module-1/find-object/output/status-code.

  3. If it is negative, the run failed - read output/status-text for the reason and do not use the result.

  4. If it is 1, read output/result. An empty matches list means the part was not there.

Read the outputs before triggering the next execute: a run empties them as it starts.

If the part is not found

Symptom What to do

Scores just below the threshold

Lower confidence-threshold; try blur-sigma-x at 1.0.

Found in some orientations only

Teach a view at the orientation being missed - no setting fixes this.

Found when close, missed when far

Teach a view at that working distance.

One part reported many times

Raise nms-min-dist to about half the part’s size in pixels.

Never found at all

Check status-code; check object-id is on system/objects; check the crop is not excluding the part.

Found except near the right or bottom edge

Expected - see the 16 pixel rule.