Language Models

The reader needs a Tesseract language model - a .traineddata file - to read anything at all.

Which models are installed is application-wide: ten modules reading the same labels want the same models, so the setting lives on system/ and not once per module.

On ctrlX OS, a model is uploaded over the ctrlX file interface.

On a PC, to upload a model is to copy its file into the directory system/tessdata-dir names.

hdvisionsystems/lumiscan-text-reader/system/
├── tessdata-dir              where you upload your own models   read-only
├── tessdata-dirs             every directory searched, in order read-only
├── languages                 the codes found across all of them read-only
├── set-tessdata-dir/         changes the upload directory
└── refresh-languages/        re-reads the directories

Two directories, searched in order

  1. Your upload directory, tessdata-dir. By default that is the folder tessdata in the application’s configuration directory, beside the application’s configuration file. It is created empty on first start.

    Platform Default tessdata-dir

    ctrlX OS

    /var/snap/rexroth-solutions/common/solutions/activeConfiguration/lumiscan-text-reader/tessdata

    PC (Windows)

    %APP_RESOURCE_DIR%\etc\lumiscan-text-reader\tessdata; without APP_RESOURCE_DIR, C:\ProgramData\HD Vision Systems GmbH\LumiScan Vision Suite\lumiscan-text-read\etc\lumiscan-text-reader\tessdata

  2. The models shipped inside the application, which are read-only and always searched.

tessdata-dirs shows both, in the order they are searched.

Uploading adds languages, it never removes the shipped ones. You do not have to copy our files anywhere, and English keeps working whatever you do. A model uploaded under the same name as a shipped one wins, which is how you would replace English with a different English model.

What is installed

English (eng) and osd. Together about 34 MB, which is why only these two ship - every extra language would grow the application for every customer, and uploading is how you get the rest.

osd is not a language. It is the orientation-and-script model that the AUTO_OSD and SPARSE_OSD layouts need. It appears in languages because it is a file in the directory, but setting input/language to osd will not read anything.

Adding a language

  1. Download the model. The Tesseract project publishes them at github.com/tesseract-ocr/tessdata_fast and github.com/tesseract-ocr/tessdata_best. fast is smaller and quicker, best is more accurate; fast is the right default for part markings.

  2. Upload it into the directory system/tessdata-dir names: on ctrlX OS over the ctrlX file interface, on a PC by copying the file there.

  3. Use it: set read-text/input/language to the file name without .traineddata.

upload deu.traineddata  ->  <tessdata-dir>/deu.traineddata
write  module-1/read-text/input/language = "deu"
write  module-1/read-text/execute

There is no code to look up

The file name is the language code. Tesseract requires the model for language deu to be called deu.traineddata, so whatever you upload defines what you type. You never have to find a code in a table, and you must never rename the file.

The codes are not all three letters. Chinese is chi_sim (simplified) or chi_tra (traditional), and there are others such as srp_latn and aze_cyrl. Take the name exactly as the Tesseract project publishes it.
Language File to upload input/language

German

deu.traineddata

deu

French

fra.traineddata

fra

Chinese, simplified

chi_sim.traineddata

chi_sim

Japanese

jpn.traineddata

jpn

No restart, and no refresh either

A model you upload is usable on the very next read-text. The reader looks the file up on disk each time it runs, so a new one is found without anything being refreshed or restarted.

This holds for replacing a model that is already in use too. The recognition engine is built once per module and kept, because building it means loading the model - 23 MB for English - and doing that on every read made a small read three times slower than it needed to be. It is rebuilt whenever the language, the directory, the character set, the engine or the layout changes, and whenever the model file itself changed: before every read the reader checks the file’s size and time, and when either changed it compares the file’s content with the one the engine was built from. So a file overwritten in place, under the same name and directory, is read with on the next read-text - no restart.

The check costs a file status per read. A changed model is hashed once, about 40 ms for 23 MB, and then loaded as any new model is; the log says The '<language>' model in '<directory>' changed on disk; loading it again. A file that cannot be loaded - an upload still in progress, say - makes the read fail with -7, and the next read after the upload completes loads it.

system/languages is the one thing that does not update on its own - it is a listing, refreshed at startup, by set-tessdata-dir and by refresh-languages:

write  system/refresh-languages/execute
read   system/languages           -> [chi_sim, deu, eng, osd]

Call it to make the listing agree with the directory. You do not need it in order to read.

Several languages at once

input/language is a per-module input, so each module reads in its own language from the same shared set of models:

module-1/read-text/input/language = "eng"
module-2/read-text/input/language = "deu"
module-3/read-text/input/language = "chi_sim"

Each module reports what it actually used in its own output/result.language. Nothing is shared between them but the models on disk.

Using a different upload directory

set-tessdata-dir points the first search directory somewhere else. On ctrlX OS a USB stick, say, since the application has the removable-media interface:

write  system/set-tessdata-dir/input/tessdata-dir = "/media/usb/tessdata"

On Windows a folder on another drive, say:

write  system/set-tessdata-dir/input/tessdata-dir = "D:\models\tessdata"

Then, on either:

write  system/set-tessdata-dir/execute
read   system/set-tessdata-dir/output/status-code   -> 1
read   system/languages                             -> [deu, eng, fra, osd]

The shipped models stay available, so an empty directory is accepted - a reasonable thing to point at just before uploading into it. The choice is saved and survives a restart.

Unlike every other path this application takes, an absolute path outside the configuration directory is accepted here. Language models are large third-party files that are legitimately kept where they are - on ctrlX OS outside the ctrlX solution - rather than copied into the configuration directory. The application only ever lists and reads them.

osd has to sit beside the language

Tesseract resolves one directory per run, so the AUTO_OSD and SPARSE_OSD layouts need osd.traineddata in the same directory as the language model - not merely somewhere on the search path. Upload deu.traineddata on its own, pick AUTO_OSD, and the run is refused saying exactly that:

error: invalid input: layout AUTO_OSD also needs osd.traineddata beside 'deu.traineddata' in
'<tessdata-dir>'; copy it from '<shipped dir>', or choose a layout that does not detect orientation

Either upload osd.traineddata alongside your model, or use a layout that does not detect orientation. The default, SINGLE_BLOCK, does not - this only comes up if you asked for orientation detection.

When set-tessdata-dir refuses

It validates before it publishes anything, so a rejected call leaves tessdata-dir, tessdata-dirs and languages exactly as they were - you never see a directory and a listing that disagree.

Situation status-code

input/tessdata-dir is empty

-2 invalid input

The path is not a directory

-4 source not found

Switched, but the choice could not be saved

-6 write failed - the switch is live, only persistence failed

An empty directory is not refused. It was, back when this was the only place models were read from and an empty one meant every read silently fell back to English. The shipped models are always searched now.

Getting the language wrong

read-text validates the language before it runs and refuses an uninstalled one with status-code -2, naming every directory searched and what is available:

error: invalid input: language 'fra' has no fra.traineddata in any model directory
('<tessdata-dir>', '<shipped dir>'); available: eng, osd. Upload fra.traineddata to
'<tessdata-dir>' - the file name is the language code

If a model disappears between that check and the read - someone pulled the USB stick - output/result.language reports what was actually used, and status-text says so:

ok: 4 word(s), mean score 0.91 (eng, SINGLE_BLOCK) - NOTE: language 'deu' is not installed, read with 'eng' instead

Read output/result.language, not input/language, when you need to know how a result was produced. They are usually the same. When they differ, only the output is telling the truth.