---
title: "Check Annotations with Two Annotators"
description: "Step by step: two annotators per image, blind annotation, automatic reference when they agree, review of the differences and the agreement statistics."
product: "LumiScan Annotation Web"
component: "lumiscan-annotation-web"
version: "2.4.0"
display_version: "2.4.0.0"
latest_version: false
language: "en"
url: "https://docs.hdvisionsystems.com/en/lumiscan-annotation-web/2.4.0/how-tos/check-annotations-with-two-annotators.html"
---

# Check Annotations with Two Annotators – LumiScan Annotation Web

Two people who annotate the same image independently rarely draw exactly the same boxes. Where they differ, the image is hard or the instructions are unclear. LumiScan Annotation Web uses this to improve the training data: the server compares the annotators, writes the reference itself when they agree and shows a reviewer exactly where they differ.

## 1. Switch it on

A project admin opens **Settings** and sets, under **Several annotators per image**:

- **Annotators per image**: `2` (or more).
- **Agree: overlap ≥ %**: how much two boxes must overlap to count as the same box, default 85 %.
- **Agree: angle within °**: default 5°.
- **Write the reference automatically when all annotators agree**: on.

## 2. Annotate blind

Annotators work as usual in the mode **Annotate**. Each sees only their own boxes, never the others'. The image list shows how many have annotated an image (`1/2`).

![Annotate mode with two annotators per image: the image list shows 1/2 or 2/2 per image, the image shows only the annotator’s own boxes](https://docs.hdvisionsystems.com/en/lumiscan-annotation-web/2.4.0/_images/screens/annotate-blind.jpg)

When the last annotator saves:

- **all agree**: same objects, same labels, overlap and angle within the limits. The server writes the mean boxes as the reference `<image>.csv`. The status says **Saved. All 2 annotators agree: reference annotation written**.
- **they differ**: the image goes to review.

## 3. Review the differences

Reviewers switch to **Review** in the top bar. The list is sorted by the largest differences first.

![Review mode: the boxes of the two annotators in blue and orange on the image, numbered red markers where they differ, and the list of differences on the right](https://docs.hdvisionsystems.com/en/lumiscan-annotation-web/2.4.0/_images/screens/review.jpg)

Every annotator has a colour; red numbered markers show the differences:

- a box only one person drew,
- overlap too small,
- angle too different,
- bottom edge on the opposite side,
- different label.

For each difference, the panel on the right offers the boxes of each person, **Average** or **None**. Click **Show** to zoom to it. When every difference is decided, **Save** writes the reference annotation. A reviewed reference is never overwritten automatically.

![Differences panel: two annotators, two of seven objects differ; for each difference the buttons Show, the annotators' names, Average and None](https://docs.hdvisionsystems.com/en/lumiscan-annotation-web/2.4.0/_images/screens/review-differences.png)

## 4. Read the statistics

**Statistics** in the top bar shows the agreement for a folder or the whole project:

![Agreement between annotators: images with two annotators, share of images and objects that agree, mean overlap; tables per annotator, per pair, differences by kind and the images with the largest differences](https://docs.hdvisionsystems.com/en/lumiscan-annotation-web/2.4.0/_images/screens/statistics.png)

- **Per annotator**: overlap with the consensus and systematic habits, for example boxes drawn wider than the others (width +3 %) or rotated.
- **Pairs of annotators**: agreement (F1) and mean overlap.
- **Differences by kind** and **Largest differences**: click an image to review it.

Use the per-annotator figures to improve the instructions: a systematic deviation usually means the rule was not clear.

More: [Several Annotators per Image](https://docs.hdvisionsystems.com/en/lumiscan-annotation-web/2.4.0/several-annotators-per-image.html).