Projects in S3 Buckets

Instead of a folder on the server, a project can keep its files in an S3 bucket: AWS S3, MinIO, SeaweedFS or any other store that speaks the S3 API. Below the project’s prefix the layout is exactly that of a project folder (date folders, <image>.png, .csv, .json), so a folder can be copied into a bucket and back with any S3 tool (aws s3 sync, mc mirror, …) and LumiScan Training reads the same files.

1. Configure the buckets

The server admin adds the buckets to the configuration. The access keys stay in the configuration; the browser only ever sees the name.

"s3_targets": [
  { "name": "aws-frankfurt", "endpoint": "https://s3.eu-central-1.amazonaws.com",
    "region": "eu-central-1", "bucket": "annotation-data", "path_style": false,
    "access_key_id": "AKIA...", "secret_access_key_file": "aws-annotation.secret" },
  { "name": "seaweedfs-lab", "endpoint": "https://s3.lab.example.com:8333", "bucket": "annotation",
    "path_style": true, "access_key_id": "annotation-server",
    "secret_access_key_file": "seaweedfs-annotation.secret", "ca_file": "root-ca.pem" }
]
Key Meaning

name

Shown to admins when they create a project; projects refer to it. Do not rename it later.

endpoint

https://s3.<region>.amazonaws.com (AWS), http(s)://host:9000 (MinIO), http(s)://host:8333 (SeaweedFS)

region

The bucket’s region for AWS, for example eu-central-1; MinIO and SeaweedFS accept us-east-1.

bucket

The bucket; it must exist.

path_style

true: endpoint/bucket/key (MinIO, SeaweedFS; default). false: bucket.endpoint/key (AWS; required for buckets in newer regions).

access_key_id, secret_access_key_file

The keys. secret_access_key directly in the file is possible, a separate file readable only by the service account is better. Without keys, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and AWS_SESSION_TOKEN from the environment are used.

ca_file, tls_verify

Own CA for self-signed certificates; "tls_verify": false only for tests.

proxy

http://proxy:3128 if the server reaches the store only through a proxy.

cache_mb, cache_dir

Local image cache, default 2048 MB in s3-cache/<name> next to the configuration; 0 switches it off.

timeout_s

Per request, default 30.

2. Create a project in a bucket

Under your name → Projects → New project (or Import project), choose Store the files in: S3 bucket … and give a folder in the bucket (the prefix, for example line-5). One bucket holds any number of projects; the prefixes of two projects must not overlap. Images already below the prefix are used as they are; an import needs an empty prefix.

How it behaves

  • Annotation files are always read from and written to the bucket, never cached. If a file was changed in the bucket meanwhile (by an S3 tool or another server), saving is refused with a conflict, as with a folder, and the outside version is kept in the history.

  • Images are cached on the server’s disk and checked against the bucket at every request, so changed images are fetched again and unchanged ones cost one small request.

  • Database, locks, sessions and backups stay on the server. Deleting a bucket project never deletes anything in the bucket.

  • Failed requests (network, 503 SlowDown) are repeated; errors are shown with the reason S3 gives (wrong key, clock skew, missing bucket, wrong region).

AWS permissions

The IAM user or role needs:

  • on the bucket: s3:ListBucket,

  • on arn:aws:s3:::<bucket>/*: s3:GetObject, s3:PutObject, s3:DeleteObject.

Without s3:ListBucket, AWS answers missing files with 403 instead of 404. Keep the server’s clock synchronised: AWS refuses requests that are more than 15 minutes off.

MinIO or SeaweedFS?

Both work the same for LumiScan Annotation Web. The vendor of MinIO stopped publishing the community edition as ready-made programs in October 2025; existing installations keep working. For a new self-hosted store, SeaweedFS (Apache 2.0, actively maintained) is the safer choice.