Troubleshooting

Problems that actually come up, and what to do about each.

Photos

An upload gets stuck on “Checking your photo”. The worker is not running, or its queue is backed up. On a self-hosted install check the worker is running (docker compose ps worker); on CardOffice cloud, contact support. The photo is safe — it processes as soon as the worker returns.

“The upload did not complete.” The browser’s PUT to object storage failed. Usually CORS: the bucket must allow PUT from your site’s origin. See deployment.

“That file is 22.4 MB. The limit is 15 MB.” Genuine limit. Raise MAX_UPLOAD_BYTES if you need to, but a 22 MB photo is almost always a screenshot of a screenshot — a fresh photo will be smaller and better.

Photos upload but never appear in the review queue. They may have been auto-approved. Check Settings → Photos: if Require photo approval is off and the score clears the threshold, photos skip the queue.

Everything is scoring badly. Check the room, not the software. Overhead-only lighting puts shadows in the eye sockets and tanks both the lighting and contrast checks. One light in front, or a window, fixes it.

HEIC photos from iPhones fail. They are supported, but the browser cannot preview them, so the dimension checks are skipped and done server-side instead. If they fail there, sharp was built without HEIF support — convert to JPEG, or rebuild sharp with libheif.

Orders

“None of the selected people have an approved photo yet.” Exactly what it says. Filter card holders by Photo: No photo yet to see who needs chasing.

“This organisation has no card template yet.” Create one in Templates. New organisations get a usable starter design; if it was deleted, make a new one.

An order is stuck in “in production” with cards still pending. Look at the production console. Either no agent is online, or the cards failed and are waiting for someone to decide. The order advances to printed by itself once every card is accounted for.

“Someone else changed this order. Reload the page and try again.” Two people acted on the same order at once. This is the optimistic-lock check doing its job — reload and redo your change. Nothing was lost.

Artwork says “Pending” forever. The render queue is not being drained. Check the worker, then idportal_queue_depth{queue="render"}.

Printing

The printer shows offline in the portal but works locally. The agent is not running or cannot reach the portal. On the print PC:

idportal-agent status

The printer says “Standby — asleep until the next card”. Working as intended — the printer has gone to sleep and the agent is leaving it alone. Nothing is stuck: the next card wakes it, taking a few seconds longer than usual. If a site would rather the printer never slept, set Wake mode to Keep it awake on the printer’s page in the portal.

Note the one thing it cannot distinguish: a printer that is switched off or unplugged also reads as standby, until a card is sent to it and fails. Where that matters, use Keep it awake and an unplugged printer shows as offline within a minute.

Windows says “Use Printer Offline” and clearing it does not stick. The ZXP7 dropping into standby is what sets that, and Windows does not clear it when the printer wakes — so jobs collect in the spooler and the card that would have woken the printer never arrives. The agent clears it for you, either before each card or on every heartbeat with Keep it awake. If it is coming back repeatedly and printing is not recovering by itself, check the agent is actually running; that latch clearing itself is the agent’s doing.

The printer never sleeps even though wake mode says it may. Check the mode that is really in force. The portal’s per-printer Wake mode overrides proactiveWake in the print PC’s config.json, so the file can say one thing while the printer does another — idportal-agent status reports the file’s value, and the agent logs the portal’s on startup and whenever it changes. Something other than the agent can also keep it awake: any driver utility or monitoring tool that polls the printer over USB resets its inactivity timer the same way.

The print agent will not download from the portal. Almost always the print PC refusing an unsigned 83 MB executable rather than anything wrong with the file — the agent is not code-signed, so Chrome Safe Browsing, SmartScreen or an AV proxy may drop it. The download often fails silently, or the browser reports it as failed with no reason.

1. Open chrome://downloads. A blocked entry can usually be recovered with Keep, without downloading it again.
2. Check Windows Security → Protection history for a blocked item.
3. Try Edge or Firefox. If one works it is browser policy, which localises the problem in under a minute.

Confirm whatever arrives is intact — the production page shows the expected checksum, and the download itself carries it in an X-Agent-SHA256 header:

Get-FileHash .\idportal-agent.exe -Algorithm SHA256

If every browser fails the same way, copy the file to the machine by another route rather than fighting it; there is nothing special about the packaged executable and it does not care how it got there.

To tell this apart from a portal-side problem: if the portal’s Settings → Downloads page shows a version, size and checksum at all, the file is present and readable on the server, and the problem is between there and the print PC.

The full download → unblock → allow sequence, including SmartScreen and AV quarantine, is in Trusting the agent downloads on Windows.

Cards print but the colours are wrong. Driver colour correction is on. Turn it off — the artwork is already colour-managed to sRGB, and correcting twice is what turns skin tones orange.

Cards are scaled or off-centre. The driver is scaling to fit. Set the card size to CR-80 and disable any fit-to-page option. The artwork already includes bleed.

Only the front prints. Either the driver is set to single-sided, or a YMCKO ribbon is fitted where the template needs two passes. Use YMCKOK for a colour front with a printed back.

The ribbon count is wrong. Correct it in Production → Consumables. The count decrements per image printed; if a run was aborted at the printer, the portal cannot know.

A job is stuck as “claimed”. The agent that took it died. Its lease expires after five minutes and the job is requeued automatically. If it does not, the maintenance worker is not running.

Magnetic stripes do not read. Check the coercivity matches the card stock — HiCo stock needs HiCo settings. On Windows, check ZEBRA_ENCODER_PATH points at Zebra’s encoding utility; without it, cards needing encoding fail rather than printing unencoded.

Sign-in

“That email or password is not correct” for a password you know is right. After eight failed attempts an account locks for 15 minutes. The message is deliberately identical either way, so that the login form cannot be used to work out which email addresses have accounts.

Someone is locked out and cannot wait. An administrator can reset their password in Users, or they can use Forgot your password?.

Invite links say they have expired. They last 48 hours. Send another — it takes two clicks.

Password reset emails never arrive. In development, they are captured by Mailpit at http://localhost:8025 and never actually sent. In production, check SMTP_* and the email_outbox table: rows stuck as pending with a last_error tell you what SMTP said.

Two-factor codes are rejected. Codes are valid for 30 seconds, with one step either side allowed. If they are consistently rejected, the phone’s clock is out — turn on automatic time. Failing that, use a recovery code.

Performance

Everything is slow under load. Check, in order:

1. GET /api/health — is the database or Redis struggling?
2. idportal_queue_depth — is the worker keeping up?
3. Postgres connection count — is (replicas × DB_POOL_MAX) near max_connections? Put PgBouncer in front.

The photo queue is backing up at enrolment time. Expected — that is the annual peak. Scale the worker:

docker compose up -d --scale worker=4

Photo processing is CPU-bound, so more replicas genuinely help.

Card holder search is slow. Confirm rls.sql was applied — it creates the trigram indexes that back partial matching. Without them, %john% is a sequential scan.

Data

“A card holder with ID STU-2026-0041 already exists.” IDs are unique per organisation. Either it is a genuine duplicate, or the record exists but is archived — archived records still hold their ID.

An import failed on most rows. Download the error report. The most common causes are a column mapped to the wrong field, and dates in a format the source system writes but the mapping did not expect.

Someone was archived by mistake. Nothing is destroyed. Filter by Status: Archived, open the record, and restore it.

Storage quota reached. storage_used_bytes is recalculated nightly from what is actually stored. If it looks wrong, run the recalculation from the maintenance queue, or raise the quota in the organisation’s settings.

Mobile wallet

No “Add to your phone” section on the card page. Work down: is FEATURE_WALLET on for the deployment; is Settings → Mobile wallet enabled for that organisation; does the person hold a card at all. A wallet pass hangs off an issued card, so someone whose order has not been printed has nothing to add — unless the organisation issues digital IDs, in which case approving their photo should have issued one.

The buttons say “Add to Apple Wallet” in plain text instead of the proper badges. The official artwork has not been installed. It is trademark-controlled and not shipped with the source; drop the SVGs into apps/web/public/wallet/ — see Deployment. The plain buttons work perfectly meanwhile.

“Your pass is being prepared” and it stays that way. The build failed or never ran. Admin → System health → Mobile wallet shows passes awaiting rebuild; the card holder’s Cards tab shows the error against each pass. Rebuild & push retries and clears the failure count — a pass parked after 20 consecutive failures will not retry on its own.

A revoked card still shows as valid on someone’s phone. Check push deliveries on that holder’s Cards tab. The usual cause is the Apple certificate having expired, which stops pushes with no other symptom — System health shows the expiry date and Test APNs connectivity confirms it in one click. The pass will also correct itself when the device next polls.

Google passes fail with a 403 on first use. The service account exists but was never added as a Developer under Users in the Google Pay & Wallet Console. Being in the right Google Cloud project is not enough; the Wallet console has its own permissions.

A pass installs on Android but not on iPhone. Almost always the signing chain. A self-signed or stub certificate cannot install on any device. Verify the built pass with openssl smime -verify … -purpose any — -purpose any is required, because a Pass Type ID certificate carries Apple’s own extended key usage rather than emailProtection.

Nothing at all happens on an iPhone with the Google button. Expected. Google withdrew Google Wallet from iOS; use the Apple button there.

Getting more detail

Turn up logging:

On a self-hosted install, set LOG_LEVEL=debug in the worker’s environment and restart it:

LOG_LEVEL=debug

Logs are one JSON object per line:

docker compose logs worker | jq 'select(.level == "error")'

Useful queries when something looks wrong:

-- What has this order actually done?
select action, actor_label, changes, created_at
from audit_log
where entity_type = 'order' and entity_id = '…'
order by created_at desc;

-- Mail that could not be delivered
select to_address, subject, attempts, last_error
from email_outbox where status = 'failed';

-- Print jobs waiting or stuck
select id, status, attempts, last_error, claim_expires_at
from print_jobs
where status in ('queued','claimed','printing','failed')
order by created_at;

A card says “Expired” (or “Replaced”) but its expiry date is in the future

That card was superseded: the person was printed a newer card, and the portal keeps one live card per person, so the older one is closed. Issued cards now labels these Replaced; the stored status is still expired, which is what the verification page and the door-system write-back treat it as. Check the holder’s page — they should have an Active card issued the same minute the old one closed.

If they have no active card, the print was not registered. Contact support with the order number and the holder’s name, and we will issue the missing card.