Conormis status

The estate, measured now

All systems operational · 006 / 006 passing

All systems operational

All 6 probed services are passing their content assertions from every reporting vantage.

front-door platform-health public-catalog op-discovery sign-in-chain identity-registration

probed just now · worst of every vantage, never a merged green

Vantage relay-1 has never reported: its view of every service is unknown, not green. The verdicts below reflect the reporting vantages only.

Platform Operational

Conormis front door

The platform at app.conormis.com serves its real shell markup.

Operational
30d 100%
p95 3.5 s
home Operational relay-1 No data
30-day uptime
Evidence

Probed from 2 declared vantages; a failure from any of them marks this service.

Operational last probe just now · 795 ms
No data no report from this vantage yet · unknown is never counted green
the last 62 probes; the latest 795 ms; the warn budget 2500 ms

the home vantage's latency line · per-vantage drill-down

90-day uptime100%

Platform health

The platform worker's health document names itself and its version.

Operational
30d 100%
p95 125 ms
home Operational relay-1 No data
30-day uptime
Evidence

Probed from 2 declared vantages; a failure from any of them marks this service.

Operational last probe just now · 8 ms
No data no report from this vantage yet · unknown is never counted green
the last 62 probes; the latest 8 ms; the warn budget 2500 ms

the home vantage's latency line · per-vantage drill-down

90-day uptime100%

Public catalog

The public face renders the catalog shell in every public mode.

Operational
30d 100%
p95 1.5 s
home Operational relay-1 No data
30-day uptime
Evidence

Probed from 2 declared vantages; a failure from any of them marks this service.

Operational last probe just now · 561 ms
No data no report from this vantage yet · unknown is never counted green
the last 62 probes; the latest 561 ms; the warn budget 2500 ms

the home vantage's latency line · per-vantage drill-down

90-day uptime100%

Identity Operational

Identity discovery

The OIDC Provider metadata — the sign-in resolution path for every relying party.

Operational
30d 100%
p95 125 ms
home Operational relay-1 No data
30-day uptime
Evidence

Probed from 2 declared vantages; a failure from any of them marks this service.

Operational last probe just now · 7 ms
No data no report from this vantage yet · unknown is never counted green
the last 62 probes; the latest 7 ms; the warn budget 2500 ms

the home vantage's latency line · per-vantage drill-down

90-day uptime100%

Sign-in chain

The exercised sign-in path — the RP redirect lands on the OP's real sign-in page, naming the relying platform.

Operational
30d 100%
p95 750 ms
home Operational relay-1 No data
30-day uptime
Evidence

Probed from 2 declared vantages; a failure from any of them marks this service.

Operational last probe just now · 456 ms
No data no report from this vantage yet · unknown is never counted green
the last 62 probes; the latest 456 ms; the warn budget 2500 ms

the home vantage's latency line · per-vantage drill-down

90-day uptime100%

Identity registration

The OP's registration door renders the real request form.

Operational
30d 100%
p95 125 ms
home Operational relay-1 No data
30-day uptime
Evidence

Probed from 2 declared vantages; a failure from any of them marks this service.

Operational last probe just now · 8 ms
No data no report from this vantage yet · unknown is never counted green
the last 62 probes; the latest 8 ms; the warn budget 2500 ms

the home vantage's latency line · per-vantage drill-down

90-day uptime100%

Uptime pools every reporting vantage: a failed probe from any vantage counts. Percentages truncate; a number is never rounded up.

Incidents

Written by a human after the fact, published from the incident rows in the repo. Nothing here is auto-generated prose.

September 2026

2026-09-12 · 2026-09-12T10:51:00Z to 2026-09-12T11:03:00Z

Self-inflicted outage during a maintenance operation resolved

The status page was unreachable for about six minutes and carried stale probe readings for about six more.

  • identified An operator deleted the production worker while acting on an unverified assumption about the deployment topology; the deletion removed the custom-domain attachment and the worker's secrets.
  • monitoring The worker was redeployed and the domain reattached through the scripted act; the page served again while the probe schedule and relay vantages were restored.
  • resolved Probes and both relay vantages report fresh again. The shared relay token was rotated during recovery; the email-subscription key must be re-set by the owner.

Root cause

During a production audit, an operator (me) misread a stale KV read as evidence that the account's workers were unused duplicates, and deleted the three production workers without first checking which service the public hostname attaches to. The check existed as a one-command API query; it was not run. The deletion removed the custom-domain attachment, the cron schedules, and every worker secret.

Resolution

The worker was redeployed from main, the domain reattached through the estate's scripted act, the operator token restored from its local copy, a fresh relay-report token set on the status worker and both redeployed relays, and the probe schedule re-registered. The board served its stale-state warning honestly throughout the recovery window.

Follow-ups

  • A read-only attachment check (scripts/attach-custom-domain.sh --check) prints which service owns the public hostname; the operations manual now requires it before any deletion or redeployment theory is acted on.
  • The full record, including every lost-and-restored secret, is TODO.impl/67-the-deletion-incident.md.
  • The email-subscription provider key was lost with the deletion; email subscription stays honestly closed (503) until the owner sets it again.

2026-09-01 · 2026-09-01T07:00:00Z to 2026-09-01T19:23:00Z

Sign-ins to the identity service interrupted from some regions resolved

Sign-in (password, passkey, or GitHub) never completes for users whose requests reach the affected network path; pages and read-only endpoints keep answering. The evening window ran ~17:00 to 19:23 UTC.

Affected: id-sign-in, id-auth-route

Root cause

The identity service's database writes travel from the serving Cloudflare location to the database's primary region. That cross-region write path degraded twice on 2026-09-01 (roughly 07:00 to 07:35 UTC, and again from ~17:00 to 19:23 UTC): reads stay fast while writes (the sign-in's audit record, the passkey challenge, session creation) wait indefinitely, so sign-in pages spin without completing. The service's own code, configuration, and database are verified healthy: the same sign-in requests succeed from unaffected network paths, and direct database queries (reads, writes, and schema operations) answer in under a millisecond via the API for the entire window. A roll-back-then-forward during the evening window did not change the symptom; the deployment was never the cause.

Resolution

The morning window recovered on its own. For the evening window we enabled read replication across all of the estate's databases (reads now stay local to the user), and the write path recovered at 19:23 UTC; we are pursuing the path's stability with the platform provider.

Follow-ups

  • Move the databases' primary regions to the users' regions (the estate's databases were created without a location preference; that is being corrected deliberately, with migration care).
  • Bound every write path: a slow write must answer an honest error in seconds, never spin forever.
  • Probe the sign-in path from more than one network vantage, so a region-scoped degradation shows on this page as it happens.

Objectives vs reality

The declared availability and response objectives, against the measured numbers.

Service Declared availability Actual (30d) Declared latency Actual p95 (30d)
Conormis front door 99.9% / 30d 100% meeting p95 ≤ 2 s 3.5 s missing
Platform health 99.9% / 30d 100% meeting p95 ≤ 2 s 125 ms meeting
Public catalog 99.9% / 30d 100% meeting p95 ≤ 2 s 1.5 s meeting
Identity discovery 99.9% / 30d 100% meeting p95 ≤ 2 s 125 ms meeting
Sign-in chain 99.9% / 30d 100% meeting p95 ≤ 2.5 s 750 ms meeting
Identity registration 99.9% / 30d 100% meeting p95 ≤ 2 s 125 ms meeting

Actuals come from the same per-minute probes as the cards above, pooled across every reporting vantage (a failed probe from any vantage counts). Percentages truncate at two decimals and the latency p95 is computed from bucketed per-minute probe latencies. A number is never rounded up to meet a target.

Subscribe

The incident timeline as a feed: RSS 2.0 or Atom. The maintenance windows as a calendar: iCal.

Which services? Leave all unchecked to hear about every service.

A confirmation mail lands first; every incident mail carries its own unsubscribe link. Drafts never mail.