Architecture · Technical brief
Two data flows: external signals from public sources, and internal signals from Redbrain’s finance system. They meet in one Postgres database, get scored together, and surface in three role-specific dashboard views. Roughly £30 per month of cloud infrastructure.
External data is pulled by Cloudflare Workers on cron. Internal data is pushed (or pulled) from Redbrain’s finance system on the same cadence.
The whole system is <1,500 lines of TypeScript plus a Postgres schema. Cloudflare Workers run the crawlers on cron. Cloudflare Queues handle anything long-running so no single Worker invocation hits the 30-second CPU limit. Neon stores everything; the dashboard reads directly from it.
This is the most important design decision the project has left. Each option trades off implementation effort against ongoing reliability and security.
How: The finance system writes a daily CSV to an S3-compatible bucket (or Cloudflare R2). A Worker reads it nightly and updates Neon.
Pros: Simple. Finance team controls what leaves. Easy to audit.
Cons: Daily-only freshness. Schema changes break silently unless versioned.
How: A read-only Postgres user with access to specific views in the finance database. A Worker syncs the views into Neon every hour.
Pros: Near-realtime. Resilient to schema change if views are stable. Standard pattern.
Cons: Requires a network path from Cloudflare into Redbrain’s infra. Needs IT sign-off.
How: Finance system fires webhooks on invoice / payment / term-change events to a Worker endpoint. Worker writes signals directly.
Pros: Real-time. Signals appear within seconds of the event.
Cons: Requires finance-system changes. More moving parts. Replay logic for missed events.
Recommendation: start with Option A. Daily freshness is enough for the first version of risk scoring, the security story is the simplest to sell internally, and switching to B or C later is straightforward once the dashboard has proved its worth.
600 requests per 5 minutes, total. With ~1,000 merchants × 3 endpoints, that’s 25 minutes minimum to drain. We coordinate concurrent queue consumers via a shared KV counter with a 20-request safety margin.
Can’t loop over the merchant book in one invocation. Hence the two-stage queue pattern everywhere: a scheduler enqueues, consumers process one merchant at a time.
Redbrain identifies merchants by internal ID; Companies House by 8-digit CRN. The merchant table will need both. Some Redbrain merchants will be subsidiaries; the right CRN to monitor isn’t always the trading name. Manual mapping for top 100, scripted matching for the long tail.
Right now the scorer weights all signals by severity. Once internal payment data is in, internal needs to outrank external for the same severity — a 60d DSO is more decisive than a critical-severity news mention. Will need backtesting against a known historical case to tune.
“Require prepayment” is a system suggestion, not a system decision. The dashboard needs to make that crystal clear, log who acted on what when, and never tell a merchant directly that their score moved. Defamation and contract-law exposure is real and worth a solicitor conversation before launch.
Many UK retailers are themselves on Cloudflare with bot protection on. Direct Worker requests get challenged. For uptime/discount monitoring we’ll need Cloudflare’s Browser Rendering (paid quota) or an external service.