Ability to block abusive clients (IP/CIDR + user-agent) from the economy-explorer web tier, so a repeat of the Blockantir scrape can be cut off without taking the site offline.
Background
2026-07-16 → 07-20 a scraper (Blockantir-BalanceCollector/1.0, IP 136.107.101.57) made ~3.25M unauthenticated requests (~10 req/s, ~30x baseline) paginating /accounts?sort=balance + /firms — a full balance/firm harvest. It pinned the shared DB host and put the explorer into a crash loop (~21 restarts over the 4 days). Stopped on its own after the in-game ban; there is currently no way to block the client at the network/app layer. Detection/alerting is now in place (PrometheusRule → Discord); this is the missing response control.
Why not a CiliumClusterwideNetworkPolicy (the obvious first instinct)
Investigated — a CCNP is a no-op for this topology:
- The Gateway is a hostNetwork Cilium Envoy that proxies every connection, so the app pod sees Envoy as the L3 source. The real client IP only survives in
X-Forwarded-For (that's where the access log reads it). A pod-targeted fromCIDR policy matches nothing.
- The only CCNP that would see the client IP is a host-firewall policy, but
enable-host-firewall is off, and enabling it + selecting the host endpoint flips host ingress to default-deny — requires allowlisting SSH/kube-API/node-to-node or you lock both nodes out. Not worth it for this.
So the effective + low-risk layer is the app itself, where the XFF IP + UA already exist.
Proposed implementation
- Denylist middleware in economy-explorer (sits alongside/after the existing access-log middleware), returns
403 early for matching requests.
- Match on client IP/CIDR (from
X-Forwarded-For, trusting the gateway hop) and User-Agent (exact + substring).
- Config-driven, not code — env var (e.g.
BLOCKED_IPS / BLOCKED_UAS, comma-separated, CIDR-aware) so updates are a values change in k8s-gitops, no rebuild.
- Emit a structured log line (
kind:"blocked") per rejected request so blocks are visible in Loki and can feed a "still being probed" metric.
- Seed with the known abuser: IP
136.107.101.57, UA Blockantir-BalanceCollector/1.0.
- Apply to both tenants (democracycraft + statecraft) via the shared image; keep values per-overlay.
Notes
- Deprioritised deliberately — parked behind the current large PR; the deterrence messaging already implies this capability exists.
- Follow-on (separate tesks if wanted): rate-limit unauth
/accounts + /firms pagination and cap page depth (the deep-OFFSET queries are what caused the max_statement_time timeouts).
Implementation and verification — 2026-10-22
- Draft PR: https://github.com/ParadauxIO/hibernia-economy/pull/81 (commit d8bb3e7d3c9ba559ffa00cc05d5767fb7701b2fe), targeting develop. Status remains In progress pending review/build acceptance and separate activation.
- Added configurable IPv4/IPv6/CIDR and exact/substring User-Agent blocking in the Explorer proxy, before auth, redirects and route rendering; rejected requests return 403 and structured blocked logs. No rules are active by default.
- Trust decision: GitOps uses a Cilium hostNetwork gateway and ClusterIP Explorer service. Production scraper-block.yaml documents useRemoteAddress=true, zero trusted XFF hops and Envoy-overwritten x-envoy-external-address. The app ignores XFF for blocking. IP rules require BLOCK_TRUST_ENVOY=1 and fail startup otherwise. Before activation, verify deployed Envoy behavior and restrict direct pod ingress to gateway; ClusterIP alone is insufficient.
- Verification: npm run test:unit -- --reporter=dot (196 passed); npm run typecheck (passed); npm run lint (passed); git diff --check (passed). Disposable localhost port 31428 with unreachable DB port 39999: blocked /accounts?sort=balance returned HTTP 403 plus kind=blocked log; /healthz returned HTTP 200. Proxy tests assert early rejection before auth/route dispatch, IPv4/IPv6/CIDR, spoofed XFF, malformed identities and allowed traffic.
- Remaining gap: npm run build compiled successfully but twice failed in Next's TypeScript --showConfig parser. Standalone tsc --showConfig and npm run typecheck succeeded. No deployment, GitOps rule activation, shared staging or production data changes were made.
- Scope note: matching logic is local to Explorer; PAR-235 and PAR-159 may have related matching concerns and remain separately owned.
Build follow-up — 2026-10-22
-
Focused comparison: Next's own getTypeScriptConfigurationCli subprocess on PR commit d8bb3e7d and on origin/develop 4339da00 both exited 0 but captured zero stdout inside the filesystem sandbox; outside the sandbox, the same locked TypeScript 5.9.3 subprocess returned 12,725 bytes of valid JSON. The earlier --showConfig parser failure is an execution-environment artifact, not caused by PAR-328 source changes.
-
Production build: npm run build completed successfully outside the sandbox on PR commit d8bb3e7d (compilation, TypeScript, 46 static pages and route optimization). No source change was needed; the PR remains draft for review and separate activation decisions.
-
PR checks at inspection: Explorer typecheck/lint/Vitest, JVM build, migrations, REST contract, CodeQL and other completed checks passed. Explorer Playwright was pending. Uploaded diagram URL and PR base/head were verified after updating the body.
-
PR check follow-up (2026-10-22): Explorer Playwright completed successfully. All required PR checks are green; live-server integration and mutation testing were skipped by workflow conditions. Draft status and separate activation boundary remain.
Merge — 2026-10-22
- PR #81 merged into develop at 10:34:44 UTC via merge commit e4631023ea7c3d9228aa253e14e7632984442c42. All required PR checks passed. The feature remains inactive by default; per-tenant rule activation and gateway trust verification remain separate deployment work. No staging/production config or data was changed here.