Add database-backed release feature flags across Hibernia Economy
Add database-backed release feature flags across Hibernia Economy
Outcome
Ship incomplete features in normal releases while keeping their execution disabled until an operator explicitly enables them through a Flyway data migration, an operational script, or Economy Explorer's backoffice. Keep current production functionality unchanged.
This is a scoped implementation proposal for Tesks/Tasklie. Confirmed scope: one value per economy database, with no per-server overrides in the first version.
Repository findings
economy-flyway owns the shared MariaDB schema; Treasury applies migrations during startup.
V1 created feature_flags; V4 removed it when its sole privacy flag was replaced with RBAC. Add a new migration; do not edit either historical migration or resurrect that privacy flag.
Treasury publishes interfaces from treasury-api through Bukkit's ServicesManager; Business already consumes those interfaces. Plugins must not reach into Treasury's internal mappers.
common is framework-free and can hold shared JVM evaluation logic. Spring and Paper need separate lifecycle/persistence adapters.
Explorer already has server actions, services, Kysely SQL access, explicit permission grants, revision checks, and before/after auditing. Feature flags are control data, never ledger writes.
Treasury's startup failure path shuts down the server. A flag-read failure must remain isolated from optional feature initialization; existing schema/release startup checks remain intact.
Proposed behavior
Boolean, namespaced, case-sensitive keys such as business.new_feature. One value per economy database; separate databases naturally provide environment isolation. No player targeting, percentage rollouts, dependency expressions, remote provider, or per-server overrides in the baseline.
A missing flag, unavailable provider, invalid row, or absent initial snapshot evaluates to false. Application startup must never seed or overwrite operator choices. Introduce each new feature with a migration registering its key and description, disabled by default.
Load a complete immutable snapshot asynchronously and publish it atomically. Refresh every five seconds with bounded query timeouts and no overlapping loads. Never query the DB during a Paper flag check. Expire enabled values after 30 seconds without a successful refresh, checking age at evaluation time even if the refresh worker stalls. Unknown/missing flags remain false. Report refresh failures and stale state without flooding logs.
A healthy deployment should observe changes within five seconds plus bounded query latency. All write paths use the same DB state, so SQL changes need no application-specific invalidation. No promise of instantaneous or atomic changes across processes.
Evaluate at the beginning of a logical operation. Disabling stops subsequent operations after propagation; already-admitted transactions finish. Queued jobs recheck before starting each independent unit of work. Enabling does not automatically replay missed jobs or backfill data.
Guard the service entry point as well as command/listener/job/HTTP/UI entry points. Disabled feature services are initialized lazily where constructors or startup hooks have side effects. Lightweight dispatch wrappers may remain registered, but they must return before resolving or invoking disabled feature code. UI hiding supplements server enforcement. New REST features return a stable disabled-feature response; in-game actions explain that the feature is unavailable.
Feature flags never replace authorization. Existing permissions and ownership checks still apply when enabled. The management UI itself is not behind a release flag.
Flags do not suppress Flyway migrations, compilation/linkage failures, or arbitrary static initialization. New schema must be backward compatible and feature startup must be isolated. A feature with irreversible effects needs its own enable/disable contract before it uses this system. When replacing existing behavior, the disabled path must retain that behavior.
Persistence and operator controls
New feature_flags table: binary/case-sensitive primary key, description, enabled default false, revision, updated timestamp, and optional change attribution. Validate key format and boolean values consistently across runtimes.
Add explicit feature_flags.read and feature_flags.update permissions, grant them to the existing admin role through migration, and integrate them into Explorer's policy, admin navigation, and role editor. Role names alone confer no authority.
/admin/feature-flags lists definitions and stored states, description, last change, and propagation guidance. Read permission allows inspection; update permission allows toggles. Recheck authorization in mutation services. Do not expose arbitrary key creation/deletion in the UI.
Use optimistic concurrency for UI writes; stale saves are rejected. Capture before/after values within the write transaction and record actor/key/change through the existing Explorer audit pipeline. Its existing audit sink is fail-open; document that limitation rather than claiming guaranteed audit persistence.
Ship an operational script that lists flags and sets a known key explicitly on/off, validates inputs, increments revision, and records attribution. It must print the target database and resulting value without exposing credentials. Unknown keys are errors, not implicit inserts. Repeated set-to-same-value calls are safe.
Document transactional SQL for explicit migration/script activation and emergency deactivation, including revision advancement and change attribution. Raw DBA SQL cannot guarantee application audit history or concurrency rules; supported recipes must follow the same update contract.
Registration migrations must not reset an existing value. A separately authored activation migration may deliberately enable a named flag. Document that this activation applies to every database receiving that migration; use the operator script for environment-specific rollout.
Polling reads full current rows, so even direct SQL updates without application notifications become visible. Application caching must not retain true beyond the stale deadline; Explorer route/data caches must not bypass server enforcement.
Implementation slices
Schema and evaluation contract: new migrations, permissions, shared fixtures, missing/error/staleness semantics, and framework-free JVM evaluator in common.
Runtime integration: Treasury owns the Paper snapshot and publishes a slim FeatureFlagApi through treasury-api; dependent plugins consume that interface and default to disabled when unavailable. Spring uses the common evaluator with its own DB and scheduling adapters. Explorer gets an equivalent server-only evaluator with bounded caching. Clean up refresh workers on shutdown.
Management: Explorer service/DAL/server actions/admin page, permission gates, revision conflicts, and audit integration; operational script and SQL recipes.
Adoption and documentation: demonstrate command/service/listener/job/HTTP/UI guards using integration-test fixtures; document lazy initialization, safe fallback behavior, rollout, disable, and eventual flag retirement. Do not place all existing functionality behind new default-off flags or change unrelated behavior. Realty is an external submodule: document the public contract; any Realty-specific adoption is a separately scoped upstream change.
Acceptance criteria
Fresh database and upgrade through all existing migrations succeed without modifying historical migration checksums; no old privacy flag returns.
Shipping a new registered flag leaves it disabled; deploying or restarting applications does not reset values.
Two independent consumers observe enable/disable through direct SQL, supported script, and Explorer within the documented healthy propagation bound.
A cached true expires after 30 seconds without successful refresh, including a stalled refresh worker; readers never see a partially loaded snapshot.
Paper checks perform no database/network work on the main thread, and optional feature constructors/hooks are not invoked while disabled.
Representative command, listener, job, service, HTTP, and UI tests prove disabled paths do not invoke feature effects; direct backend calls cannot bypass the guard. Enabled paths still enforce authorization.
Disabling stops newly admitted work without interrupting transactions or replaying skipped scheduled work on re-enable.
Anonymous, unprivileged, and read-only users cannot mutate flags; stale saves conflict; permission revocation takes effect on the next mutation.
Successful Explorer changes produce the expected before/after audit entry when the audit sink is available; failures follow documented existing audit semantics.
Operator script handles unknown keys, invalid inputs, explicit on/off, repeated calls, and DB failures with useful exit status; secrets never appear in output.
Existing economy behavior remains unchanged with no new feature enabled. Refresh failures are observable without crashing unrelated services.
Documentation includes a complete example of shipping disabled, enabling in one database, disabling, and retiring a flag only after supported deployments no longer reference it. Flag keys are not reused for unrelated features.
Verification plan
Use unit tests with a controllable clock for snapshots/expiry and shared evaluator fixtures; MariaDB integration tests for migration upgrades, SQL updates, optimistic concurrency, and independent readers; Paper lifecycle/service tests for inactive feature isolation; Spring tests for service/HTTP guards; Explorer integration and browser tests for permission-sensitive management and enable/disable. Run the affected Gradle checks, migration checks, and Explorer lint/typecheck/test/build commands from existing CI. This scoping task does not implement or run these tests.