Skip to content

Engine Rewriting

ObsessionDB uses Shared engine variants — SharedMergeTree, SharedReplacingMergeTree, SharedAggregatingMergeTree — to deliver managed replication without operator intervention. These engines do not exist in regular ClickHouse, so DDL that names them fails on a local Docker ClickHouse or self-hosted staging.

chkit therefore writes the standard engine name for every target: a table declared with SharedReplacingMergeTree(ts) is generated as ENGINE = ReplacingMergeTree(ts), and the snapshot stores the same standard name. ObsessionDB transparently substitutes the Shared variant on the server, and chkit drift treats both spellings as the same engine. One set of schema files works against both.

The ObsessionDB plugin handles what the engine name cannot: the storage_policy table setting. It is a standard ClickHouse setting, but the storage policies ObsessionDB provides, such as 's3', are not defined on a regular ClickHouse server, where CREATE TABLE fails with Unknown storage policy. When the target is not ObsessionDB, the plugin removes storage_policy from every table, whatever its value, before diff and planning, so it doesn’t leak into local migrations.

The plugin applies this rewrite when chkit generate or chkit snapshot rebuild loads the schema, so a rebuilt snapshot.json matches the one chkit generate writes for the same target. The migration files record the result, and chkit migrate applies them as written.

By default the plugin inspects clickhouse.url. If the host is obsessiondb.com or one of its subdomains, storage_policy is kept. Otherwise, it is stripped.

A config pointing at https://my-service.obsessiondb.com keeps it, while http://localhost:8123 drops it — no flags needed.

In the TypeScript CLI, two flags override auto-detection for a single run of generate or snapshot rebuild:

  • --force-shared-engines — keep storage_policy even when the URL is not recognized as ObsessionDB, for example an ObsessionDB service behind a custom domain, or a self-hosted server that defines the policy the schema names.
  • --no-shared-engines — strip storage_policy even when targeting ObsessionDB.

If both are passed, --force-shared-engines wins.

Terminal window
# Strip storage_policy even when targeting ObsessionDB
chkit generate --no-shared-engines
# Keep it for an ObsessionDB service behind a custom domain
chkit generate --force-shared-engines
chkit snapshot rebuild --force-shared-engines

The migration files and snapshot.json keep what the flag decided, so pass the same flag every time generate or snapshot rebuild runs for that target. generate applies the flag’s decision to the previous snapshot too, so dropping the flag plans no RESET SETTING storage_policy for existing tables (the server would reject it); those tables keep the policy they were created with. Adding the flag later plans MODIFY SETTING storage_policy for existing tables that declare one.

ObsessionDB writes storage_policy = 's3' into every Shared table that does not name a policy. A Shared table needs a policy on remote object storage, and an ALTER may only switch to one that keeps the current policy’s volumes; ObsessionDB services offer only 's3' for Shared tables today. When the target is an ObsessionDB host, the plugin treats a table that does not declare storage_policy as storage_policy: "'s3'", in the schema and in the snapshot, so CREATE TABLE writes it and a schema pulled with chkit pull (which reads it back, with a comment) plans nothing. A schema that changes it is rejected by chkit generate with the reason, instead of failing at migrate. These two rules apply only to a recognized ObsessionDB host, not to a self-hosted server reached with --force-shared-engines, where any policy, 's3' included, is an ordinary setting.

migrate, status, drift and check also accept both flags, so scripts can pass them to all six commands, but those four ignore them: they work from the migration files and snapshot that generate already wrote.

SchemaGenerated DDL (every target)
engine: 'SharedMergeTree'ENGINE = MergeTree()
engine: 'SharedReplacingMergeTree(ts)'ENGINE = ReplacingMergeTree(ts)
engine: 'SharedAggregatingMergeTree'ENGINE = AggregatingMergeTree()
engine: 'MergeTree'ENGINE = MergeTree()
Table settingRegular ClickHouseObsessionDB
storage_policy not declared—'s3' (ObsessionDB sets it on every Shared table)
storage_policy (any value)strippedkept; a change is rejected by generate
Any other settingkeptkept

Only tables are affected. Views and materialized views pass through unchanged.

ObsessionDB runs one shared-storage service with no shards: every replica reads the same data, so a Distributed table has nothing to route to, and ObsessionDB users have no grant for the Distributed engine. When the target is ObsessionDB (the same detection as above, including --force-shared-engines), the plugin rejects every Distributed table in the schema:

  • chkit generate and chkit snapshot rebuild fail before writing a migration.
  • chkit check fails with the obsessiondb_distributed_unsupported finding.
Table analytics.events uses the Distributed engine, which ObsessionDB does not support: ObsessionDB has no shards, so every replica already reads the same shared storage. Remove the Distributed table from the schema and query or insert into its local table directly.

Against regular ClickHouse the plugin leaves Distributed tables alone.

  • Services — the runtime counterpart: route a single command at a specific service with --service.
  • chkit drift — normalizes SharedMergeTree to MergeTree when comparing engines, so managed-side engine substitutions don’t show up as drift.
  • Schema Overview — where engines are declared in the DSL.