Python Core API (beta)
chkit.core exposes the pipeline underneath the beta Python CLI as plain functions over Pydantic models — load definitions, validate them, diff against a snapshot, and render SQL, all without the CLI.
from chkit import ( load_schema_definitions, validate_definitions, plan_diff, to_create_sql, apply_on_cluster_to_plan, define_config, resolve_config,)from chkit.core import create_snapshot, canonicalize_definitions, assert_valid_definitionsThe pipeline
Section titled “The pipeline”chkit generate is, in essence, this sequence:
from chkit import load_schema_definitions, plan_diff
old = snapshot.definitions # last applied state (empty list on first run)new = load_schema_definitions("./src/db/schema/**/*.py")plan = plan_diff(old, new)
for op in plan.operations: print(op.risk, op.type, op.sql)Each stage is available on its own.
Loading definitions
Section titled “Loading definitions”load_schema_definitions(schema_globs, *, cwd=None)
Section titled “load_schema_definitions(schema_globs, *, cwd=None)”Resolves one glob (or a list of globs), imports each matching Python module, collects every module-level schema definition, and returns them canonicalized. Raises SchemaLoaderError when no files match (NO_MATCH_MESSAGE) and ModuleLoadError when a schema module fails to import.
definitions = load_schema_definitions("./src/db/schema/**/*.py")collect_definitions_from_module(mod)
Section titled “collect_definitions_from_module(mod)”The lower-level collector: takes a module’s __dict__-style mapping, walks its values (including nested lists and tuples), and returns the schema definitions found, deduplicated and canonicalized.
Canonicalization
Section titled “Canonicalization”canonicalize_definitions(definitions, *, index_granularity=None)
Section titled “canonicalize_definitions(definitions, *, index_granularity=None)”Normalizes definitions into the canonical form the diff engine compares — engine normalization, key-clause splitting, codec canonicalization — deduplicates by identity, and sorts deterministically (tables, then views, then materialized views; then by database and name). Both sides of every diff are canonicalized first, so cosmetic differences ('MergeTree' vs 'MergeTree()', ['id, org_id'] vs ['id', 'org_id']) never produce operations. A MergeTree table without index_granularity gets index_granularity (the server’s default, clickhouse.indexGranularity), 8192 when None.
definition_key(definition)
Section titled “definition_key(definition)”Returns the stable identity string "<kind>:<database>.<name>" used for deduplication and operation keys.
Validation
Section titled “Validation”validate_definitions(definitions)
Section titled “validate_definitions(definitions)”Returns a list of ValidationIssue objects — empty when the definitions are valid. Each issue has a code (e.g. duplicate_column_name, order_by_missing_column, codec_chain_must_end_with_general), the offending object’s kind, database, and name, and a human-readable message.
assert_valid_definitions(definitions)
Section titled “assert_valid_definitions(definitions)”Same checks, but raises ChxValidationError (carrying the issue list as .issues) instead of returning them. The planner and SQL renderer call this internally, so invalid definitions cannot reach SQL generation.
collect_definition_warnings(definitions)
Section titled “collect_definition_warnings(definitions)”Returns the ValidationIssue objects that generate and check print as warnings but never fail on, such as quoted_string_default for a plain string default written as a quoted SQL literal ("'standard'").
Planning
Section titled “Planning”plan_diff(old_definitions, new_definitions, *, index_granularity=None)
Section titled “plan_diff(old_definitions, new_definitions, *, index_granularity=None)”Canonicalizes both sides (with index_granularity, as canonicalize_definitions), validates the new side, and returns a MigrationPlan:
operations— orderedMigrationOperationlist; each has atype(e.g.create_table,alter_table_add_column,drop_table), akeyidentifying the object, arisklevel, and the renderedsql.risk_summary— counts ofsafe/caution/dangeroperations.rename_suggestions— detected drop+add column pairs that look like renames, each with theconfirmation_sqlto apply the rename instead.
Risk levels drive CLI behavior: danger operations (drops, table recreates) are blocked by chkit migrate unless --allow-destructive is passed.
plan = plan_diff(old, new)if plan.risk_summary.danger > 0: raise SystemExit("plan contains destructive operations")Snapshots
Section titled “Snapshots”create_snapshot(definitions, *, index_granularity=None)
Section titled “create_snapshot(definitions, *, index_granularity=None)”Canonicalizes the definitions (with index_granularity, as canonicalize_definitions) and returns a SnapshotV1 — version: 1, a UTC generated_at timestamp, and the canonical definition list. Serialized with camelCase aliases it has the same JSON shape as the TypeScript chkit/meta/snapshot.json:
snapshot = create_snapshot(definitions)payload = snapshot.model_dump(mode="json", by_alias=True)Loading goes through the same model: SnapshotV1.model_validate_json(text) accepts snapshots written by either implementation.
SQL rendering
Section titled “SQL rendering”to_create_sql(definition)
Section titled “to_create_sql(definition)”Renders a single definition to its CREATE TABLE / CREATE VIEW / CREATE MATERIALIZED VIEW DDL string. Validates first, so it raises ChxValidationError on an invalid definition.
apply_on_cluster_to_plan(plan, cluster)
Section titled “apply_on_cluster_to_plan(plan, cluster)”Post-pass that stamps ON CLUSTER '<name>' into every DDL statement of a plan (and into each rename suggestion’s confirmation SQL). No-op when cluster is None. This is how clickhouse.cluster from the config takes effect — the planner itself stays cluster-agnostic.
Configuration
Section titled “Configuration”define_config(config)
Section titled “define_config(config)”Anchors the config object in clickhouse.config.py. Accepts a ChxUserConfig instance, a plain dict (validated through Pydantic on entry), or a callable (env: ChxConfigEnv) -> config for dynamic per-command configs (see the CI/CD guide):
import os
from chkit import define_config
config = define_config( { "schema": "./src/db/schema/**/*.py", "outDir": "./chkit", "clickhouse": { "url": os.environ.get("CLICKHOUSE_URL", "http://localhost:8123"), "username": os.environ.get("CLICKHOUSE_USER", "default"), "password": os.environ.get("CLICKHOUSE_PASSWORD", ""), "database": os.environ.get("CLICKHOUSE_DB", "default"), }, })resolve_config(config)
Section titled “resolve_config(config)”Fills in defaults and returns a ChxResolvedConfig: out_dir defaults to ./chkit, migrations_dir to <outDir>/migrations, meta_dir to <outDir>/meta; the check flags default to true and safety.allowDestructive to false; ClickHouse credentials default to default / empty password / database default. A clickhouse.cluster value is validated here (identifier or {macro} form) — invalid names fail fast with a ValueError. clickhouse.indexGranularity defaults to 8192 and must be a positive integer; configured_index_granularity(config) reads it.
The option keys and defaults match the TypeScript config — see the Configuration Overview.
Related
Section titled “Related”- Schema DSL Reference — building the definitions this pipeline consumes, with synced TypeScript/Python examples.
- CLI:
chkit generate— the CLI wrapper aroundplan_diff. - CLI:
chkit migrate— how plans are applied and journaled.