# Re-creating a Table Whose Keys Are Foreign Keys Elsewhere > **Problem:** you need to rebuild a table (drop a column that participates in an > FK, change nullability, etc. — anything SQLite's limited `ALTER TABLE` cannot > do), but other tables hold foreign keys into it and the database has live rows. > A naive `DROP TABLE` fails with `FOREIGN KEY constraint failed`, on D1 no > matter which executor you use (dashboard, CLI, `migrations apply`). > > Last reviewed: 2026-10-05 (written after `0007_region_scoping.sql` forced us > to rebuild a 13-table subgraph on D1). ## 1. Why the textbook pattern dies on D1 The standard SQLite table-rebuild recipe relies on switching enforcement off around the rebuild: ```sql PRAGMA foreign_keys = OFF; -- or PRAGMA defer_foreign_keys = ON; -- ... create-copy-drop-rename ... PRAGMA foreign_keys = ON; ``` D1 **enforces foreign keys and ignores both pragmas**. `DROP TABLE parent` runs an implicit `DELETE FROM parent` first, which trips every live child row (`ON DELETE no action`). Verified locally: the DROP fails with exit 19 while the identical statements pass with enforcement off. There is no executor-side workaround — dashboard, `d1 execute --file`, and `migrations apply` all fail the same way. (Side effect: `scripts/clear-db.sql`'s `PRAGMA foreign_keys=OFF` likewise cannot protect a full wipe on D1 remote.) On an **empty** database the same migration passes (no child rows → the implicit delete violates nothing). So this failure only appears on populated DBs, which is why it slips past dev-time testing. ## 2. The pattern: rebuild the whole FK subgraph, leaves first Since no referenced table with live rows can be dropped, rebuild **every table in the connected subgraph** so that at each DROP, nothing references the table being dropped. Every step is individually FK-legal, so per-statement autocommit (D1 dashboard/CLI) is fine — no transaction wrapping needed. ### Step 0 — Map the full subgraph (not just direct children) Audit `schema.ts` for every `references(() => T)` where `T` is in your set, **recursively**. Grandchildren matter: in our case `coupon_usage.order_item_id -> order_items(id)` meant `order_items` was not a leaf and `coupon_usage` had to join the rebuild set. Also audit for anything the schema may not know about (legacy tables, views, triggers) — see pre-flight below. Classify each table: - **Leaf** — nothing references it. Rebuild directly with the final schema (NOT NULL preserved natively, no temp columns needed). - **Mid-level** — referenced by other tables in the set. Build the replacement under a `__new_` name first; it gets dropped/renamed later once unreferenced. ### Step 1 — Pre-flight on the live target (read-only, mandatory) ```sql SELECT type, name FROM sqlite_master WHERE (sql LIKE '%%' OR sql LIKE '%%') AND name NOT LIKE 'sqlite_%'; ``` Repeat the `LIKE` for every table in the set. The result must show **only** the known tables plus their indexes. Anything else (legacy table, view, trigger) must be folded into the plan before proceeding — the migration cannot branch at runtime. Also confirm no index covers a column you plan to `DROP COLUMN` (SQLite blocks dropping indexed columns; FK-child columns are droppable, PK/UNIQUE columns are not). ### Step 2 — Create + copy, parents before children ```sql CREATE TABLE `__new_parent` ( ... final schema ... ); INSERT INTO `__new_parent` (...) SELECT ... FROM `parent`; CREATE TABLE `__new_child` ( ... FOREIGN KEY (`pid`) REFERENCES `__new_parent`(`id`) ON UPDATE no action ON DELETE no action ); INSERT INTO `__new_child` (...) SELECT ... FROM `child`; ``` - Preserve explicit `id`s so existing cross-references stay valid. - `INSERT` validates FKs immediately under autocommit, so fill parents first. - Point `__new_` FKs at `__new_` names. SQLite resolves FK parents **by name at enforcement time**, and `RENAME TABLE` (3.25+) rewrites other tables' FK clauses to the new name — so after the renames everything points at the final tables. (Verified: post-rename `sqlite_master` shows `REFERENCES "parent"`, and `foreign_key_check` is clean.) - Copy **all** columns explicitly; `sqlite_sequence` stays correct automatically since ids are preserved. ### Step 3 — Drop originals, leaves first ```sql DROP TABLE `leaf_1`; DROP TABLE `leaf_2`; -- ... DROP TABLE `mid_child`; -- now unreferenced DROP TABLE `parent`; -- now unreferenced ``` Each DROP succeeds because nothing references the table at that moment. ### Step 4 — Rename top-down, then recreate indexes ```sql ALTER TABLE `__new_parent` RENAME TO `parent`; ALTER TABLE `__new_child` RENAME TO `child`; -- ... CREATE UNIQUE INDEX `` ON `` (...); ``` Create indexes **after** the drops: index names live in a global namespace, so reusing the original names before dropping the old tables collides. (Non-unique indexes too — same namespace.) ### Step 5 — Verify ```sql PRAGMA foreign_key_check; -- must return zero rows -- plus per-table COUNT(*) compared against pre-migration counts ``` Also exercise post-rename writes once: an INSERT violating a CHECK / FK / NOT NULL must still fail (proves constraints survived the rename — see pitfalls). ## 3. Pitfalls (each one bit us) 1. **CHECK constraints are NOT rewritten by RENAME.** Only other tables' FK clauses follow a rename. A drizzle-rendered `CHECK("old_name"."col" ...)` dangles after promotion and breaks all subsequent writes. Write CHECKs **unqualified**: `CHECK(col >= 1 AND ...)`. 2. **No `ALTER COLUMN` in SQLite** (syntax error). Nullability can only change via rebuild — which is why mid-level tables go through `__new_` tables instead of `ADD/DROP COLUMN` surgery. (`ADD ... REFERENCES ... NOT NULL DEFAULT` is explicitly rejected by SQLite, so temp FK columns would be stuck nullable — another reason to rebuild leaves directly.) 3. **`DROP COLUMN` works on FK-child columns** (verified) but is unnecessary in this pattern — prefer full rebuilds so NOT NULL survives everywhere. 4. **Transcribe schemas from the migration target state, not from history.** Old migration files (e.g. `0000`) show stale shapes (integer timestamps, pre-rename columns). The `__new_` definitions must match post-migration reality = current `schema.ts` (we cross-checked every column/index/default against it). 5. **`NUMERIC` affinity** (`numericText`) has no precedent in our migration history but is valid SQLite — safe to emit. 6. **Fix the audit, not just the symptom.** The `coupon_usage -> order_items` edge was found by grepping all `references()` targets, not by trial and error. One missed edge = same FK failure one level down. ## 4. Proof harness `0007`'s chain was proven before shipping by extracting the migration block verbatim and running it statement-by-statement in autocommit with `PRAGMA foreign_keys=ON` against an in-memory DB with all tables populated, asserting: row counts preserved, `foreign_key_check` clean, no `__new_` leftovers, indexes present, and post-rename CHECK/FK/NOT NULL enforcement alive. Reuse that approach for the next rebuild: populate → extract → run → assert. (Scratch script used: `/tmp`-adjacent `proof_0007.py`; not committed.)