7.2 KiB
Re-creating a Table Whose Keys Are Foreign Keys Elsewhere
Problem: you need to rebuild a table (drop a column that participates in an FK, change nullability, etc. — anything SQLite's limited
ALTER TABLEcannot do), but other tables hold foreign keys into it and the database has live rows. A naiveDROP TABLEfails withFOREIGN KEY constraint failed, on D1 no matter which executor you use (dashboard, CLI,migrations apply).Last reviewed: 2026-10-05 (written after
0007_region_scoping.sqlforced us to rebuild a 13-table subgraph on D1).
1. Why the textbook pattern dies on D1
The standard SQLite table-rebuild recipe relies on switching enforcement off around the rebuild:
PRAGMA foreign_keys = OFF; -- or PRAGMA defer_foreign_keys = ON;
-- ... create-copy-drop-rename ...
PRAGMA foreign_keys = ON;
D1 enforces foreign keys and ignores both pragmas. DROP TABLE parent
runs an implicit DELETE FROM parent first, which trips every live child row
(ON DELETE no action). Verified locally: the DROP fails with exit 19 while
the identical statements pass with enforcement off. There is no executor-side
workaround — dashboard, d1 execute --file, and migrations apply all fail
the same way. (Side effect: scripts/clear-db.sql's PRAGMA foreign_keys=OFF
likewise cannot protect a full wipe on D1 remote.)
On an empty database the same migration passes (no child rows → the implicit delete violates nothing). So this failure only appears on populated DBs, which is why it slips past dev-time testing.
2. The pattern: rebuild the whole FK subgraph, leaves first
Since no referenced table with live rows can be dropped, rebuild every table in the connected subgraph so that at each DROP, nothing references the table being dropped. Every step is individually FK-legal, so per-statement autocommit (D1 dashboard/CLI) is fine — no transaction wrapping needed.
Step 0 — Map the full subgraph (not just direct children)
Audit schema.ts for every references(() => T) where T is in your set,
recursively. Grandchildren matter: in our case coupon_usage.order_item_id -> order_items(id) meant order_items was not a leaf and coupon_usage had
to join the rebuild set. Also audit for anything the schema may not know about
(legacy tables, views, triggers) — see pre-flight below.
Classify each table:
- Leaf — nothing references it. Rebuild directly with the final schema (NOT NULL preserved natively, no temp columns needed).
- Mid-level — referenced by other tables in the set. Build the replacement
under a
__new_<t>name first; it gets dropped/renamed later once unreferenced.
Step 1 — Pre-flight on the live target (read-only, mandatory)
SELECT type, name FROM sqlite_master
WHERE (sql LIKE '%<child_table>%' OR sql LIKE '%<parent_table>%')
AND name NOT LIKE 'sqlite_%';
Repeat the LIKE for every table in the set. The result must show only the
known tables plus their indexes. Anything else (legacy table, view, trigger)
must be folded into the plan before proceeding — the migration cannot branch
at runtime.
Also confirm no index covers a column you plan to DROP COLUMN (SQLite blocks
dropping indexed columns; FK-child columns are droppable, PK/UNIQUE columns
are not).
Step 2 — Create + copy, parents before children
CREATE TABLE `__new_parent` ( ... final schema ... );
INSERT INTO `__new_parent` (...) SELECT ... FROM `parent`;
CREATE TABLE `__new_child` (
...
FOREIGN KEY (`pid`) REFERENCES `__new_parent`(`id`) ON UPDATE no action ON DELETE no action
);
INSERT INTO `__new_child` (...) SELECT ... FROM `child`;
- Preserve explicit
ids so existing cross-references stay valid. INSERTvalidates FKs immediately under autocommit, so fill parents first.- Point
__new_FKs at__new_names. SQLite resolves FK parents by name at enforcement time, andRENAME TABLE(3.25+) rewrites other tables' FK clauses to the new name — so after the renames everything points at the final tables. (Verified: post-renamesqlite_mastershowsREFERENCES "parent", andforeign_key_checkis clean.) - Copy all columns explicitly;
sqlite_sequencestays correct automatically since ids are preserved.
Step 3 — Drop originals, leaves first
DROP TABLE `leaf_1`;
DROP TABLE `leaf_2`;
-- ...
DROP TABLE `mid_child`; -- now unreferenced
DROP TABLE `parent`; -- now unreferenced
Each DROP succeeds because nothing references the table at that moment.
Step 4 — Rename top-down, then recreate indexes
ALTER TABLE `__new_parent` RENAME TO `parent`;
ALTER TABLE `__new_child` RENAME TO `child`;
-- ...
CREATE UNIQUE INDEX `<original_name>` ON `<table>` (...);
Create indexes after the drops: index names live in a global namespace, so reusing the original names before dropping the old tables collides. (Non-unique indexes too — same namespace.)
Step 5 — Verify
PRAGMA foreign_key_check; -- must return zero rows
-- plus per-table COUNT(*) compared against pre-migration counts
Also exercise post-rename writes once: an INSERT violating a CHECK / FK / NOT NULL must still fail (proves constraints survived the rename — see pitfalls).
3. Pitfalls (each one bit us)
- CHECK constraints are NOT rewritten by RENAME. Only other tables' FK
clauses follow a rename. A drizzle-rendered
CHECK("old_name"."col" ...)dangles after promotion and breaks all subsequent writes. Write CHECKs unqualified:CHECK(col >= 1 AND ...). - No
ALTER COLUMNin SQLite (syntax error). Nullability can only change via rebuild — which is why mid-level tables go through__new_tables instead ofADD/DROP COLUMNsurgery. (ADD ... REFERENCES ... NOT NULL DEFAULTis explicitly rejected by SQLite, so temp FK columns would be stuck nullable — another reason to rebuild leaves directly.) DROP COLUMNworks on FK-child columns (verified) but is unnecessary in this pattern — prefer full rebuilds so NOT NULL survives everywhere.- Transcribe schemas from the migration target state, not from history.
Old migration files (e.g.
0000) show stale shapes (integer timestamps, pre-rename columns). The__new_definitions must match post-migration reality = currentschema.ts(we cross-checked every column/index/default against it). NUMERICaffinity (numericText) has no precedent in our migration history but is valid SQLite — safe to emit.- Fix the audit, not just the symptom. The
coupon_usage -> order_itemsedge was found by grepping allreferences()targets, not by trial and error. One missed edge = same FK failure one level down.
4. Proof harness
0007's chain was proven before shipping by extracting the migration block
verbatim and running it statement-by-statement in autocommit with
PRAGMA foreign_keys=ON against an in-memory DB with all tables populated,
asserting: row counts preserved, foreign_key_check clean, no __new_
leftovers, indexes present, and post-rename CHECK/FK/NOT NULL enforcement
alive. Reuse that approach for the next rebuild: populate → extract → run →
assert. (Scratch script used: /tmp-adjacent proof_0007.py; not committed.)