Convex Migration Helper

1---2name: convex-migration-helper3description: Plans and executes safe Convex schema and data migrations using the widen-migrate-narrow workflow and the @convex-dev/migrations component. Use this skill when a deployment fails schema validation, existing documents need backfilling, fields need adding or removing or changing type, tables need splitting or merging, or a zero-downtime migration strategy is needed. Also use when the user mentions breaking schema changes, multi-deploy rollouts, or data transformations on existing Convex tables.4---5 6# Convex Migration Helper7 8Safely migrate Convex schemas and data when making breaking changes.9 10## When to Use11 12- Adding new required fields to existing tables13- Changing field types or structure14- Splitting or merging tables15- Renaming or deleting fields16- Migrating from nested to relational data17 18## When Not to Use19 20- Greenfield schema with no existing data in production or dev21- Adding optional fields that do not need backfilling22- Adding new tables with no existing data to migrate23- Adding or removing indexes with no correctness concern24- Questions about Convex schema design without a migration need25 26## Key Concepts27 28### Schema Validation Drives the Workflow29 30Convex will not let you deploy a schema that does not match the data at rest. This is the fundamental constraint that shapes every migration:31 32- You cannot add a required field if existing documents don't have it33- You cannot change a field's type if existing documents have the old type34- You cannot remove a field from the schema if existing documents still have it35 36This means migrations follow a predictable pattern: **widen the schema, migrate the data, narrow the schema**.37 38### Online Migrations39 40Convex migrations run online, meaning the app continues serving requests while data is updated asynchronously in batches. During the migration window, your code must handle both old and new data formats.41 42### Prefer New Fields Over Changing Types43 44When changing the shape of data, create a new field rather than modifying an existing one. This makes the transition safer and easier to roll back.45 46### Don't Delete Data47 48Unless you are certain, prefer deprecating fields over deleting them. Mark the field as `v.optional` and add a code comment explaining it is deprecated and why it existed.49 50## Safe Changes (No Migration Needed)51 52### Adding Optional Field53 54```typescript55// Before56users: defineTable({57  name: v.string(),58});59 60// After - safe, new field is optional61users: defineTable({62  name: v.string(),63  bio: v.optional(v.string()),64});65```66 67### Adding New Table68 69```typescript70posts: defineTable({71  userId: v.id("users"),72  title: v.string(),73}).index("by_user", ["userId"]);74```75 76### Adding Index77 78```typescript79users: defineTable({80  name: v.string(),81  email: v.string(),82}).index("by_email", ["email"]);83```84 85## Breaking Changes: The Deployment Workflow86 87Every breaking migration follows the same multi-deploy pattern:88 89**Deploy 1 - Widen the schema:**90 911. Update schema to allow both old and new formats (e.g., add optional new field)922. Update code to handle both formats when reading933. Update code to write the new format for new documents944. Deploy95 96**Between deploys - Migrate data:**97 985. Run migration to backfill existing documents996. Verify all documents are migrated100 101**Deploy 2 - Narrow the schema:**102 1037. Update schema to require the new format only1048. Remove code that handles the old format1059. Deploy106 107## Using the Migrations Component108 109For any non-trivial migration, use the [`@convex-dev/migrations`](https://www.convex.dev/components/migrations) component. It handles batching, cursor-based pagination, state tracking, resume from failure, dry runs, and progress monitoring.110 111See `references/migrations-component.md` for installation, setup, defining and running migrations, dry runs, status monitoring, and configuration options.112 113## Common Migration Patterns114 115See `references/migration-patterns.md` for complete patterns with code examples covering:116 117- Adding a required field118- Deleting a field119- Changing a field type120- Splitting nested data into a separate table121- Cleaning up orphaned documents122- Zero-downtime strategies (dual write, dual read)123- Small table shortcut (single internalMutation without the component)124- Verifying a migration is complete125 126## Common Pitfalls127 1281. **Making a field required before migrating data**: Convex rejects the deploy because existing documents lack the field. Always widen the schema first.1292. **Using `.collect()` on large tables**: Hits transaction limits or causes timeouts. Use the migrations component for proper batched pagination. `.collect()` is only safe for tables you know are small.1303. **Not writing the new format before migrating**: Documents created during the migration window will be missed, leaving unmigrated data after the migration "completes."1314. **Skipping the dry run**: Use `dryRun: true` to validate migration logic before committing changes to production data. Catches bugs before they touch real documents.1325. **Deleting fields prematurely**: Prefer deprecating with `v.optional` and a comment. Only delete after you are confident the data is no longer needed and no code references it.1336. **Using crons for migration batches**: The migrations component handles batching via recursive scheduling internally. Crons require manual cleanup and an extra deploy to remove.134 135## Migration Checklist136 137- [ ] Identify the breaking change and plan the multi-deploy workflow138- [ ] Update schema to allow both old and new formats139- [ ] Update code to handle both formats when reading140- [ ] Update code to write the new format for new documents141- [ ] Deploy widened schema and updated code142- [ ] Define migration using the `@convex-dev/migrations` component143- [ ] Test with `dryRun: true`144- [ ] Run migration and monitor status145- [ ] Verify all documents are migrated146- [ ] Update schema to require new format only147- [ ] Clean up code that handled old format148- [ ] Deploy final schema and code149- [ ] Remove migration code once confirmed stable