Migrations and retention · Consumer

Install the inbox/outbox schema explicitly and distinguish startup preflight from bounded cleanup.

Lily does not create inbox/outbox tables or collections during ordinary Consumer startup. Migration is an explicit deployment step; startup performs a read-only check of the storage contract already installed. Starting a new binary therefore does not silently change the schema.

Deployment sequence#

Prepare connection configuration and the secrets resolver for the selected database or factory cell.
Install business tables or collections with your application’s migration tool.
Run the Lily component migrator through the same DI database authority.
Close the migration command’s container and start Consumer after success.
Observe startup preflight and readiness.

The complete PostgreSQL and MongoDB examples include a migrate(&ApplicationContainer) function. These are compilable migration bodies, rather than standalone executables that load configuration, build a container and manage its shutdown. Your deployment command supplies that environment. The transactional migration installs only framework storage; it does not automatically create the example’s business orders table.

Storage installed by each backend#

Backend	Public migration API	Owned storage
PostgreSQL	`PostgresInboxOutboxMigrator::new(database).migrate().await`	The `lily_queue` schema; component version 1 and a migration advisory lock
MongoDB	`MongoInboxOutboxMigrator::new(database)?.apply(&operation).await`	`_lily_queue_transactional_schema`, `_lily_queue_inbox`, `_lily_queue_inbox_leases`, `_lily_queue_outbox`

Preflight checks more than the version number. PostgreSQL checks the actual schema, constraint and index contract; MongoDB also checks its marker, fingerprint, validators and indexes. MongoDB deployment support for transactions is validated separately. Manually changing a marker or removing a required index is not a compatible migration.

Retention and cleanup policy#

[rabbitmq.topology.queues.transactional_inbox]
backend = "postgresql"
inbox_retention_secs = 604800
outbox_retention_secs = 604800
cleanup_interval_secs = 3600

This table represents the existing queue binding; do not add a second table with the same path. Change the backend for MongoDB and add database_cell for factory mode. Retention defaults to seven days and the cleanup interval to one hour. The cleanup interval cannot exceed either retention horizon.

Cleanup removes completed inbox and delivered outbox records in bounded batches. MongoDB also handles expired lease records. Undelivered outbox records are not silently discarded just because a retention period elapsed. CleanupReport describes cleanup work; it is not a complete audit ledger of application events.

Design the replay window around retention#

After an inbox record is removed, the same handler/event key can be accepted as new work. Account for broker redelivery, manual replay and remote-service retry windows when choosing retention. If business idempotency must last longer, implement it with a business-data uniqueness constraint or idempotency record; a short inbox retention window is not a permanent exactly-once guarantee.

Queues sharing one database cell must use the same complete transactional policy, including retention and cleanup. Consider separate storage authorities or cells when retention requirements differ. When changing retention, account for when existing inbox history will be deleted and whether old events may be processed again.