Skip to main content

Module migrations

Module migrations 

Source
Expand description

One-time database migrations.

A fresh database is stamped current and a legacy database is walked through the named migrations, once the version and server name gates decide it is safe to touch. Each step stamps a marker once it has run, and removing the marker of a step safe to run twice schedules it for the next start.

Re-exportsΒ§

use std::cmp::Ordering; πŸ”’
use std::time::Duration; πŸ”’
use futures::FutureExt; πŸ”’
use futures::StreamExt; πŸ”’
use futures::TryStreamExt; πŸ”’
use ruma::OwnedUserId; πŸ”’
use ruma::RoomId; πŸ”’
use ruma::ServerName; πŸ”’
use ruma::UserId; πŸ”’
use tokio::time::sleep; πŸ”’
use tuwunel_core::Err; πŸ”’
use tuwunel_core::Result; πŸ”’
use tuwunel_core::err; πŸ”’
use tuwunel_core::format_small_string; πŸ”’
use tuwunel_core::info; πŸ”’
use tuwunel_core::info; πŸ”’
use tuwunel_core::result::NotFound; πŸ”’
use tuwunel_core::smallstr::SmallString; πŸ”’
use tuwunel_core::smallvec::SmallVec; πŸ”’
use tuwunel_core::utils::BoolExt; πŸ”’
use tuwunel_core::utils::IterStream; πŸ”’
use tuwunel_core::utils::ReadyExt; πŸ”’
use tuwunel_core::utils::TryReadyExt; πŸ”’
use tuwunel_core::warn; πŸ”’
use tuwunel_database::Deserialized; πŸ”’
use self::Rerun::Clear; πŸ”’
use self::Rerun::Refuse; πŸ”’
use self::account_status::migrate_account_status; πŸ”’
use self::clear_servername_status::clear_servername_status; πŸ”’
use self::clear_state_local_error_memos::clear_state_local_error_memos; πŸ”’
use self::conduit::migrate_pdus; πŸ”’
use self::email_bindings::migrate_email_bindings; πŸ”’
use self::fix_bad_double_separator_in_state_cache::fix_bad_double_separator_in_state_cache; πŸ”’
use self::fix_hashed_sentinel_passwords::fix_hashed_sentinel_passwords; πŸ”’
use self::fix_readreceiptid_readreceipt_duplicates::fix_readreceiptid_readreceipt_duplicates; πŸ”’
use self::fix_referencedevents_missing_sep::fix_referencedevents_missing_sep; πŸ”’
use self::import_conduit_knocks::import_conduit_knocks; πŸ”’
use self::injectivity::fix as fix_injectivity; πŸ”’
use self::injectivity::mark_clean as mark_clean_injectivity; πŸ”’
use self::migrate_media::migrate_media; πŸ”’
use self::migrate_profile_keys::migrate_profile_keys; πŸ”’
use self::moderation::migrate as migrate_moderation; πŸ”’
use self::rebuild_roomid_tscount_pducount::rebuild_roomid_tscount_pducount; πŸ”’
use self::redactions::restore_redactions; πŸ”’
use self::remove_remote_media_userid::remove_remote_media_userid; πŸ”’
use self::retroactively_fix_bad_data_from_roomuserid_joined::retroactively_fix_bad_data_from_roomuserid_joined; πŸ”’
use self::split_conduit_highlight_counts::split_conduit_highlight_counts; πŸ”’
use self::token_expiry::Restore; πŸ”’
use self::token_expiry::migrate as migrate_token_expiry; πŸ”’
use self::token_expiry::restore as restore_token_expiry; πŸ”’
use self::token_expiry::restore_due; πŸ”’
use self::upgrade_legacy_mediaid_user::upgrade_legacy_mediaid_user; πŸ”’
use crate::Services; πŸ”’
use crate::integrity::find as find_check; πŸ”’
use crate::integrity::forbidden_names::Username; πŸ”’
use crate::integrity::forbidden_names::matched as forbidden_patterns; πŸ”’
use crate::integrity::forbidden_names::usernames as forbidden_usernames; πŸ”’

ModulesΒ§

account_status πŸ”’
clear_servername_status πŸ”’
clear_state_local_error_memos πŸ”’
conduit πŸ”’
email_bindings πŸ”’
fix_bad_double_separator_in_state_cache πŸ”’
fix_hashed_sentinel_passwords πŸ”’
fix_readreceiptid_readreceipt_duplicates πŸ”’
fix_referencedevents_missing_sep πŸ”’
import_conduit_knocks πŸ”’
injectivity πŸ”’
Repair reported short-id residue while preserving unproven data.
migrate_media πŸ”’
migrate_profile_keys πŸ”’
moderation πŸ”’
rebuild_roomid_tscount_pducount πŸ”’
redactions πŸ”’
remove_remote_media_userid πŸ”’
retroactively_fix_bad_data_from_roomuserid_joined πŸ”’
split_conduit_highlight_counts πŸ”’
tally πŸ”’
token_expiry πŸ”’
upgrade_legacy_mediaid_user πŸ”’

StructsΒ§

Stamp
A step stamp that rerun removed or found absent.

EnumsΒ§

Rerun πŸ”’
What rerun does for an integrity check.

ConstantsΒ§

ADOPTION πŸ”’
ADOPT_ACCOUNT_STATUS πŸ”’
ADOPT_EMAIL_BINDINGS πŸ”’
CLEAR_STATE_LOCAL_ERROR_MEMOS πŸ”’
DATABASE_VERSION πŸ”’
The current schema version.
EVERY_START πŸ”’
FIX_PDU_MISSING_ROOM_ID πŸ”’
FORCE_MIGRATION_DELAY πŸ”’
FOREIGN_LINEAGE_MARKER πŸ”’
A marker written by a sibling conduwuit-lineage server but never by tuwunel. Its presence identifies a foreign database at a higher schema number even after tuwunel has stamped its own server_name, so a database opened by both servers in turn keeps booting rather than being refused as too new.
MARKERS πŸ”’
The steps every database records as settled, whether they ran or it began current.
MEDIA_CHECKUP πŸ”’
MEDIA_RENAME πŸ”’
NO_REPAIR πŸ”’
OUTCOME_RECORD πŸ”’
PROFILE_COPY πŸ”’
REBUILD_LIVE_THREAD_ACTIVITY πŸ”’
REBUILD_RELATESTO_TYPED πŸ”’
REBUILD_THREAD_SUMMARIES πŸ”’
REDACTIONS πŸ”’
SERVER_NAME_KEY πŸ”’
SHA256_MEDIA πŸ”’
Stamped once the media directory names files by their sha256 names.
SPLIT_CONDUIT_HIGHLIGHT πŸ”’
UNVERIFIED πŸ”’

StaticsΒ§

RERUNS πŸ”’
How the repair of each integrity check is scheduled, or why it cannot be.

FunctionsΒ§

backfill_server_name πŸ”’
Stamp the marker on a database that pre-dates SERVER_NAME_KEY by probing for any user from the configured server. If none, the database belongs to a different server and reuse is refused.
check_database_version πŸ”’
Gate the discovered schema version before migrations and the server_name backfill run. The integer is comparable only within tuwunel’s own lineage; a foreign database (Conduit and forks) numbers schema on a colliding ladder and is recognized as foreign by is_foreign_lineage, so its number is not gated. Within our lineage a version below 13 is refused as unmigratable and one above this build as too new to open safely; force_migration overrides the latter for a deliberate downgrade.
check_server_name πŸ”’
Matrix resource ownership is based on the server name; changing it requires recreating the database from scratch. The marker is stamped once in fresh(); pre-marker databases are backfilled by probing for any user from the configured server.
fresh πŸ”’
is_foreign_lineage πŸ”’
Whether the database comes from a foreign (non-tuwunel) lineage: it predates our SERVER_NAME_KEY stamp, or carries a conduwuit-lineage migration marker that persists even after we stamp ours. Must be read before the server_name backfill, which removes the first signal.
local_user_id πŸ”’
Assembles a local user id from a localpart a foreign column records.
marker_present πŸ”’
Whether a migration step has stamped its marker.
migrate πŸ”’
Apply any migrations
migrations πŸ”’
pending πŸ”’
Whether a named migration step still needs to run, refusing once shutdown begins.
rebuild_thread_summaries πŸ”’
Rebuilds every stored thread summary, logging how many roots changed.
rerun
Schedules the repair of an integrity check for the next start.
rerun_markers πŸ”’
run_once πŸ”’
Runs a pass that finishes whenever it succeeds, stamping its marker.
scan_room_aliases πŸ”’
stamp πŸ”’
until_finished πŸ”’
Runs a pass whose work may wait on a condition outside the database.
warn_forbidden_names πŸ”’
Warns about existing names the configuration now forbids.

Type AliasesΒ§

Stamps πŸ”’
UserIdBuf πŸ”’
Inline budget for a local user id assembled from a foreign localpart.