Skip to main content

Module conduit

Module conduit 

Source

Re-exportsΒ§

use std::iter::from_fn; πŸ”’
use std::path::Path; πŸ”’
use std::path::PathBuf; πŸ”’
use std::sync::Arc; πŸ”’
use std::time::Duration; πŸ”’
use bytes::Bytes; πŸ”’
use futures::Stream; πŸ”’
use futures::StreamExt; πŸ”’
use futures::TryStreamExt; πŸ”’
use object_store::Error as ObjectStoreError; πŸ”’
use ruma::CanonicalJsonObject; πŸ”’
use ruma::CanonicalJsonValue; πŸ”’
use ruma::Mxc; πŸ”’
use ruma::OwnedUserId; πŸ”’
use ruma::RoomId; πŸ”’
use ruma::ServerName; πŸ”’
use ruma::UserId; πŸ”’
use tokio::time::sleep; πŸ”’
use tuwunel_core::Err; πŸ”’
use tuwunel_core::Error; πŸ”’
use tuwunel_core::Result; πŸ”’
use tuwunel_core::debug_warn; πŸ”’
use tuwunel_core::err; πŸ”’
use tuwunel_core::error; πŸ”’
use tuwunel_core::error; πŸ”’
use tuwunel_core::info; πŸ”’
use tuwunel_core::info; πŸ”’
use tuwunel_core::itertools::Itertools; πŸ”’
use tuwunel_core::result::NotFound; πŸ”’
use tuwunel_core::utils::ReadyExt; πŸ”’
use tuwunel_core::utils::TryReadyExt; πŸ”’
use tuwunel_core::utils::content_disposition::make_content_disposition; πŸ”’
use tuwunel_core::utils::stream::TryBroadbandExt; πŸ”’
use tuwunel_core::utils::stream::TryIgnore; πŸ”’
use tuwunel_core::warn; πŸ”’
use tuwunel_database::Map; πŸ”’
use tuwunel_database::SEP; πŸ”’
use super::tally::Outcome; πŸ”’
use super::tally::Tally; πŸ”’
use crate::Services; πŸ”’
use crate::integrity::pdu_room_ids::Finding; πŸ”’
use crate::integrity::pdu_room_ids::classify; πŸ”’
use crate::integrity::pdu_room_ids::outliers; πŸ”’
use crate::integrity::pdu_room_ids::timeline; πŸ”’
use crate::storage::Provider; πŸ”’

StructsΒ§

ConduitMediaEntry πŸ”’
One parsed servernamemediaid_metadata entry, borrowing the raw key/value.

EnumsΒ§

MediaSource πŸ”’
Selects the source of a Conduit database’s original media files.

ConstantsΒ§

PROVIDER_READ_ATTEMPTS πŸ”’
Attempts to read each original from a source storage provider before the import gives up: one initial try plus one retry. The object store performs its own internal retries under each attempt, so a transient provider blip rarely reaches this outer limit.
PROVIDER_READ_RETRY_DELAY πŸ”’
Pause between provider read retries, giving a transient fault time to clear.

FunctionsΒ§

conduit_media_blocked πŸ”’
Whether a Conduit media entry was blocked by a moderator. Conduit keeps the file and refuses it only at read time (blocked_servername_mediaid); tuwunel has no per-media blocklist, so importing a blocked original would serve it again. The blocklist key is server_name 0xff media_id, the same bytes as the servernamemediaid_metadata key, so the entry’s raw key probes it directly. Only a clean miss imports; a hard read error aborts the import (like an unreachable source) rather than silently re-serving blocked media.
conduit_media_key πŸ”’
The object-store key of a Conduit content-addressed media object, the same shard segments as conduit_media_path joined by /. The source provider’s base_path supplies any bucket prefix (Conduit’s media.path).
conduit_media_owner πŸ”’
The local owner of a Conduit media entry from servernamemediaid_userlocalpart; None for remote media, which has no such entry.
conduit_media_path πŸ”’
Reconstructs the on-disk path of a Conduit content-addressed media file from the lowercase SHA-256 hex digest naming it, matching Conduit’s split_media_path: media_dir joined with the digest’s shard segments.
conduit_shards πŸ”’
Splits a lowercase SHA-256 hex digest into Conduit’s shard segments: depth segments of length characters then the remainder, or the whole digest when depth is zero (a flat layout). config::check bounds depth * length below the digest length so the segments never overrun it.
copy_cf πŸ”’
Copies every row of one column verbatim into another.
import_conduit_original πŸ”’
Imports one servernamemediaid_metadata entry and reports whether it was stored.
is_missing_object πŸ”’
Whether a provider read failed because the object is absent (a 404 or a dangling metadata row), which is skipped rather than retried. tuwunel’s Error::is_not_found does not cover the object-store variant, so match it directly.
media_source πŸ”’
Resolves the configured Conduit media source. A named provider must already be defined under [storage_provider]; its absence is an operator error that aborts the import rather than silently dropping every file.
migrate_conduit_knocks πŸ”’
Imports Conduit’s pending knocks. Conduit names the columns roomuserid_knockcount / userroomid_knockstate; tuwunel renamed them to *knocked* but kept the byte layout (room_id 0xff user_id -> u64 count; user_id 0xff room_id -> JSON stripped state), so each row copies verbatim. Imported once: tuwunel clears a knock on the user’s join or leave, so a re-import would resurrect a knock the user has already resolved.
migrate_conduit_media πŸ”’
Imports the original media files of a Conduit database, re-uploading each servernamemediaid_metadata entry through media.create.
migrate_pdus πŸ”’
Writes its room into each stored event lacking its room_id.
parse_conduit_media_entry πŸ”’
Parses a servernamemediaid_metadata entry: the key is servername 0xff media_id, the value is the digest, filename and content type.
parse_conduit_media_value πŸ”’
Splits a servernamemediaid_metadata value into its digest, filename, and content type. The value is sha256(32) | filename | 0xff | content_type with an optional trailing 0xff that Conduit’s media-auth migration appends to flag unauthenticated access; that flag is ignored.
read_conduit_original πŸ”’
Reads one Conduit original, named by its content digest, from the configured media source. A filesystem file that cannot be read is reported as None (skipped, like a dangling metadata row). A source storage provider is retried on a transient fault; a persistent one returns Err so the import aborts instead of dropping reachable media.
read_provider_original πŸ”’
Reads one original from the source storage provider. An absent object is skipped (Ok(None)) like a dangling filesystem row; a transient fault is retried up to PROVIDER_READ_ATTEMPTS times, and a persistent one aborts the import with an Err.
repair πŸ”’
Writes the room into the event a finding names, as it stands now.
repair_all πŸ”’
rooted πŸ”’
sha256_hex πŸ”’
Lowercase hex digest, matching the names Conduit gives its media files.
write πŸ”’
Writes an event back with its room.