1
0
Fork 0
mirror of https://github.com/element-hq/synapse.git synced 2025-03-06 16:06:52 +00:00
synapse/tests/storage
Erik Johnston c46d452c7c
Fix bug where purging history could lead to increase in disk space usage (#18131)
When purging history, we try and delete any state groups that become
unreferenced (i.e. there are no longer any events that directly
reference them). When we delete a state group that is referenced by
another state group, we "de-delta" that state group so that it no longer
refers to the state group that is deleted.

There are two bugs with this approach that we fix here:
1. There is a common pattern where we end up storing two state groups
when persisting a state event: the state before and after the new state
event, where the latter is stored as a delta to the former. When
deleting state groups we only deleted the "new" state and left (and
potentially de-deltaed) the old state. This was due to a bug/typo when
trying to find referenced state groups.
2. There are times where we store unreferenced state groups in the DB,
during the purging of history these would not get rechecked and instead
always de-deltaed. Instead, we should check for this case and delete any
unreferenced state groups rather than de-deltaing them.

The effect of the above bugs is that when purging history we'd end up
with lots of unreferenced state groups that had been de-deltaed (i.e.
stored as the full state). This can lead to dramatic increases in
storage space used.
2025-02-03 19:04:19 +00:00
..
databases Sliding Sync: Slight optimization when fetching state for the room (get_events_as_list(...)) (#17718) 2024-10-14 13:47:35 +01:00
util Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
__init__.py Reference Matrix Home Server 2014-08-12 15:10:52 +01:00
test__base.py Sliding Sync: Pre-populate room data for quick filtering/sorting (#17512) 2024-08-29 16:09:51 +01:00
test_account_data.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_appservice.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_background_update.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_base.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_cleanup_extrems.py Bump black from 23.10.1 to 24.2.0 (#16936) 2024-03-13 16:46:44 +00:00
test_client_ips.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_database.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_devices.py Reintroduce "Reduce device lists replication traffic."" (#17361) 2024-06-25 10:34:34 +01:00
test_directory.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_e2e_room_keys.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_end_to_end_keys.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_event_chain.py Reintroduce #17291. (#17338) 2024-06-24 14:40:28 +00:00
test_event_federation.py Reintroduce #17291. (#17338) 2024-06-24 14:40:28 +00:00
test_event_metrics.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_event_push_actions.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_events.py Sliding Sync: Pre-populate room data for quick filtering/sorting (#17512) 2024-08-29 16:09:51 +01:00
test_id_generators.py Set our own stream position from the current sequence value on startup (#17309) 2024-06-17 11:50:00 +00:00
test_main.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_monthly_active_users.py Update license headers 2023-11-21 15:29:58 -05:00
test_profile.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_purge.py Fix bug where purging history could lead to increase in disk space usage (#18131) 2025-02-03 19:04:19 +00:00
test_receipts.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_redaction.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_registration.py Add support for MSC3823 - Account Suspension (#17051) 2024-05-01 17:45:17 +01:00
test_relations.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_rollback_worker.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_room.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_room_search.py Return the search terms as search highlights for SQLite instead of nothing (#17000) 2024-04-26 09:43:52 +01:00
test_roommember.py Sliding Sync: Pre-populate room data for quick filtering/sorting (#17512) 2024-08-29 16:09:51 +01:00
test_sliding_sync_tables.py Sliding Sync: Reset forgotten status when membership changes (like rejoining a room) (#17835) 2024-10-22 11:06:46 +01:00
test_state.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_state_deletion.py Add locking to more safely delete state groups: Part 2 (#18130) 2025-02-03 17:58:55 +00:00
test_stream.py Bust _membership_stream_cache cache when current state changes (#17732) 2025-01-08 10:11:09 -06:00
test_transactions.py Update license headers 2023-11-21 15:29:58 -05:00
test_txn_limit.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_unsafe_locale.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
test_user_directory.py Handle hyphens in user dir search porperly (#17254) 2024-06-05 10:40:34 +01:00
test_user_filters.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00