1
0
Fork 0
mirror of https://github.com/element-hq/synapse.git synced 2025-03-14 11:48:44 +00:00
synapse/synapse/util
Erik Johnston 81e0f57800
Fix perf when streams don't change often (#17767)
There is a bug with the `StreamChangeCache` where it would incorrectly
return that all entities had changed if asked for entities changed
*since* the earliest stream position.

Note that for streams we use the inequalities: `$min_stream_id <
stream_id <= $max_stream_id`, i.e. when we ask the stream change cache
for all things that have changed since `$stream_id` we don't care for
events that happened *at* `$stream_id`.

Specifically: `_earliest_known_stream_pos` is the position at which we
know that we'll have entries for all changes since that point, we can
use the cache for any stream IDs that equal
`_earliest_known_stream_pos`.

`_earliest_known_stream_pos` is set in three places:
- On startup we set it either to:
  - the current maximum stream ID, with not prefilled values; or
  - the minimum of the latest N values we pulled from the DB
- When we evict items from the bottom, we set it to the stream ID of the
evicted items.

This was changed in https://github.com/matrix-org/synapse/pull/14435,
but I think we were overly conservative there.

---------

Co-authored-by: Andrew Morgan <1342360+anoadragon453@users.noreply.github.com>
2024-09-30 13:52:33 +01:00
..
caches Fix perf when streams don't change often (#17767) 2024-09-30 13:52:33 +01:00
__init__.py Ensure that pending to-device events are sent over federation at startup (#16925) 2024-03-22 13:24:11 +00:00
async_helpers.py Reduce concurrent thread usage in media (#17567) 2024-08-14 12:41:53 +01:00
batching_queue.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
cancellation.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
check_dependencies.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
daemonize.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
distributor.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
events.py Add a utility function for generating fake event IDs (#17557) 2024-08-13 16:55:05 +00:00
file_consumer.py Update license headers 2023-11-21 15:29:58 -05:00
frozenutils.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
gai_resolver.py Bump twisted from 22.10.0 to 23.8.0 (#16235) 2023-09-05 11:14:14 +00:00
hash.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
httpresourcetree.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
iterutils.py Bump black from 23.10.1 to 24.2.0 (#16936) 2024-03-13 16:46:44 +00:00
linked_list.py Format files with Ruff (#17643) 2024-09-02 12:39:04 +01:00
logcontext.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
logformatter.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
macaroons.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
manhole.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
metrics.py Format files with Ruff (#17643) 2024-09-02 12:39:04 +01:00
module_loader.py Update license headers 2023-11-21 15:29:58 -05:00
msisdn.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
patch_inline_callbacks.py Format files with Ruff (#17643) 2024-09-02 12:39:04 +01:00
ratelimitutils.py Format files with Ruff (#17643) 2024-09-02 12:39:04 +01:00
retryutils.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
rlimit.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
rust.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
stringutils.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
task_scheduler.py Add logging to tasks managed by the task scheduler, showing CPU and database usage. (#17219) 2024-05-22 14:12:58 +01:00
templates.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00
threepids.py Update license headers 2023-11-21 15:29:58 -05:00
wheel_timer.py Correctly mention previous copyright (#16820) 2024-01-23 11:26:48 +00:00