Merge branch 'develop' into madlittlemods/17368-bust-_membership_stream_cache

2024-12-14 11:57:44 +00:00 · 2024-11-27 13:11:39 -06:00 · 2024-11-27 13:11:39 -06:00 · f40e64940a
commit f40e64940a
parent 20ff239b62 59ad4b18fc
66 changed files with 947 additions and 441 deletions
--- a/CHANGES.md
+++ b/CHANGES.md
@ -1,3 +1,59 @@
 # Synapse 1.120.0 (2024-11-26)
 ### Bugfixes
 - Fix a bug introduced in Synapse v1.120rc1 which would cause the newly-introduced `delete_old_otks` job to fail in worker-mode deployments. ([\#17960](https://github.com/element-hq/synapse/issues/17960))
 # Synapse 1.120.0rc1 (2024-11-20)
 This release enables the enforcement of authenticated media by default, with exemptions for media that is already present in the
 homeserver's media store.
 Most homeservers operating in the public federation will not be impacted by this change, given that
 the large homeserver `matrix.org` enabled this in September 2024 and therefore most clients and servers
 will already have updated as a result.
 Some server administrators may still wish to disable this enforcement for the time being, in the interest of compatibility with older clients
 and older federated homeservers.
 See the [upgrade notes](https://element-hq.github.io/synapse/v1.120/upgrade.html#authenticated-media-is-now-enforced-by-default) for more information.
 ### Features
 - Enforce authenticated media by default. Administrators can revert this by configuring `enable_authenticated_media` to `false`. In a future release of Synapse, this option will be removed and become always-on. ([\#17889](https://github.com/element-hq/synapse/issues/17889))
 - Add a one-off task to delete old One-Time Keys, to guard against us having old OTKs in the database that the client has long forgotten about. ([\#17934](https://github.com/element-hq/synapse/issues/17934))
 ### Improved Documentation
 - Clarify the semantics of the `enable_authenticated_media` configuration option. ([\#17913](https://github.com/element-hq/synapse/issues/17913))
 - Add documentation about backing up Synapse. ([\#17931](https://github.com/element-hq/synapse/issues/17931))
 ### Deprecations and Removals
 - Remove support for [MSC3886: Simple client rendezvous capability](https://github.com/matrix-org/matrix-spec-proposals/pull/3886), which has been superseded by [MSC4108](https://github.com/matrix-org/matrix-spec-proposals/pull/4108) and therefore closed. ([\#17638](https://github.com/element-hq/synapse/issues/17638))
 ### Internal Changes
 - Addressed some typos in docs and returned error message for unknown MXC ID. ([\#17865](https://github.com/element-hq/synapse/issues/17865))
 - Unpin the upload release GHA action. ([\#17923](https://github.com/element-hq/synapse/issues/17923))
 - Bump macOS version used to build wheels during release, as current version used is end-of-life. ([\#17924](https://github.com/element-hq/synapse/issues/17924))
 - Move server event filtering logic to Rust. ([\#17928](https://github.com/element-hq/synapse/issues/17928))
 - Support new package name of PyPI package `python-multipart` 0.0.13 so that distro packagers do not need to work around name conflict with PyPI package `multipart`. ([\#17932](https://github.com/element-hq/synapse/issues/17932))
 - Speed up slow initial sliding syncs on large servers. ([\#17946](https://github.com/element-hq/synapse/issues/17946))
 ### Updates to locked dependencies
 * Bump anyhow from 1.0.92 to 1.0.93. ([\#17920](https://github.com/element-hq/synapse/issues/17920))
 * Bump bleach from 6.1.0 to 6.2.0. ([\#17918](https://github.com/element-hq/synapse/issues/17918))
 * Bump immutabledict from 4.2.0 to 4.2.1. ([\#17941](https://github.com/element-hq/synapse/issues/17941))
 * Bump packaging from 24.1 to 24.2. ([\#17940](https://github.com/element-hq/synapse/issues/17940))
 * Bump phonenumbers from 8.13.49 to 8.13.50. ([\#17942](https://github.com/element-hq/synapse/issues/17942))
 * Bump pygithub from 2.4.0 to 2.5.0. ([\#17917](https://github.com/element-hq/synapse/issues/17917))
 * Bump ruff from 0.7.2 to 0.7.3. ([\#17919](https://github.com/element-hq/synapse/issues/17919))
 * Bump serde from 1.0.214 to 1.0.215. ([\#17938](https://github.com/element-hq/synapse/issues/17938))
 # Synapse 1.119.0 (2024-11-13)
 No significant changes since 1.119.0rc2.
--- a/Cargo.lock
+++ b/Cargo.lock
@ -35,12 +35,6 @@ version = "0.21.7"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "9d297deb1925b89f2ccc13d7635fa0714f12c87adce1c75356b39ca9b7178567"
 [[package]]
 name = "bitflags"
 version = "2.5.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "cf4b9d6a944f767f8e5e0db018570623c85f3d925ac718db4e06d0187adb21c1"
 [[package]]
 name = "blake2"
 version = "0.10.6"
@ -162,9 +156,9 @@ dependencies = [
 [[package]]
 name = "heck"
-version = "0.4.1"
+version = "0.5.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "95505c38b4572b2d910cecb0281560f54b440a19336cbbcb27bf6ce6adc6f5a8"
+checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
 [[package]]
 name = "hex"
@ -222,16 +216,6 @@ version = "0.2.154"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "ae743338b92ff9146ce83992f766a31066a91a8c84a45e0e9f21e7cf6de6d346"
 [[package]]
 name = "lock_api"
 version = "0.4.12"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "07af8b9cdd281b7915f413fa73f29ebd5d55d0d3f0155584dade1ff18cea1b17"
 dependencies = [
 "autocfg",
 "scopeguard",
 ]
 [[package]]
 name = "log"
 version = "0.4.22"
@ -265,29 +249,6 @@ version = "1.19.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "3fdb12b2476b595f9358c5161aa467c2438859caa136dec86c26fdd2efe17b92"
 [[package]]
 name = "parking_lot"
 version = "0.12.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "7e4af0ca4f6caed20e900d564c242b8e5d4903fdacf31d3daf527b66fe6f42fb"
 dependencies = [
 "lock_api",
 "parking_lot_core",
 ]
 [[package]]
 name = "parking_lot_core"
 version = "0.9.10"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "1e401f977ab385c9e4e3ab30627d6f26d00e2c73eef317493c4ec6d468726cf8"
 dependencies = [
 "cfg-if",
 "libc",
 "redox_syscall",
 "smallvec",
 "windows-targets",
 ]
 [[package]]
 name = "portable-atomic"
 version = "1.6.0"
@ -311,16 +272,16 @@ dependencies = [
 [[package]]
 name = "pyo3"
-version = "0.21.2"
+version = "0.23.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "a5e00b96a521718e08e03b1a622f01c8a8deb50719335de3f60b3b3950f069d8"
+checksum = "f54b3d09cbdd1f8c20650b28e7b09e338881482f4aa908a5f61a00c98fba2690"
 dependencies = [
 "anyhow",
 "cfg-if",
 "indoc",
 "libc",
 "memoffset",
- "parking_lot",
+ "once_cell",
 "portable-atomic",
 "pyo3-build-config",
 "pyo3-ffi",
@ -330,9 +291,9 @@ dependencies = [
 [[package]]
 name = "pyo3-build-config"
-version = "0.21.2"
+version = "0.23.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "7883df5835fafdad87c0d888b266c8ec0f4c9ca48a5bed6bbb592e8dedee1b50"
+checksum = "3015cf985888fe66cfb63ce0e321c603706cd541b7aec7ddd35c281390af45d8"
 dependencies = [
 "once_cell",
 "target-lexicon",
@ -340,9 +301,9 @@ dependencies = [
 [[package]]
 name = "pyo3-ffi"
-version = "0.21.2"
+version = "0.23.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "01be5843dc60b916ab4dad1dca6d20b9b4e6ddc8e15f50c47fe6d85f1fb97403"
+checksum = "6fca7cd8fd809b5ac4eefb89c1f98f7a7651d3739dfb341ca6980090f554c270"
 dependencies = [
 "libc",
 "pyo3-build-config",
@ -350,9 +311,9 @@ dependencies = [
 [[package]]
 name = "pyo3-log"
-version = "0.10.0"
+version = "0.12.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "2af49834b8d2ecd555177e63b273b708dea75150abc6f5341d0a6e1a9623976c"
+checksum = "3eb421dc86d38d08e04b927b02424db480be71b777fa3a56f32e2f2a3a1a3b08"
 dependencies = [
 "arc-swap",
 "log",
@ -361,9 +322,9 @@ dependencies = [
 [[package]]
 name = "pyo3-macros"
-version = "0.21.2"
+version = "0.23.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "77b34069fc0682e11b31dbd10321cbf94808394c56fd996796ce45217dfac53c"
+checksum = "34e657fa5379a79151b6ff5328d9216a84f55dc93b17b08e7c3609a969b73aa0"
 dependencies = [
 "proc-macro2",
 "pyo3-macros-backend",
@ -373,9 +334,9 @@ dependencies = [
 [[package]]
 name = "pyo3-macros-backend"
-version = "0.21.2"
+version = "0.23.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "08260721f32db5e1a5beae69a55553f56b99bd0e1c3e6e0a5e8851a9d0f5a85c"
+checksum = "295548d5ffd95fd1981d2d3cf4458831b21d60af046b729b6fd143b0ba7aee2f"
 dependencies = [
 "heck",
 "proc-macro2",
@ -386,9 +347,9 @@ dependencies = [
 [[package]]
 name = "pythonize"
-version = "0.21.1"
+version = "0.23.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "9d0664248812c38cc55a4ed07f88e4df516ce82604b93b1ffdc041aa77a6cb3c"
+checksum = "91a6ee7a084f913f98d70cdc3ebec07e852b735ae3059a1500db2661265da9ff"
 dependencies = [
 "pyo3",
 "serde",
@ -433,15 +394,6 @@ dependencies = [
 "getrandom",
 ]
 [[package]]
 name = "redox_syscall"
 version = "0.5.1"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "469052894dcb553421e483e4209ee581a45100d31b4018de03e5a7ad86374a7e"
 dependencies = [
 "bitflags",
 ]
 [[package]]
 name = "regex"
 version = "1.11.1"
@ -477,12 +429,6 @@ version = "1.0.18"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "f3cb5ba0dc43242ce17de99c180e96db90b235b8a9fdc9543c96d2209116bd9f"
 [[package]]
 name = "scopeguard"
 version = "1.2.0"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "94143f37725109f92c262ed2cf5e59bce7498c01bcc1502d7b9afe439a4e9f49"
 [[package]]
 name = "serde"
 version = "1.0.215"
@ -505,9 +451,9 @@ dependencies = [
 [[package]]
 name = "serde_json"
-version = "1.0.132"
+version = "1.0.133"
 source = "registry+https://github.com/rust-lang/crates.io-index"
-checksum = "d726bfaff4b320266d395898905d0eba0345aae23b54aee3a737e260fd46db03"
+checksum = "c7fceb2473b9166b2294ef05efcb65a3db80803f0b03ef86a5fc88a2b85ee377"
 dependencies = [
 "itoa",
 "memchr",
@ -537,12 +483,6 @@ dependencies = [
 "digest",
 ]
 [[package]]
 name = "smallvec"
 version = "1.13.2"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "3c5e1a9a646d36c3599cd173a41282daf47c44583ad367b8e6837255952e5c67"
 [[package]]
 name = "subtle"
 version = "2.5.0"
@ -694,67 +634,3 @@ dependencies = [
 "js-sys",
 "wasm-bindgen",
 ]
 [[package]]
 name = "windows-targets"
 version = "0.52.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "6f0713a46559409d202e70e28227288446bf7841d3211583a4b53e3f6d96e7eb"
 dependencies = [
 "windows_aarch64_gnullvm",
 "windows_aarch64_msvc",
 "windows_i686_gnu",
 "windows_i686_gnullvm",
 "windows_i686_msvc",
 "windows_x86_64_gnu",
 "windows_x86_64_gnullvm",
 "windows_x86_64_msvc",
 ]
 [[package]]
 name = "windows_aarch64_gnullvm"
 version = "0.52.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "7088eed71e8b8dda258ecc8bac5fb1153c5cffaf2578fc8ff5d61e23578d3263"
 [[package]]
 name = "windows_aarch64_msvc"
 version = "0.52.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "9985fd1504e250c615ca5f281c3f7a6da76213ebd5ccc9561496568a2752afb6"
 [[package]]
 name = "windows_i686_gnu"
 version = "0.52.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "88ba073cf16d5372720ec942a8ccbf61626074c6d4dd2e745299726ce8b89670"
 [[package]]
 name = "windows_i686_gnullvm"
 version = "0.52.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "87f4261229030a858f36b459e748ae97545d6f1ec60e5e0d6a3d32e0dc232ee9"
 [[package]]
 name = "windows_i686_msvc"
 version = "0.52.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "db3c2bf3d13d5b658be73463284eaf12830ac9a26a90c717b7f771dfe97487bf"
 [[package]]
 name = "windows_x86_64_gnu"
 version = "0.52.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "4e4246f76bdeff09eb48875a0fd3e2af6aada79d409d33011886d3e1581517d9"
 [[package]]
 name = "windows_x86_64_gnullvm"
 version = "0.52.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "852298e482cd67c356ddd9570386e2862b5673c85bd5f88df9ab6802b334c596"
 [[package]]
 name = "windows_x86_64_msvc"
 version = "0.52.5"
 source = "registry+https://github.com/rust-lang/crates.io-index"
 checksum = "bec47e5bfd1bff0eeaf6d8b485cc1074891a197ab4225d504cb7a1ab88b02bf0"
--- a/build_rust.py
+++ b/build_rust.py
@ -1,8 +1,10 @@
 # A build script for poetry that adds the rust extension.
 import itertools
 import os
 from typing import Any, Dict
 from packaging.specifiers import SpecifierSet
 from setuptools_rust import Binding, RustExtension
@ -14,6 +16,8 @@ def build(setup_kwargs: Dict[str, Any]) -> None:
        target="synapse.synapse_rust",
        path=cargo_toml_path,
        binding=Binding.PyO3,
        # This flag is a no-op in the latest versions. Instead, we need to
        # specify this in the `bdist_wheel` config below.
        py_limited_api=True,
        # We force always building in release mode, as we can't tell the
        # difference between using `poetry` in development vs production.
@ -21,3 +25,18 @@ def build(setup_kwargs: Dict[str, Any]) -> None:
    )
    setup_kwargs.setdefault("rust_extensions", []).append(extension)
    setup_kwargs["zip_safe"] = False
    # We lookup the minimum supported python version by looking at
    # `python_requires` (e.g. ">=3.9.0,<4.0.0") and finding the first python
    # version that matches. We then convert that into the `py_limited_api` form,
    # e.g. cp39 for python 3.9.
    py_limited_api: str
    python_bounds = SpecifierSet(setup_kwargs["python_requires"])
    for minor_version in itertools.count(start=8):
        if f"3.{minor_version}.0" in python_bounds:
            py_limited_api = f"cp3{minor_version}"
            break
    setup_kwargs.setdefault("options", {}).setdefault("bdist_wheel", {})[
        "py_limited_api"
    ] = py_limited_api
--- a/changelog.d/17253.misc
+++ b/changelog.d/17253.misc
@ -0,0 +1 @@
 [MSC4108](https://github.com/matrix-org/matrix-spec-proposals/pull/4108): Add a `Content-Type` header on the `PUT` response to work around a faulty behavior in some caching reverse proxies.
--- a/changelog.d/17638.removal
+++ b/changelog.d/17638.removal
@ -1 +0,0 @@
 Remove support for closed [MSC3886](https://github.com/matrix-org/matrix-spec-proposals/pull/3886).
--- a/changelog.d/17865.misc
+++ b/changelog.d/17865.misc
@ -1 +0,0 @@
 Addressed some typos in docs and returned error message for unknown MXC ID.
--- a/changelog.d/17872.doc
+++ b/changelog.d/17872.doc
@ -0,0 +1 @@
 Add OIDC example configuration for Forgejo (fork of Gitea).
--- a/changelog.d/17913.doc
+++ b/changelog.d/17913.doc
@ -1 +0,0 @@
 Clarify the semantics of the `enable_authenticated_media` configuration option.
--- a/changelog.d/17923.misc
+++ b/changelog.d/17923.misc
@ -1 +0,0 @@
 Unpin the upload release GHA action.
--- a/changelog.d/17924.misc
+++ b/changelog.d/17924.misc
@ -1 +0,0 @@
 Bump macos version used to build wheels during release, as current version used is end-of-life.
--- a/changelog.d/17928.misc
+++ b/changelog.d/17928.misc
@ -1 +0,0 @@
 Move server event filtering logic to rust.
--- a/changelog.d/17931.doc
+++ b/changelog.d/17931.doc
@ -1 +0,0 @@
 Add documentation about backing up Synapse.
--- a/changelog.d/17933.bugfix
+++ b/changelog.d/17933.bugfix
@ -0,0 +1 @@
 Fix long-standing bug where read receipts could get overly delayed being sent over federation.
--- a/changelog.d/17936.misc
+++ b/changelog.d/17936.misc
@ -0,0 +1 @@
 Fix incorrect comment in new schema delta.
--- a/changelog.d/17944.misc
+++ b/changelog.d/17944.misc
@ -0,0 +1 @@
 Raise setuptools_rust version cap to 1.10.2.
--- a/changelog.d/17945.misc
+++ b/changelog.d/17945.misc
@ -0,0 +1 @@
 Enable encrypted appservice related experimental features in the complement docker image.
--- a/changelog.d/17952.misc
+++ b/changelog.d/17952.misc
@ -0,0 +1 @@
 Return whether the user is suspended when querying the user account in the Admin API.
--- a/changelog.d/17953.doc
+++ b/changelog.d/17953.doc
@ -0,0 +1 @@
 Link to element-docker-demo from contrib/docker*.
--- a/changelog.d/17966.misc
+++ b/changelog.d/17966.misc
@ -0,0 +1 @@
 Bump pyo3 and dependencies to v0.23.2.
--- a/changelog.d/17969.misc
+++ b/changelog.d/17969.misc
@ -0,0 +1 @@
 Update setuptools-rust and fix building abi3 wheels in latest version.
--- a/contrib/docker/README.md
+++ b/contrib/docker/README.md
@ -30,3 +30,6 @@ docker-compose up -d
 ### More information
 For more information on required environment variables and mounts, see the main docker documentation at [/docker/README.md](../../docker/README.md)
 **For a more comprehensive Docker Compose example showcasing a full Matrix 2.0 stack, please see
 https://github.com/element-hq/element-docker-demo**
--- a/contrib/docker_compose_workers/README.md
+++ b/contrib/docker_compose_workers/README.md
@ -8,6 +8,9 @@ All examples and snippets assume that your Synapse service is called `synapse` i
 An example Docker Compose file can be found [here](docker-compose.yaml).
 **For a more comprehensive Docker Compose example, showcasing a full Matrix 2.0 stack (originally based on this
 docker-compose.yaml), please see https://github.com/element-hq/element-docker-demo**
 ## Worker Service Examples in Docker Compose
 In order to start the Synapse container as a worker, you must specify an `entrypoint` that loads both the `homeserver.yaml` and the configuration for the worker (`synapse-generic-worker-1.yaml` in the example below). You must also include the worker type in the environment variable `SYNAPSE_WORKER` or alternatively pass `-m synapse.app.generic_worker` as part of the `entrypoint` after `"/start.py", "run"`).
--- a/debian/changelog
+++ b/debian/changelog
@ -1,3 +1,15 @@
 matrix-synapse-py3 (1.120.0) stable; urgency=medium
  * New synapse release 1.120.0.
 -- Synapse Packaging team <packages@matrix.org>  Tue, 26 Nov 2024 13:10:23 +0000
 matrix-synapse-py3 (1.120.0~rc1) stable; urgency=medium
  * New Synapse release 1.120.0rc1.
 -- Synapse Packaging team <packages@matrix.org>  Wed, 20 Nov 2024 15:02:21 +0000
 matrix-synapse-py3 (1.119.0) stable; urgency=medium
  * New Synapse release 1.119.0.
--- a/docker/complement/conf/workers-shared-extra.yaml.j2
+++ b/docker/complement/conf/workers-shared-extra.yaml.j2
@ -104,6 +104,16 @@ experimental_features:
  msc3967_enabled: true
  # Expose a room summary for public rooms
  msc3266_enabled: true
  # Send to-device messages to application services
  msc2409_to_device_messages_enabled: true
  # Allow application services to masquerade devices
  msc3202_device_masquerading: true
  # Sending device list changes, one-time key counts and fallback key usage to application services
  msc3202_transaction_extensions: true
  # Proxy OTK claim requests to exclusive ASes
  msc3983_appservice_otk_claims: true
  # Proxy key queries to exclusive ASes
  msc3984_appservice_key_query: true
 server_notices:
  system_mxid_localpart: _server
--- a/docs/admin_api/user_admin_api.md
+++ b/docs/admin_api/user_admin_api.md
@ -55,7 +55,8 @@ It returns a JSON body like the following:
        }
    ],
    "user_type": null,
-    "locked": false
+    "locked": false,
    "suspended": false
 }
 ```
--- a/docs/openid.md
+++ b/docs/openid.md
@ -336,6 +336,36 @@ but it has a `response_types_supported` which excludes "code" (which we rely on,
 is even mentioned in their [documentation](https://developers.facebook.com/docs/facebook-login/manually-build-a-login-flow#login)),
 so we have to disable discovery and configure the URIs manually.
 ### Forgejo
 Forgejo is a fork of Gitea that can act as an OAuth2 provider.
 The implementation of OAuth2 is improved compared to Gitea, as it provides a correctly defined `subject_claim` and `scopes`.
 Synapse config:
 ```yaml
 oidc_providers:
  - idp_id: forgejo
    idp_name: Forgejo
    discover: false
    issuer: "https://your-forgejo.com/"
    client_id: "your-client-id" # TO BE FILLED
    client_secret: "your-client-secret" # TO BE FILLED
    client_auth_method: client_secret_post
    scopes: ["openid", "profile", "email", "groups"]
    authorization_endpoint: "https://your-forgejo.com/login/oauth/authorize"
    token_endpoint: "https://your-forgejo.com/login/oauth/access_token"
    userinfo_endpoint: "https://your-forgejo.com/api/v1/user"
    user_mapping_provider:
      config:
        subject_claim: "sub"
        picture_claim: "picture"
        localpart_template: "{{ user.preferred_username }}"
        display_name_template: "{{ user.name }}"
        email_template: "{{ user.email }}"
 ```
 ### GitHub
 [GitHub][github-idp] is a bit special as it is not an OpenID Connect compliant provider, but
--- a/docs/upgrade.md
+++ b/docs/upgrade.md
@ -128,6 +128,29 @@ removing the experimental support for it in this release.
 The `experimental_features.msc3886_endpoint` configuration option has
 been removed.
 ## Authenticated media is now enforced by default
 The [`enable_authenticated_media`] configuration option now defaults to true.
 This means that clients and remote (federated) homeservers now need to use
 the authenticated media endpoints in order to download media from your
 homeserver.
 As an exception, existing media that was stored on the server prior to
 this option changing to `true` will still be accessible over the
 unauthenticated endpoints.
 The matrix.org homeserver has already been running with this option enabled
 since September 2024, so most common clients and homeservers should already
 be compatible.
 With that said, administrators who wish to disable this feature for broader
 compatibility can still do so by manually configuring
 `enable_authenticated_media: False`.
 [`enable_authenticated_media`]: usage/configuration/config_documentation.md#enable_authenticated_media
 # Upgrading to v1.119.0
 ## Minimum supported Python version
--- a/docs/usage/configuration/config_documentation.md
+++ b/docs/usage/configuration/config_documentation.md
@ -1887,8 +1887,7 @@ Config options related to Synapse's media store.
 When set to true, all subsequent media uploads will be marked as authenticated, and will not be available over legacy
 unauthenticated media endpoints (`/_matrix/media/(r0|v3|v1)/download` and `/_matrix/media/(r0|v3|v1)/thumbnail`) - requests for authenticated media over these endpoints will result in a 404. All media, including authenticated media, will be available over the authenticated media endpoints `_matrix/client/v1/media/download` and `_matrix/client/v1/media/thumbnail`. Media uploaded prior to setting this option to true will still be available over the legacy endpoints. Note if the setting is switched to false
-after enabling, media marked as authenticated will be available over legacy endpoints. Defaults to false, but
+after enabling, media marked as authenticated will be available over legacy endpoints. Defaults to true (previously false). In a future release of Synapse, this option will be removed and become always-on.
 this will change to true in a future Synapse release.
 In all cases, authenticated requests to download media will succeed, but for unauthenticated requests, this
 case-by-case breakdown describes whether media downloads are permitted:
@ -1910,9 +1909,11 @@ will perpetually be available over the legacy, unauthenticated endpoint, even af
 This is for backwards compatibility with older clients and homeservers that do not yet support requesting authenticated media;
 those older clients or homeservers will not be cut off from media they can already see.
 _Changed in Synapse 1.120:_ This option now defaults to `True` when not set, whereas before this version it defaulted to `False`.
 Example configuration:
 ```yaml
-enable_authenticated_media: true
+enable_authenticated_media: false
 ```
 ---
 ### `enable_media_repo`
--- a/flake.lock
+++ b/flake.lock
@ -56,24 +56,6 @@
        "type": "github"
      }
    },
    "flake-utils_2": {
      "inputs": {
        "systems": "systems_2"
      },
      "locked": {
        "lastModified": 1681202837,
        "narHash": "sha256-H+Rh19JDwRtpVPAWp64F+rlEtxUWBAQW28eAi3SRSzg=",
        "owner": "numtide",
        "repo": "flake-utils",
        "rev": "cfacdce06f30d2b68473a46042957675eebb3401",
        "type": "github"
      },
      "original": {
        "owner": "numtide",
        "repo": "flake-utils",
        "type": "github"
      }
    },
    "gitignore": {
      "inputs": {
        "nixpkgs": [
@ -202,11 +184,11 @@
    },
    "nixpkgs_3": {
      "locked": {
-        "lastModified": 1681358109,
+        "lastModified": 1728538411,
-        "narHash": "sha256-eKyxW4OohHQx9Urxi7TQlFBTDWII+F+x2hklDOQPB50=",
+        "narHash": "sha256-f0SBJz1eZ2yOuKUr5CA9BHULGXVSn6miBuUWdTyhUhU=",
        "owner": "NixOS",
        "repo": "nixpkgs",
-        "rev": "96ba1c52e54e74c3197f4d43026b3f3d92e83ff9",
+        "rev": "b69de56fac8c2b6f8fd27f2eca01dcda8e0a4221",
        "type": "github"
      },
      "original": {
@ -249,20 +231,19 @@
        "devenv": "devenv",
        "nixpkgs": "nixpkgs_2",
        "rust-overlay": "rust-overlay",
-        "systems": "systems_3"
+        "systems": "systems_2"
      }
    },
    "rust-overlay": {
      "inputs": {
        "flake-utils": "flake-utils_2",
        "nixpkgs": "nixpkgs_3"
      },
      "locked": {
-        "lastModified": 1693966243,
+        "lastModified": 1731897198,
-        "narHash": "sha256-a2CA1aMIPE67JWSVIGoGtD3EGlFdK9+OlJQs0FOWCKY=",
+        "narHash": "sha256-Ou7vLETSKwmE/HRQz4cImXXJBr/k9gp4J4z/PF8LzTE=",
        "owner": "oxalica",
        "repo": "rust-overlay",
-        "rev": "a8b4bb4cbb744baaabc3e69099f352f99164e2c1",
+        "rev": "0be641045af6d8666c11c2c40e45ffc9667839b5",
        "type": "github"
      },
      "original": {
@ -300,21 +281,6 @@
        "repo": "default",
        "type": "github"
      }
    },
    "systems_3": {
      "locked": {
        "lastModified": 1681028828,
        "narHash": "sha256-Vy1rq5AaRuLzOxct8nz4T6wlgyUR7zLU309k9mBC768=",
        "owner": "nix-systems",
        "repo": "default",
        "rev": "da67096a3b9bf56a91d16901293e51ba5b49a27e",
        "type": "github"
      },
      "original": {
        "owner": "nix-systems",
        "repo": "default",
        "type": "github"
      }
    }
  },
  "root": "root",
--- a/flake.nix
+++ b/flake.nix
@ -82,7 +82,7 @@
                  #
                  # NOTE: We currently need to set the Rust version unnecessarily high
                  # in order to work around https://github.com/matrix-org/synapse/issues/15939
-                  (rust-bin.stable."1.71.1".default.override {
+                  (rust-bin.stable."1.82.0".default.override {
                    # Additionally install the "rust-src" extension to allow diving into the
                    # Rust source code in an IDE (rust-analyzer will also make use of it).
                    extensions = [ "rust-src" ];
@ -205,7 +205,7 @@
                # corresponding Nix packages on https://search.nixos.org/packages.
                #
                # This was done until `./install-deps.pl --dryrun` produced no output.
-                env.PERL5LIB = "${with pkgs.perl536Packages; makePerlPath [
+                env.PERL5LIB = "${with pkgs.perl538Packages; makePerlPath [
                  DBI
                  ClassMethodModifiers
                  CryptEd25519
--- a/poetry.lock
+++ b/poetry.lock
@ -1,4 +1,4 @@
-# This file is automatically @generated by Poetry 1.8.3 and should not be changed by hand.
+# This file is automatically @generated by Poetry 1.8.4 and should not be changed by hand.
 [[package]]
 name = "annotated-types"
@ -2405,19 +2405,18 @@ test = ["build[virtualenv] (>=1.0.3)", "filelock (>=3.4.0)", "importlib-metadata
 [[package]]
 name = "setuptools-rust"
-version = "1.8.1"
+version = "1.10.2"
 description = "Setuptools Rust extension plugin"
 optional = false
 python-versions = ">=3.8"
 files = [
-    {file = "setuptools-rust-1.8.1.tar.gz", hash = "sha256:94b1dd5d5308b3138d5b933c3a2b55e6d6927d1a22632e509fcea9ddd0f7e486"},
+    {file = "setuptools_rust-1.10.2-py3-none-any.whl", hash = "sha256:4b39c435ae9670315d522ed08fa0e8cb29f2a6048033966b6be2571a90ce4f1c"},
-    {file = "setuptools_rust-1.8.1-py3-none-any.whl", hash = "sha256:b5324493949ccd6aa0c03890c5f6b5f02de4512e3ac1697d02e9a6c02b18aa8e"},
+    {file = "setuptools_rust-1.10.2.tar.gz", hash = "sha256:5d73e7eee5f87a6417285b617c97088a7c20d1a70fcea60e3bdc94ff567c29dc"},
 ]
 [package.dependencies]
 semantic-version = ">=2.8.2,<3"
 setuptools = ">=62.4"
 tomli = {version = ">=1.2.1", markers = "python_version < \"3.11\""}
 [[package]]
 name = "signedjson"
@ -2515,33 +2514,33 @@ twisted = ["twisted"]
 [[package]]
 name = "tomli"
-version = "2.0.2"
+version = "2.1.0"
 description = "A lil' TOML parser"
 optional = false
 python-versions = ">=3.8"
 files = [
-    {file = "tomli-2.0.2-py3-none-any.whl", hash = "sha256:2ebe24485c53d303f690b0ec092806a085f07af5a5aa1464f3931eec36caaa38"},
+    {file = "tomli-2.1.0-py3-none-any.whl", hash = "sha256:a5c57c3d1c56f5ccdf89f6523458f60ef716e210fc47c4cfb188c5ba473e0391"},
-    {file = "tomli-2.0.2.tar.gz", hash = "sha256:d46d457a85337051c36524bc5349dd91b1877838e2979ac5ced3e710ed8a60ed"},
+    {file = "tomli-2.1.0.tar.gz", hash = "sha256:3f646cae2aec94e17d04973e4249548320197cfabdf130015d023de4b74d8ab8"},
 ]
 [[package]]
 name = "tornado"
-version = "6.4.1"
+version = "6.4.2"
 description = "Tornado is a Python web framework and asynchronous networking library, originally developed at FriendFeed."
 optional = true
 python-versions = ">=3.8"
 files = [
-    {file = "tornado-6.4.1-cp38-abi3-macosx_10_9_universal2.whl", hash = "sha256:163b0aafc8e23d8cdc3c9dfb24c5368af84a81e3364745ccb4427669bf84aec8"},
+    {file = "tornado-6.4.2-cp38-abi3-macosx_10_9_universal2.whl", hash = "sha256:e828cce1123e9e44ae2a50a9de3055497ab1d0aeb440c5ac23064d9e44880da1"},
-    {file = "tornado-6.4.1-cp38-abi3-macosx_10_9_x86_64.whl", hash = "sha256:6d5ce3437e18a2b66fbadb183c1d3364fb03f2be71299e7d10dbeeb69f4b2a14"},
+    {file = "tornado-6.4.2-cp38-abi3-macosx_10_9_x86_64.whl", hash = "sha256:072ce12ada169c5b00b7d92a99ba089447ccc993ea2143c9ede887e0937aa803"},
-    {file = "tornado-6.4.1-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:e2e20b9113cd7293f164dc46fffb13535266e713cdb87bd2d15ddb336e96cfc4"},
+    {file = "tornado-6.4.2-cp38-abi3-manylinux_2_17_aarch64.manylinux2014_aarch64.whl", hash = "sha256:1a017d239bd1bb0919f72af256a970624241f070496635784d9bf0db640d3fec"},
-    {file = "tornado-6.4.1-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:8ae50a504a740365267b2a8d1a90c9fbc86b780a39170feca9bcc1787ff80842"},
+    {file = "tornado-6.4.2-cp38-abi3-manylinux_2_5_i686.manylinux1_i686.manylinux_2_17_i686.manylinux2014_i686.whl", hash = "sha256:c36e62ce8f63409301537222faffcef7dfc5284f27eec227389f2ad11b09d946"},
-    {file = "tornado-6.4.1-cp38-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:613bf4ddf5c7a95509218b149b555621497a6cc0d46ac341b30bd9ec19eac7f3"},
+    {file = "tornado-6.4.2-cp38-abi3-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl", hash = "sha256:bca9eb02196e789c9cb5c3c7c0f04fb447dc2adffd95265b2c7223a8a615ccbf"},
-    {file = "tornado-6.4.1-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:25486eb223babe3eed4b8aecbac33b37e3dd6d776bc730ca14e1bf93888b979f"},
+    {file = "tornado-6.4.2-cp38-abi3-musllinux_1_2_aarch64.whl", hash = "sha256:304463bd0772442ff4d0f5149c6f1c2135a1fae045adf070821c6cdc76980634"},
-    {file = "tornado-6.4.1-cp38-abi3-musllinux_1_2_i686.whl", hash = "sha256:454db8a7ecfcf2ff6042dde58404164d969b6f5d58b926da15e6b23817950fc4"},
+    {file = "tornado-6.4.2-cp38-abi3-musllinux_1_2_i686.whl", hash = "sha256:c82c46813ba483a385ab2a99caeaedf92585a1f90defb5693351fa7e4ea0bf73"},
-    {file = "tornado-6.4.1-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:a02a08cc7a9314b006f653ce40483b9b3c12cda222d6a46d4ac63bb6c9057698"},
+    {file = "tornado-6.4.2-cp38-abi3-musllinux_1_2_x86_64.whl", hash = "sha256:932d195ca9015956fa502c6b56af9eb06106140d844a335590c1ec7f5277d10c"},
-    {file = "tornado-6.4.1-cp38-abi3-win32.whl", hash = "sha256:d9a566c40b89757c9aa8e6f032bcdb8ca8795d7c1a9762910c722b1635c9de4d"},
+    {file = "tornado-6.4.2-cp38-abi3-win32.whl", hash = "sha256:2876cef82e6c5978fde1e0d5b1f919d756968d5b4282418f3146b79b58556482"},
-    {file = "tornado-6.4.1-cp38-abi3-win_amd64.whl", hash = "sha256:b24b8982ed444378d7f21d563f4180a2de31ced9d8d84443907a0a64da2072e7"},
+    {file = "tornado-6.4.2-cp38-abi3-win_amd64.whl", hash = "sha256:908b71bf3ff37d81073356a5fadcc660eb10c1476ee6e2725588626ce7e5ca38"},
-    {file = "tornado-6.4.1.tar.gz", hash = "sha256:92d3ab53183d8c50f8204a51e6f91d18a15d5ef261e84d452800d4ff6fc504e9"},
+    {file = "tornado-6.4.2.tar.gz", hash = "sha256:92bad5b4746e9879fd7bf1eb21dce4e3fc5128d71601f80005afa39237ad620b"},
 ]
 [[package]]
--- a/pyproject.toml
+++ b/pyproject.toml
@ -97,7 +97,7 @@ module-name = "synapse.synapse_rust"
 [tool.poetry]
 name = "matrix-synapse"
-version = "1.119.0"
+version = "1.120.0"
 description = "Homeserver for the Matrix decentralised comms protocol"
 authors = ["Matrix.org Team and Contributors <packages@matrix.org>"]
 license = "AGPL-3.0-or-later"
@ -370,7 +370,7 @@ tomli = ">=1.2.3"
 # runtime errors caused by build system changes.
 # We are happy to raise these upper bounds upon request,
 # provided we check that it's safe to do so (i.e. that CI passes).
-requires = ["poetry-core>=1.1.0,<=1.9.1", "setuptools_rust>=1.3,<=1.8.1"]
+requires = ["poetry-core>=1.1.0,<=1.9.1", "setuptools_rust>=1.3,<=1.10.2"]
 build-backend = "poetry.core.masonry.api"
--- a/rust/Cargo.toml
+++ b/rust/Cargo.toml
@ -30,14 +30,14 @@ http = "1.1.0"
 lazy_static = "1.4.0"
 log = "0.4.17"
 mime = "0.3.17"
-pyo3 = { version = "0.21.0", features = [
+pyo3 = { version = "0.23.2", features = [
    "macros",
    "anyhow",
    "abi3",
    "abi3-py38",
 ] }
-pyo3-log = "0.10.0"
+pyo3-log = "0.12.0"
-pythonize = "0.21.0"
+pythonize = "0.23.0"
 regex = "1.6.0"
 sha2 = "0.10.8"
 serde = { version = "1.0.144", features = ["derive"] }
--- a/rust/src/acl/mod.rs
+++ b/rust/src/acl/mod.rs
@ -32,14 +32,14 @@ use crate::push::utils::{glob_to_regex, GlobMatchType};
 /// Called when registering modules with python.
 pub fn register_module(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> {
-    let child_module = PyModule::new_bound(py, "acl")?;
+    let child_module = PyModule::new(py, "acl")?;
    child_module.add_class::<ServerAclEvaluator>()?;
    m.add_submodule(&child_module)?;
    // We need to manually add the module to sys.modules to make `from
    // synapse.synapse_rust import acl` work.
-    py.import_bound("sys")?
+    py.import("sys")?
        .getattr("modules")?
        .set_item("synapse.synapse_rust.acl", child_module)?;
--- a/rust/src/events/internal_metadata.rs
+++ b/rust/src/events/internal_metadata.rs
@ -41,9 +41,11 @@ use pyo3::{
    pybacked::PyBackedStr,
    pyclass, pymethods,
    types::{PyAnyMethods, PyDict, PyDictMethods, PyString},
-    Bound, IntoPy, PyAny, PyObject, PyResult, Python,
+    Bound, IntoPyObject, PyAny, PyObject, PyResult, Python,
 };
 use crate::UnwrapInfallible;
 /// Definitions of the various fields of the internal metadata.
 #[derive(Clone)]
 enum EventInternalMetadataData {
@ -60,31 +62,59 @@ enum EventInternalMetadataData {
 impl EventInternalMetadataData {
    /// Convert the field to its name and python object.
-    fn to_python_pair<'a>(&self, py: Python<'a>) -> (&'a Bound<'a, PyString>, PyObject) {
+    fn to_python_pair<'a>(&self, py: Python<'a>) -> (&'a Bound<'a, PyString>, Bound<'a, PyAny>) {
        match self {
-            EventInternalMetadataData::OutOfBandMembership(o) => {
+            EventInternalMetadataData::OutOfBandMembership(o) => (
-                (pyo3::intern!(py, "out_of_band_membership"), o.into_py(py))
+                pyo3::intern!(py, "out_of_band_membership"),
-            }
+                o.into_pyobject(py)
-            EventInternalMetadataData::SendOnBehalfOf(o) => {
+                    .unwrap_infallible()
-                (pyo3::intern!(py, "send_on_behalf_of"), o.into_py(py))
+                    .to_owned()
-            }
+                    .into_any(),
-            EventInternalMetadataData::RecheckRedaction(o) => {
+            ),
-                (pyo3::intern!(py, "recheck_redaction"), o.into_py(py))
+            EventInternalMetadataData::SendOnBehalfOf(o) => (
-            }
+                pyo3::intern!(py, "send_on_behalf_of"),
-            EventInternalMetadataData::SoftFailed(o) => {
+                o.into_pyobject(py).unwrap_infallible().into_any(),
-                (pyo3::intern!(py, "soft_failed"), o.into_py(py))
+            ),
-            }
+            EventInternalMetadataData::RecheckRedaction(o) => (
-            EventInternalMetadataData::ProactivelySend(o) => {
+                pyo3::intern!(py, "recheck_redaction"),
-                (pyo3::intern!(py, "proactively_send"), o.into_py(py))
+                o.into_pyobject(py)
-            }
+                    .unwrap_infallible()
-            EventInternalMetadataData::Redacted(o) => {
+                    .to_owned()
-                (pyo3::intern!(py, "redacted"), o.into_py(py))
+                    .into_any(),
-            }
+            ),
-            EventInternalMetadataData::TxnId(o) => (pyo3::intern!(py, "txn_id"), o.into_py(py)),
+            EventInternalMetadataData::SoftFailed(o) => (
-            EventInternalMetadataData::TokenId(o) => (pyo3::intern!(py, "token_id"), o.into_py(py)),
+                pyo3::intern!(py, "soft_failed"),
-            EventInternalMetadataData::DeviceId(o) => {
+                o.into_pyobject(py)
-                (pyo3::intern!(py, "device_id"), o.into_py(py))
+                    .unwrap_infallible()
-            }
+                    .to_owned()
                    .into_any(),
            ),
            EventInternalMetadataData::ProactivelySend(o) => (
                pyo3::intern!(py, "proactively_send"),
                o.into_pyobject(py)
                    .unwrap_infallible()
                    .to_owned()
                    .into_any(),
            ),
            EventInternalMetadataData::Redacted(o) => (
                pyo3::intern!(py, "redacted"),
                o.into_pyobject(py)
                    .unwrap_infallible()
                    .to_owned()
                    .into_any(),
            ),
            EventInternalMetadataData::TxnId(o) => (
                pyo3::intern!(py, "txn_id"),
                o.into_pyobject(py).unwrap_infallible().into_any(),
            ),
            EventInternalMetadataData::TokenId(o) => (
                pyo3::intern!(py, "token_id"),
                o.into_pyobject(py).unwrap_infallible().into_any(),
            ),
            EventInternalMetadataData::DeviceId(o) => (
                pyo3::intern!(py, "device_id"),
                o.into_pyobject(py).unwrap_infallible().into_any(),
            ),
        }
    }
@ -247,7 +277,7 @@ impl EventInternalMetadata {
    ///
    /// Note that `outlier` and `stream_ordering` are stored in separate columns so are not returned here.
    fn get_dict(&self, py: Python<'_>) -> PyResult<PyObject> {
-        let dict = PyDict::new_bound(py);
+        let dict = PyDict::new(py);
        for entry in &self.data {
            let (key, value) = entry.to_python_pair(py);
--- a/rust/src/events/mod.rs
+++ b/rust/src/events/mod.rs
@ -30,7 +30,7 @@ mod internal_metadata;
 /// Called when registering modules with python.
 pub fn register_module(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> {
-    let child_module = PyModule::new_bound(py, "events")?;
+    let child_module = PyModule::new(py, "events")?;
    child_module.add_class::<internal_metadata::EventInternalMetadata>()?;
    child_module.add_function(wrap_pyfunction!(filter::event_visible_to_server_py, m)?)?;
@ -38,7 +38,7 @@ pub fn register_module(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()>
    // We need to manually add the module to sys.modules to make `from
    // synapse.synapse_rust import events` work.
-    py.import_bound("sys")?
+    py.import("sys")?
        .getattr("modules")?
        .set_item("synapse.synapse_rust.events", child_module)?;
--- a/rust/src/http.rs
+++ b/rust/src/http.rs
@ -70,7 +70,7 @@ pub fn http_request_from_twisted(request: &Bound<'_, PyAny>) -> PyResult<Request
    let headers_iter = request
        .getattr("requestHeaders")?
        .call_method0("getAllRawHeaders")?
-        .iter()?;
+        .try_iter()?;
    for header in headers_iter {
        let header = header?;
--- a/rust/src/lib.rs
+++ b/rust/src/lib.rs
@ -1,3 +1,5 @@
 use std::convert::Infallible;
 use lazy_static::lazy_static;
 use pyo3::prelude::*;
 use pyo3_log::ResetHandle;
@ -52,3 +54,16 @@ fn synapse_rust(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> {
    Ok(())
 }
 pub trait UnwrapInfallible<T> {
    fn unwrap_infallible(self) -> T;
 }
 impl<T> UnwrapInfallible<T> for Result<T, Infallible> {
    fn unwrap_infallible(self) -> T {
        match self {
            Ok(val) => val,
            Err(never) => match never {},
        }
    }
 }
--- a/rust/src/push/evaluator.rs
+++ b/rust/src/push/evaluator.rs
@ -167,6 +167,7 @@ impl PushRuleEvaluator {
    ///
    /// Returns the set of actions, if any, that match (filtering out any
    /// `dont_notify` and `coalesce` actions).
    #[pyo3(signature = (push_rules, user_id=None, display_name=None))]
    pub fn run(
        &self,
        push_rules: &FilteredPushRules,
@ -236,6 +237,7 @@ impl PushRuleEvaluator {
    }
    /// Check if the given condition matches.
    #[pyo3(signature = (condition, user_id=None, display_name=None))]
    fn matches(
        &self,
        condition: Condition,
--- a/rust/src/push/mod.rs
+++ b/rust/src/push/mod.rs
@ -65,8 +65,8 @@ use anyhow::{Context, Error};
 use log::warn;
 use pyo3::exceptions::PyTypeError;
 use pyo3::prelude::*;
-use pyo3::types::{PyBool, PyList, PyLong, PyString};
+use pyo3::types::{PyBool, PyInt, PyList, PyString};
-use pythonize::{depythonize_bound, pythonize};
+use pythonize::{depythonize, pythonize, PythonizeError};
 use serde::de::Error as _;
 use serde::{Deserialize, Serialize};
 use serde_json::Value;
@ -79,7 +79,7 @@ pub mod utils;
 /// Called when registering modules with python.
 pub fn register_module(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> {
-    let child_module = PyModule::new_bound(py, "push")?;
+    let child_module = PyModule::new(py, "push")?;
    child_module.add_class::<PushRule>()?;
    child_module.add_class::<PushRules>()?;
    child_module.add_class::<FilteredPushRules>()?;
@ -90,7 +90,7 @@ pub fn register_module(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()>
    // We need to manually add the module to sys.modules to make `from
    // synapse.synapse_rust import push` work.
-    py.import_bound("sys")?
+    py.import("sys")?
        .getattr("modules")?
        .set_item("synapse.synapse_rust.push", child_module)?;
@ -182,12 +182,16 @@ pub enum Action {
    Unknown(Value),
 }
-impl IntoPy<PyObject> for Action {
+impl<'py> IntoPyObject<'py> for Action {
-    fn into_py(self, py: Python<'_>) -> PyObject {
+    type Target = PyAny;
    type Output = Bound<'py, Self::Target>;
    type Error = PythonizeError;
    fn into_pyobject(self, py: Python<'py>) -> Result<Self::Output, Self::Error> {
        // When we pass the `Action` struct to Python we want it to be converted
        // to a dict. We use `pythonize`, which converts the struct using the
        // `serde` serialization.
-        pythonize(py, &self).expect("valid action")
+        pythonize(py, &self)
    }
 }
@ -270,13 +274,13 @@ pub enum SimpleJsonValue {
 }
 impl<'source> FromPyObject<'source> for SimpleJsonValue {
-    fn extract(ob: &'source PyAny) -> PyResult<Self> {
+    fn extract_bound(ob: &Bound<'source, PyAny>) -> PyResult<Self> {
        if let Ok(s) = ob.downcast::<PyString>() {
            Ok(SimpleJsonValue::Str(Cow::Owned(s.to_string())))
        // A bool *is* an int, ensure we try bool first.
        } else if let Ok(b) = ob.downcast::<PyBool>() {
            Ok(SimpleJsonValue::Bool(b.extract()?))
-        } else if let Ok(i) = ob.downcast::<PyLong>() {
+        } else if let Ok(i) = ob.downcast::<PyInt>() {
            Ok(SimpleJsonValue::Int(i.extract()?))
        } else if ob.is_none() {
            Ok(SimpleJsonValue::Null)
@ -298,15 +302,19 @@ pub enum JsonValue {
 }
 impl<'source> FromPyObject<'source> for JsonValue {
-    fn extract(ob: &'source PyAny) -> PyResult<Self> {
+    fn extract_bound(ob: &Bound<'source, PyAny>) -> PyResult<Self> {
        if let Ok(l) = ob.downcast::<PyList>() {
-            match l.iter().map(SimpleJsonValue::extract).collect() {
+            match l
                .iter()
                .map(|it| SimpleJsonValue::extract_bound(&it))
                .collect()
            {
                Ok(a) => Ok(JsonValue::Array(a)),
                Err(e) => Err(PyTypeError::new_err(format!(
                    "Can't convert to JsonValue::Array: {e}"
                ))),
            }
-        } else if let Ok(v) = SimpleJsonValue::extract(ob) {
+        } else if let Ok(v) = SimpleJsonValue::extract_bound(ob) {
            Ok(JsonValue::Value(v))
        } else {
            Err(PyTypeError::new_err(format!(
@ -363,15 +371,19 @@ pub enum KnownCondition {
    },
 }
-impl IntoPy<PyObject> for Condition {
+impl<'source> IntoPyObject<'source> for Condition {
-    fn into_py(self, py: Python<'_>) -> PyObject {
+    type Target = PyAny;
-        pythonize(py, &self).expect("valid condition")
+    type Output = Bound<'source, Self::Target>;
    type Error = PythonizeError;
    fn into_pyobject(self, py: Python<'source>) -> Result<Self::Output, Self::Error> {
        pythonize(py, &self)
    }
 }
 impl<'source> FromPyObject<'source> for Condition {
    fn extract_bound(ob: &Bound<'source, PyAny>) -> PyResult<Self> {
-        Ok(depythonize_bound(ob.clone())?)
+        Ok(depythonize(ob)?)
    }
 }
--- a/rust/src/rendezvous/mod.rs
+++ b/rust/src/rendezvous/mod.rs
@ -29,7 +29,7 @@ use pyo3::{
    exceptions::PyValueError,
    pyclass, pymethods,
    types::{PyAnyMethods, PyModule, PyModuleMethods},
-    Bound, Py, PyAny, PyObject, PyResult, Python, ToPyObject,
+    Bound, IntoPyObject, Py, PyAny, PyObject, PyResult, Python,
 };
 use ulid::Ulid;
@ -37,6 +37,7 @@ use self::session::Session;
 use crate::{
    errors::{NotFoundError, SynapseError},
    http::{http_request_from_twisted, http_response_to_twisted, HeaderMapPyExt},
    UnwrapInfallible,
 };
 mod session;
@ -125,7 +126,11 @@ impl RendezvousHandler {
        let base = Uri::try_from(format!("{base}_synapse/client/rendezvous"))
            .map_err(|_| PyValueError::new_err("Invalid base URI"))?;
-        let clock = homeserver.call_method0("get_clock")?.to_object(py);
+        let clock = homeserver
            .call_method0("get_clock")?
            .into_pyobject(py)
            .unwrap_infallible()
            .unbind();
        // Construct a Python object so that we can get a reference to the
        // evict method and schedule it to run.
@ -288,6 +293,13 @@ impl RendezvousHandler {
        let mut response = Response::new(Bytes::new());
        *response.status_mut() = StatusCode::ACCEPTED;
        prepare_headers(response.headers_mut(), session);
        // Even though this isn't mandated by the MSC, we set a Content-Type on the response. It
        // doesn't do any harm as the body is empty, but this helps escape a bug in some reverse
        // proxy/cache setup which strips the ETag header if there is no Content-Type set.
        // Specifically, we noticed this behaviour when placing Synapse behind Cloudflare.
        response.headers_mut().typed_insert(ContentType::text());
        http_response_to_twisted(twisted_request, response)?;
        Ok(())
@ -311,7 +323,7 @@ impl RendezvousHandler {
 }
 pub fn register_module(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()> {
-    let child_module = PyModule::new_bound(py, "rendezvous")?;
+    let child_module = PyModule::new(py, "rendezvous")?;
    child_module.add_class::<RendezvousHandler>()?;
@ -319,7 +331,7 @@ pub fn register_module(py: Python<'_>, m: &Bound<'_, PyModule>) -> PyResult<()>
    // We need to manually add the module to sys.modules to make `from
    // synapse.synapse_rust import rendezvous` work.
-    py.import_bound("sys")?
+    py.import("sys")?
        .getattr("modules")?
        .set_item("synapse.synapse_rust.rendezvous", child_module)?;
--- a/synapse/_scripts/synapse_port_db.py
+++ b/synapse/_scripts/synapse_port_db.py
@ -88,6 +88,7 @@ from synapse.storage.databases.main.relations import RelationsWorkerStore
 from synapse.storage.databases.main.room import RoomBackgroundUpdateStore
 from synapse.storage.databases.main.roommember import RoomMemberBackgroundUpdateStore
 from synapse.storage.databases.main.search import SearchBackgroundUpdateStore
 from synapse.storage.databases.main.sliding_sync import SlidingSyncStore
 from synapse.storage.databases.main.state import MainStateBackgroundUpdateStore
 from synapse.storage.databases.main.stats import StatsStore
 from synapse.storage.databases.main.user_directory import (
@ -255,6 +256,7 @@ class Store(
    ReceiptsBackgroundUpdateStore,
    RelationsWorkerStore,
    EventFederationWorkerStore,
    SlidingSyncStore,
 ):
    def execute(self, f: Callable[..., R], *args: Any, **kwargs: Any) -> Awaitable[R]:
        return self.db_pool.runInteraction(f.__name__, f, *args, **kwargs)
--- a/synapse/config/repository.py
+++ b/synapse/config/repository.py
@ -272,9 +272,7 @@ class ContentRepositoryConfig(Config):
                remote_media_lifetime
            )
-        self.enable_authenticated_media = config.get(
+        self.enable_authenticated_media = config.get("enable_authenticated_media", True)
            "enable_authenticated_media", False
        )
    def generate_config_section(self, data_dir_path: str, **kwargs: Any) -> str:
        assert data_dir_path is not None
--- a/synapse/federation/sender/init.py
+++ b/synapse/federation/sender/init.py
@ -140,7 +140,6 @@ from typing import (
    Iterable,
    List,
    Optional,
    Set,
    Tuple,
 )
@ -170,7 +169,13 @@ from synapse.metrics.background_process_metrics import (
    run_as_background_process,
    wrap_as_background_process,
 )
-from synapse.types import JsonDict, ReadReceipt, RoomStreamToken, StrCollection
+from synapse.types import (
    JsonDict,
    ReadReceipt,
    RoomStreamToken,
    StrCollection,
    get_domain_from_id,
 )
 from synapse.util import Clock
 from synapse.util.metrics import Measure
 from synapse.util.retryutils import filter_destinations_by_retry_limiter
@ -297,12 +302,10 @@ class _DestinationWakeupQueue:
    # being woken up.
    _MAX_TIME_IN_QUEUE = 30.0
    # The maximum duration in seconds between waking up consecutive destination
    # queues.
    _MAX_DELAY = 0.1
    sender: "FederationSender" = attr.ib()
    clock: Clock = attr.ib()
    max_delay_s: int = attr.ib()
    queue: "OrderedDict[str, Literal[None]]" = attr.ib(factory=OrderedDict)
    processing: bool = attr.ib(default=False)
@ -332,7 +335,7 @@ class _DestinationWakeupQueue:
            # We also add an upper bound to the delay, to gracefully handle the
            # case where the queue only has a few entries in it.
            current_sleep_seconds = min(
-                self._MAX_DELAY, self._MAX_TIME_IN_QUEUE / len(self.queue)
+                self.max_delay_s, self._MAX_TIME_IN_QUEUE / len(self.queue)
            )
            while self.queue:
@ -416,19 +419,14 @@ class FederationSender(AbstractFederationSender):
        self._is_processing = False
        self._last_poked_id = -1
        # map from room_id to a set of PerDestinationQueues which we believe are
        # awaiting a call to flush_read_receipts_for_room. The presence of an entry
        # here for a given room means that we are rate-limiting RR flushes to that room,
        # and that there is a pending call to _flush_rrs_for_room in the system.
        self._queues_awaiting_rr_flush_by_room: Dict[str, Set[PerDestinationQueue]] = {}
        self._rr_txn_interval_per_room_ms = (
            1000.0
            / hs.config.ratelimiting.federation_rr_transactions_per_room_per_second
        )
        self._external_cache = hs.get_external_cache()
-        self._destination_wakeup_queue = _DestinationWakeupQueue(self, self.clock)
+
        rr_txn_interval_per_room_s = (
            1.0 / hs.config.ratelimiting.federation_rr_transactions_per_room_per_second
        )
        self._destination_wakeup_queue = _DestinationWakeupQueue(
            self, self.clock, max_delay_s=rr_txn_interval_per_room_s
        )
        # Regularly wake up destinations that have outstanding PDUs to be caught up
        self.clock.looping_call_now(
@ -745,37 +743,48 @@ class FederationSender(AbstractFederationSender):
        # Some background on the rate-limiting going on here.
        #
-        # It turns out that if we attempt to send out RRs as soon as we get them from
+        # It turns out that if we attempt to send out RRs as soon as we get them
-        # a client, then we end up trying to do several hundred Hz of federation
+        # from a client, then we end up trying to do several hundred Hz of
-        # transactions. (The number of transactions scales as O(N^2) on the size of a
+        # federation transactions. (The number of transactions scales as O(N^2)
-        # room, since in a large room we have both more RRs coming in, and more servers
+        # on the size of a room, since in a large room we have both more RRs
-        # to send them to.)
+        # coming in, and more servers to send them to.)
        #
-        # This leads to a lot of CPU load, and we end up getting behind. The solution
+        # This leads to a lot of CPU load, and we end up getting behind. The
-        # currently adopted is as follows:
+        # solution currently adopted is to differentiate between receipts and
        # destinations we should immediately send to, and those we can trickle
        # the receipts to.
        #
-        # The first receipt in a given room is sent out immediately, at time T0. Any
+        # The current logic is to send receipts out immediately if:
-        # further receipts are, in theory, batched up for N seconds, where N is calculated
+        #   - the room is "small", i.e. there's only N servers to send receipts
-        # based on the number of servers in the room to achieve a transaction frequency
+        #     to, and so sending out the receipts immediately doesn't cause too
-        # of around 50Hz. So, for example, if there were 100 servers in the room, then
+        #     much load; or
-        # N would be 100 / 50Hz = 2 seconds.
+        #   - the receipt is for an event that happened recently, as users
        #     notice if receipts are delayed when they know other users are
        #     currently reading the room; or
        #   - the receipt is being sent to the server that sent the event, so
        #     that users see receipts for their own receipts quickly.
        #
-        # Then, after T+N, we flush out any receipts that have accumulated, and restart
+        # For destinations that we should delay sending the receipt to, we queue
-        # the timer to flush out more receipts at T+2N, etc. If no receipts accumulate,
+        # the receipts up to be sent in the next transaction, but don't trigger
-        # we stop the cycle and go back to the start.
+        # a new transaction to be sent. We then add the destination to the
        # `DestinationWakeupQueue`, which will slowly iterate over each
        # destination and trigger a new transaction to be sent.
        #
-        # However, in practice, it is often possible to flush out receipts earlier: in
+        # However, in practice, it is often possible to send out delayed
-        # particular, if we are sending a transaction to a given server anyway (for
+        # receipts earlier: in particular, if we are sending a transaction to a
-        # example, because we have a PDU or a RR in another room to send), then we may
+        # given server anyway (for example, because we have a PDU or a RR in
-        # as well send out all of the pending RRs for that server. So it may be that
+        # another room to send), then we may as well send out all of the pending
-        # by the time we get to T+N, we don't actually have any RRs left to send out.
+        # RRs for that server. So it may be that by the time we get to waking up
-        # Nevertheless we continue to buffer up RRs for the room in question until we
+        # the destination, we don't actually have any RRs left to send out.
        # reach the point that no RRs arrive between timer ticks.
        #
-        # For even more background, see https://github.com/matrix-org/synapse/issues/4730.
+        # For even more background, see
        # https://github.com/matrix-org/synapse/issues/4730.
        room_id = receipt.room_id
        # Local read receipts always have 1 event ID.
        event_id = receipt.event_ids[0]
        # Work out which remote servers should be poked and poke them.
        domains_set = await self._storage_controllers.state.get_current_hosts_in_room_or_partial_state_approximation(
            room_id
@ -797,49 +806,51 @@ class FederationSender(AbstractFederationSender):
        if not domains:
            return
-        queues_pending_flush = self._queues_awaiting_rr_flush_by_room.get(room_id)
+        # We now split which domains we want to wake up immediately vs which we
        # want to delay waking up.
        immediate_domains: StrCollection
        delay_domains: StrCollection
-        # if there is no flush yet scheduled, we will send out these receipts with
+        if len(domains) < 10:
-        # immediate flushes, and schedule the next flush for this room.
+            # For "small" rooms send to all domains immediately
-        if queues_pending_flush is not None:
+            immediate_domains = domains
-            logger.debug("Queuing receipt for: %r", domains)
+            delay_domains = ()
        else:
-            logger.debug("Sending receipt to: %r", domains)
+            metadata = await self.store.get_metadata_for_event(
-            self._schedule_rr_flush_for_room(room_id, len(domains))
+                receipt.room_id, event_id
            )
            assert metadata is not None
-        for domain in domains:
+            sender_domain = get_domain_from_id(metadata.sender)
            if self.clock.time_msec() - metadata.received_ts < 60_000:
                # We always send receipts for recent messages immediately
                immediate_domains = domains
                delay_domains = ()
            else:
                # Otherwise, we delay waking up all destinations except for the
                # sender's domain.
                immediate_domains = []
                delay_domains = []
                for domain in domains:
                    if domain == sender_domain:
                        immediate_domains.append(domain)
                    else:
                        delay_domains.append(domain)
        for domain in immediate_domains:
            # Add to destination queue and wake the destination up
            queue = self._get_per_destination_queue(domain)
            queue.queue_read_receipt(receipt)
            queue.attempt_new_transaction()
        for domain in delay_domains:
            # Add to destination queue...
            queue = self._get_per_destination_queue(domain)
            queue.queue_read_receipt(receipt)
-            # if there is already a RR flush pending for this room, then make sure this
+            # ... and schedule the destination to be woken up.
-            # destination is registered for the flush
+            self._destination_wakeup_queue.add_to_queue(domain)
            if queues_pending_flush is not None:
                queues_pending_flush.add(queue)
            else:
                queue.flush_read_receipts_for_room(room_id)
    def _schedule_rr_flush_for_room(self, room_id: str, n_domains: int) -> None:
        # that is going to cause approximately len(domains) transactions, so now back
        # off for that multiplied by RR_TXN_INTERVAL_PER_ROOM
        backoff_ms = self._rr_txn_interval_per_room_ms * n_domains
        logger.debug("Scheduling RR flush in %s in %d ms", room_id, backoff_ms)
        self.clock.call_later(backoff_ms, self._flush_rrs_for_room, room_id)
        self._queues_awaiting_rr_flush_by_room[room_id] = set()
    def _flush_rrs_for_room(self, room_id: str) -> None:
        queues = self._queues_awaiting_rr_flush_by_room.pop(room_id)
        logger.debug("Flushing RRs in %s to %s", room_id, queues)
        if not queues:
            # no more RRs arrived for this room; we are done.
            return
        # schedule the next flush
        self._schedule_rr_flush_for_room(room_id, len(queues))
        for queue in queues:
            queue.flush_read_receipts_for_room(room_id)
    async def send_presence_to_destinations(
        self, states: Iterable[UserPresenceState], destinations: Iterable[str]
--- a/synapse/federation/sender/per_destination_queue.py
+++ b/synapse/federation/sender/per_destination_queue.py
@ -156,7 +156,6 @@ class PerDestinationQueue:
        # Each receipt can only have a single receipt per
        # (room ID, receipt type, user ID, thread ID) tuple.
        self._pending_receipt_edus: List[Dict[str, Dict[str, Dict[str, dict]]]] = []
        self._rrs_pending_flush = False
        # stream_id of last successfully sent to-device message.
        # NB: may be a long or an int.
@ -258,15 +257,7 @@ class PerDestinationQueue:
                }
            )
-    def flush_read_receipts_for_room(self, room_id: str) -> None:
+        self.mark_new_data()
        # If there are any pending receipts for this room then force-flush them
        # in a new transaction.
        for edu in self._pending_receipt_edus:
            if room_id in edu:
                self._rrs_pending_flush = True
                self.attempt_new_transaction()
                # No use in checking remaining EDUs if the room was found.
                break
    def send_keyed_edu(self, edu: Edu, key: Hashable) -> None:
        self._pending_edus_keyed[(edu.edu_type, key)] = edu
@ -603,12 +594,9 @@ class PerDestinationQueue:
                    self._destination, last_successful_stream_ordering
                )
-    def _get_receipt_edus(self, force_flush: bool, limit: int) -> Iterable[Edu]:
+    def _get_receipt_edus(self, limit: int) -> Iterable[Edu]:
        if not self._pending_receipt_edus:
            return
        if not force_flush and not self._rrs_pending_flush:
            # not yet time for this lot
            return
        # Send at most limit EDUs for receipts.
        for content in self._pending_receipt_edus[:limit]:
@ -747,7 +735,7 @@ class _TransactionQueueManager:
            )
        # Add read receipt EDUs.
-        pending_edus.extend(self.queue._get_receipt_edus(force_flush=False, limit=5))
+        pending_edus.extend(self.queue._get_receipt_edus(limit=5))
        edu_limit = MAX_EDUS_PER_TRANSACTION - len(pending_edus)
        # Next, prioritize to-device messages so that existing encryption channels
@ -795,13 +783,6 @@ class _TransactionQueueManager:
        if not self._pdus and not pending_edus:
            return [], []
        # if we've decided to send a transaction anyway, and we have room, we
        # may as well send any pending RRs
        if edu_limit:
            pending_edus.extend(
                self.queue._get_receipt_edus(force_flush=True, limit=edu_limit)
            )
        if self._pdus:
            self._last_stream_ordering = self._pdus[
                -1
--- a/synapse/handlers/admin.py
+++ b/synapse/handlers/admin.py
@ -124,6 +124,7 @@ class AdminHandler:
            "consent_ts": user_info.consent_ts,
            "user_type": user_info.user_type,
            "is_guest": user_info.is_guest,
            "suspended": user_info.suspended,
        }
        if self._msc3866_enabled:
--- a/synapse/handlers/e2e_keys.py
+++ b/synapse/handlers/e2e_keys.py
@ -39,6 +39,8 @@ from synapse.replication.http.devices import ReplicationUploadKeysForUserRestSer
 from synapse.types import (
    JsonDict,
    JsonMapping,
    ScheduledTask,
    TaskStatus,
    UserID,
    get_domain_from_id,
    get_verify_key_from_cross_signing_key,
@ -70,6 +72,7 @@ class E2eKeysHandler:
        self.is_mine = hs.is_mine
        self.clock = hs.get_clock()
        self._worker_lock_handler = hs.get_worker_locks_handler()
        self._task_scheduler = hs.get_task_scheduler()
        federation_registry = hs.get_federation_registry()
@ -116,6 +119,10 @@ class E2eKeysHandler:
            hs.config.experimental.msc3984_appservice_key_query
        )
        self._task_scheduler.register_action(
            self._delete_old_one_time_keys_task, "delete_old_otks"
        )
    @trace
    @cancellable
    async def query_devices(
@ -1574,6 +1581,45 @@ class E2eKeysHandler:
                    return True
        return False
    async def _delete_old_one_time_keys_task(
        self, task: ScheduledTask
    ) -> Tuple[TaskStatus, Optional[JsonMapping], Optional[str]]:
        """Scheduler task to delete old one time keys.
        Until Synapse 1.119, Synapse used to issue one-time-keys in a random order, leading to the possibility
        that it could still have old OTKs that the client has dropped. This task is scheduled exactly once
        by a database schema delta file, and it clears out old one-time-keys that look like they came from libolm.
        """
        last_user = task.result.get("from_user", "") if task.result else ""
        while True:
            # We process users in batches of 100
            users, rowcount = await self.store.delete_old_otks_for_next_user_batch(
                last_user, 100
            )
            if len(users) == 0:
                # We're done!
                return TaskStatus.COMPLETE, None, None
            logger.debug(
                "Deleted %i old one-time-keys for users '%s'..'%s'",
                rowcount,
                users[0],
                users[-1],
            )
            last_user = users[-1]
            # Store our progress
            await self._task_scheduler.update_task(
                task.id, result={"from_user": last_user}
            )
            # Sleep a little before doing the next user.
            #
            # matrix.org has about 15M users in the e2e_one_time_keys_json table
            # (comprising 20M devices). We want this to take about a week, so we need
            # to do about one batch of 100 users every 4 seconds.
            await self.clock.sleep(4)
 def _check_cross_signing_key(
    key: JsonDict, user_id: str, key_type: str, signing_key: Optional[VerifyKey] = None
--- a/synapse/http/client.py
+++ b/synapse/http/client.py
@ -36,7 +36,6 @@ from typing import (
 )
 import attr
 import multipart
 import treq
 from canonicaljson import encode_canonical_json
 from netaddr import AddrFormatError, IPAddress, IPSet
@ -93,6 +92,20 @@ from synapse.util.async_helpers import timeout_deferred
 if TYPE_CHECKING:
    from synapse.server import HomeServer
 # Support both import names for the `python-multipart` (PyPI) library,
 # which renamed its package name from `multipart` to `python_multipart`
 # in 0.0.13 (though supports the old import name for compatibility).
 # Note that the `multipart` package name conflicts with `multipart` (PyPI)
 # so we should prefer importing from `python_multipart` when possible.
 try:
    from python_multipart import MultipartParser
    if TYPE_CHECKING:
        from python_multipart import multipart
 except ImportError:
    from multipart import MultipartParser  # type: ignore[no-redef]
 logger = logging.getLogger(__name__)
 outgoing_requests_counter = Counter("synapse_http_client_requests", "", ["method"])
@ -1039,7 +1052,7 @@ class _MultipartParserProtocol(protocol.Protocol):
        self.deferred = deferred
        self.boundary = boundary
        self.max_length = max_length
-        self.parser: Optional[multipart.MultipartParser] = None
+        self.parser: Optional[MultipartParser] = None
        self.multipart_response = MultipartResponse()
        self.has_redirect = False
        self.in_json = False
@ -1097,12 +1110,12 @@ class _MultipartParserProtocol(protocol.Protocol):
                        self.deferred.errback()
                    self.file_length += end - start
-            callbacks: "multipart.multipart.MultipartCallbacks" = {
+            callbacks: "multipart.MultipartCallbacks" = {
                "on_header_field": on_header_field,
                "on_header_value": on_header_value,
                "on_part_data": on_part_data,
            }
-            self.parser = multipart.MultipartParser(self.boundary, callbacks)
+            self.parser = MultipartParser(self.boundary, callbacks)
        self.total_length += len(incoming_data)
        if self.max_length is not None and self.total_length >= self.max_length:
--- a/synapse/server.py
+++ b/synapse/server.py
@ -254,6 +254,7 @@ class HomeServer(metaclass=abc.ABCMeta):
        "auth",
        "deactivate_account",
        "delayed_events",
        "e2e_keys",  # for the `delete_old_otks` scheduled-task handler
        "message",
        "pagination",
        "profile",
--- a/synapse/storage/databases/main/cache.py
+++ b/synapse/storage/databases/main/cache.py
@ -333,6 +333,7 @@ class CacheInvalidationWorkerStore(SQLBaseStore):
        self._attempt_to_invalidate_cache(
            "get_unread_event_push_actions_by_room_for_user", (room_id,)
        )
        self._attempt_to_invalidate_cache("get_metadata_for_event", (room_id, event_id))
        self._attempt_to_invalidate_cache("_get_max_event_pos", (room_id,))
@ -457,6 +458,7 @@ class CacheInvalidationWorkerStore(SQLBaseStore):
        self._attempt_to_invalidate_cache("_get_state_group_for_event", None)
        self._attempt_to_invalidate_cache("get_event_ordering", None)
        self._attempt_to_invalidate_cache("get_metadata_for_event", (room_id,))
        self._attempt_to_invalidate_cache("is_partial_state_event", None)
        self._attempt_to_invalidate_cache("_get_joined_profile_from_event_id", None)
--- a/synapse/storage/databases/main/end_to_end_keys.py
+++ b/synapse/storage/databases/main/end_to_end_keys.py
@ -1453,6 +1453,54 @@ class EndToEndKeyWorkerStore(EndToEndKeyBackgroundStore, CacheInvalidationWorker
            impl,
        )
    async def delete_old_otks_for_next_user_batch(
        self, after_user_id: str, number_of_users: int
    ) -> Tuple[List[str], int]:
        """Deletes old OTKs belonging to the next batch of users
        Returns:
            `(users, rows)`, where:
             * `users` is the user IDs of the updated users. An empty list if we are done.
             * `rows` is the number of deleted rows
        """
        def impl(txn: LoggingTransaction) -> Tuple[List[str], int]:
            # Find a batch of users
            txn.execute(
                """
                SELECT DISTINCT(user_id) FROM e2e_one_time_keys_json
                    WHERE user_id > ?
                    ORDER BY user_id
                    LIMIT ?
                """,
                (after_user_id, number_of_users),
            )
            users = [row[0] for row in txn.fetchall()]
            if len(users) == 0:
                return users, 0
            # Delete any old OTKs belonging to those users.
            #
            # We only actually consider OTKs whose key ID is 6 characters long. These
            # keys were likely made by libolm rather than Vodozemac; libolm only kept
            # 100 private OTKs, so was far more vulnerable than Vodozemac to throwing
            # away keys prematurely.
            clause, args = make_in_list_sql_clause(
                txn.database_engine, "user_id", users
            )
            sql = f"""
                DELETE FROM e2e_one_time_keys_json
                WHERE {clause} AND ts_added_ms < ? AND length(key_id) = 6
                """
            args.append(self._clock.time_msec() - (7 * 24 * 3600 * 1000))
            txn.execute(sql, args)
            return users, txn.rowcount
        return await self.db_pool.runInteraction(
            "delete_old_otks_for_next_user_batch", impl
        )
 class EndToEndKeyStore(EndToEndKeyWorkerStore, SQLBaseStore):
    def __init__(
--- a/synapse/storage/databases/main/events_worker.py
+++ b/synapse/storage/databases/main/events_worker.py
@ -193,6 +193,14 @@ class _EventRow:
    outlier: bool
@attr.s(slots=True, frozen=True, auto_attribs=True)
 class EventMetadata:
    """Event metadata returned by `get_metadata_for_event(..)`"""
    sender: str
    received_ts: int
 class EventRedactBehaviour(Enum):
    """
    What to do when retrieving a redacted event from the database.
@ -2580,3 +2588,22 @@ class EventsWorkerStore(SQLBaseStore):
                _BackgroundUpdates.SLIDING_SYNC_MEMBERSHIP_SNAPSHOTS_BG_UPDATE,
            )
        )
    @cached(tree=True)
    async def get_metadata_for_event(
        self, room_id: str, event_id: str
    ) -> Optional[EventMetadata]:
        row = await self.db_pool.simple_select_one(
            table="events",
            keyvalues={"room_id": room_id, "event_id": event_id},
            retcols=("sender", "received_ts"),
            allow_none=True,
            desc="get_metadata_for_event",
        )
        if row is None:
            return None
        return EventMetadata(
            sender=row[0],
            received_ts=row[1],
        )
--- a/synapse/storage/databases/main/sliding_sync.py
+++ b/synapse/storage/databases/main/sliding_sync.py
@ -21,7 +21,11 @@ import attr
 from synapse.api.errors import SlidingSyncUnknownPosition
 from synapse.logging.opentracing import log_kv
 from synapse.storage._base import SQLBaseStore, db_to_json
-from synapse.storage.database import LoggingTransaction
+from synapse.storage.database import (
    DatabasePool,
    LoggingDatabaseConnection,
    LoggingTransaction,
 )
 from synapse.types import MultiWriterStreamToken, RoomStreamToken
 from synapse.types.handlers.sliding_sync import (
    HaveSentRoom,
@ -35,12 +39,28 @@ from synapse.util import json_encoder
 from synapse.util.caches.descriptors import cached
 if TYPE_CHECKING:
    from synapse.server import HomeServer
    from synapse.storage.databases.main import DataStore
 logger = logging.getLogger(__name__)
 class SlidingSyncStore(SQLBaseStore):
    def __init__(
        self,
        database: DatabasePool,
        db_conn: LoggingDatabaseConnection,
        hs: "HomeServer",
    ):
        super().__init__(database, db_conn, hs)
        self.db_pool.updates.register_background_index_update(
            update_name="sliding_sync_connection_room_configs_required_state_id_idx",
            index_name="sliding_sync_connection_room_configs_required_state_id_idx",
            table="sliding_sync_connection_room_configs",
            columns=("required_state_id",),
        )
    async def get_latest_bump_stamp_for_room(
        self,
        room_id: str,
--- a/synapse/storage/schema/main/delta/88/04_current_state_delta_index.sql
+++ b/synapse/storage/schema/main/delta/88/04_current_state_delta_index.sql
@ -12,7 +12,7 @@
 -- <https://www.gnu.org/licenses/agpl-3.0.html>.
-- Add an index on (user_id, device_id, algorithm, ts_added_ms) on e2e_one_time_keys_json, so that OTKs can
+-- Add an index on `current_state_delta_stream(room_id, stream_id)` to allow
-- efficiently be issued in the same order they were uploaded.
+-- efficient per-room lookups.
 INSERT INTO background_updates (ordering, update_name, progress_json) VALUES
    (8804, 'current_state_delta_stream_room_index', '{}');
--- a/synapse/storage/schema/main/delta/88/05_drop_old_otks.sql.postgres
+++ b/synapse/storage/schema/main/delta/88/05_drop_old_otks.sql.postgres
@ -0,0 +1,19 @@
 --
 -- This file is licensed under the Affero General Public License (AGPL) version 3.
 --
 -- Copyright (C) 2024 New Vector, Ltd
 --
 -- This program is free software: you can redistribute it and/or modify
 -- it under the terms of the GNU Affero General Public License as
 -- published by the Free Software Foundation, either version 3 of the
 -- License, or (at your option) any later version.
 --
 -- See the GNU Affero General Public License for more details:
 -- <https://www.gnu.org/licenses/agpl-3.0.html>.
 -- Until Synapse 1.119, Synapse used to issue one-time-keys in a random order, leading to the possibility
 -- that it could still have old OTKs that the client has dropped.
 --
 -- We create a scheduled task which will drop old OTKs, to flush them out.
 INSERT INTO scheduled_tasks(id, action, status, timestamp)
    VALUES ('delete_old_otks_task', 'delete_old_otks', 'scheduled', extract(epoch from current_timestamp) * 1000);
--- a/synapse/storage/schema/main/delta/88/05_drop_old_otks.sql.sqlite
+++ b/synapse/storage/schema/main/delta/88/05_drop_old_otks.sql.sqlite
@ -0,0 +1,19 @@
 --
 -- This file is licensed under the Affero General Public License (AGPL) version 3.
 --
 -- Copyright (C) 2024 New Vector, Ltd
 --
 -- This program is free software: you can redistribute it and/or modify
 -- it under the terms of the GNU Affero General Public License as
 -- published by the Free Software Foundation, either version 3 of the
 -- License, or (at your option) any later version.
 --
 -- See the GNU Affero General Public License for more details:
 -- <https://www.gnu.org/licenses/agpl-3.0.html>.
 -- Until Synapse 1.119, Synapse used to issue one-time-keys in a random order, leading to the possibility
 -- that it could still have old OTKs that the client has dropped.
 --
 -- We create a scheduled task which will drop old OTKs, to flush them out.
 INSERT INTO scheduled_tasks(id, action, status, timestamp)
    VALUES ('delete_old_otks_task', 'delete_old_otks', 'scheduled', strftime('%s', 'now') * 1000);
--- a/synapse/storage/schema/main/delta/88/05_sliding_sync_room_config_index.sql
+++ b/synapse/storage/schema/main/delta/88/05_sliding_sync_room_config_index.sql
@ -0,0 +1,20 @@
 --
 -- This file is licensed under the Affero General Public License (AGPL) version 3.
 --
 -- Copyright (C) 2024 New Vector, Ltd
 --
 -- This program is free software: you can redistribute it and/or modify
 -- it under the terms of the GNU Affero General Public License as
 -- published by the Free Software Foundation, either version 3 of the
 -- License, or (at your option) any later version.
 --
 -- See the GNU Affero General Public License for more details:
 -- <https://www.gnu.org/licenses/agpl-3.0.html>.
 -- Add an index on sliding_sync_connection_room_configs(required_state_id), so
 -- that when we delete entries in `sliding_sync_connection_required_state` it's
 -- efficient for Postgres to check they've been deleted from
 -- `sliding_sync_connection_room_configs` too
 INSERT INTO background_updates (ordering, update_name, progress_json) VALUES
    (8805, 'sliding_sync_connection_room_configs_required_state_id_idx', '{}');
--- a/tests/federation/test_federation_sender.py
+++ b/tests/federation/test_federation_sender.py
@ -34,6 +34,7 @@ from synapse.handlers.device import DeviceHandler
 from synapse.rest import admin
 from synapse.rest.client import login
 from synapse.server import HomeServer
 from synapse.storage.databases.main.events_worker import EventMetadata
 from synapse.types import JsonDict, ReadReceipt
 from synapse.util import Clock
@ -55,12 +56,15 @@ class FederationSenderReceiptsTestCases(HomeserverTestCase):
            federation_transport_client=self.federation_transport_client,
        )
-        hs.get_storage_controllers().state.get_current_hosts_in_room = AsyncMock(  # type: ignore[method-assign]
+        self.main_store = hs.get_datastores().main
        self.state_controller = hs.get_storage_controllers().state
        self.state_controller.get_current_hosts_in_room = AsyncMock(  # type: ignore[method-assign]
            return_value={"test", "host2"}
        )
-        hs.get_storage_controllers().state.get_current_hosts_in_room_or_partial_state_approximation = (  # type: ignore[method-assign]
+        self.state_controller.get_current_hosts_in_room_or_partial_state_approximation = (  # type: ignore[method-assign]
-            hs.get_storage_controllers().state.get_current_hosts_in_room
+            self.state_controller.get_current_hosts_in_room
        )
        return hs
@ -185,12 +189,15 @@ class FederationSenderReceiptsTestCases(HomeserverTestCase):
            ],
        )
-    def test_send_receipts_with_backoff(self) -> None:
+    def test_send_receipts_with_backoff_small_room(self) -> None:
-        """Send two receipts in quick succession; the second should be flushed, but
+        """Read receipt in small rooms should not be delayed"""
        only after 20ms"""
        mock_send_transaction = self.federation_transport_client.send_transaction
        mock_send_transaction.return_value = {}
        self.state_controller.get_current_hosts_in_room_or_partial_state_approximation = AsyncMock(  # type: ignore[method-assign]
            return_value={"test", "host2"}
        )
        sender = self.hs.get_federation_sender()
        receipt = ReadReceipt(
            "room_id",
@ -206,7 +213,104 @@ class FederationSenderReceiptsTestCases(HomeserverTestCase):
        # expect a call to send_transaction
        mock_send_transaction.assert_called_once()
-        json_cb = mock_send_transaction.call_args[0][1]
+        self._assert_edu_in_call(mock_send_transaction.call_args[0][1])
    def test_send_receipts_with_backoff_recent_event(self) -> None:
        """Read receipt for a recent message should not be delayed"""
        mock_send_transaction = self.federation_transport_client.send_transaction
        mock_send_transaction.return_value = {}
        # Pretend this is a big room
        self.state_controller.get_current_hosts_in_room_or_partial_state_approximation = AsyncMock(  # type: ignore[method-assign]
            return_value={"test"} | {f"host{i}" for i in range(20)}
        )
        self.main_store.get_metadata_for_event = AsyncMock(
            return_value=EventMetadata(
                received_ts=self.clock.time_msec(),
                sender="@test:test",
            )
        )
        sender = self.hs.get_federation_sender()
        receipt = ReadReceipt(
            "room_id",
            "m.read",
            "user_id",
            ["event_id"],
            thread_id=None,
            data={"ts": 1234},
        )
        self.get_success(sender.send_read_receipt(receipt))
        self.pump()
        # expect a call to send_transaction for each host
        self.assertEqual(mock_send_transaction.call_count, 20)
        self._assert_edu_in_call(mock_send_transaction.call_args.args[1])
        mock_send_transaction.reset_mock()
    def test_send_receipts_with_backoff_sender(self) -> None:
        """Read receipt for a message should not be delayed to the sender, but
        is delayed to everyone else"""
        mock_send_transaction = self.federation_transport_client.send_transaction
        mock_send_transaction.return_value = {}
        # Pretend this is a big room
        self.state_controller.get_current_hosts_in_room_or_partial_state_approximation = AsyncMock(  # type: ignore[method-assign]
            return_value={"test"} | {f"host{i}" for i in range(20)}
        )
        self.main_store.get_metadata_for_event = AsyncMock(
            return_value=EventMetadata(
                received_ts=self.clock.time_msec() - 5 * 60_000,
                sender="@test:host1",
            )
        )
        sender = self.hs.get_federation_sender()
        receipt = ReadReceipt(
            "room_id",
            "m.read",
            "user_id",
            ["event_id"],
            thread_id=None,
            data={"ts": 1234},
        )
        self.get_success(sender.send_read_receipt(receipt))
        self.pump()
        # First, expect a call to send_transaction for the sending host
        mock_send_transaction.assert_called()
        transaction = mock_send_transaction.call_args_list[0].args[0]
        self.assertEqual(transaction.destination, "host1")
        self._assert_edu_in_call(mock_send_transaction.call_args_list[0].args[1])
        # We also expect a call to one of the other hosts, as the first
        # destination to wake up.
        self.assertEqual(mock_send_transaction.call_count, 2)
        self._assert_edu_in_call(mock_send_transaction.call_args.args[1])
        mock_send_transaction.reset_mock()
        # We now expect to see 18 more transactions to the remaining hosts
        # periodically.
        for _ in range(18):
            self.reactor.advance(
                1.0
                / self.hs.config.ratelimiting.federation_rr_transactions_per_room_per_second
            )
            mock_send_transaction.assert_called_once()
            self._assert_edu_in_call(mock_send_transaction.call_args.args[1])
            mock_send_transaction.reset_mock()
    def _assert_edu_in_call(self, json_cb: Callable[[], JsonDict]) -> None:
        """Assert that the given `json_cb` from a `send_transaction` has a
        receipt in it."""
        data = json_cb()
        self.assertEqual(
            data["edus"],
@ -226,46 +330,6 @@ class FederationSenderReceiptsTestCases(HomeserverTestCase):
                }
            ],
        )
        mock_send_transaction.reset_mock()
        # send the second RR
        receipt = ReadReceipt(
            "room_id",
            "m.read",
            "user_id",
            ["other_id"],
            thread_id=None,
            data={"ts": 1234},
        )
        self.successResultOf(defer.ensureDeferred(sender.send_read_receipt(receipt)))
        self.pump()
        mock_send_transaction.assert_not_called()
        self.reactor.advance(19)
        mock_send_transaction.assert_not_called()
        self.reactor.advance(10)
        mock_send_transaction.assert_called_once()
        json_cb = mock_send_transaction.call_args[0][1]
        data = json_cb()
        self.assertEqual(
            data["edus"],
            [
                {
                    "edu_type": EduTypes.RECEIPT,
                    "content": {
                        "room_id": {
                            "m.read": {
                                "user_id": {
                                    "event_ids": ["other_id"],
                                    "data": {"ts": 1234},
                                }
                            }
                        }
                    },
                }
            ],
        )
 class FederationSenderPresenceTestCases(HomeserverTestCase):
--- a/tests/handlers/test_e2e_keys.py
+++ b/tests/handlers/test_e2e_keys.py
@ -19,6 +19,7 @@
 # [This file includes modifications made by New Vector Limited]
 #
 #
 import time
 from typing import Dict, Iterable
 from unittest import mock
@ -1826,3 +1827,72 @@ class E2eKeysHandlerTestCase(unittest.HomeserverTestCase):
        )
        self.assertIs(exists, True)
        self.assertIs(replaceable_without_uia, False)
    def test_delete_old_one_time_keys(self) -> None:
        """Test the db migration that clears out old OTKs"""
        # We upload two sets of keys, one just over a week ago, and one just less than
        # a week ago. Each batch contains some keys that match the deletion pattern
        # (key IDs of 6 chars), and some that do not.
        #
        # Finally, set the scheduled task going, and check what gets deleted.
        user_id = "@user000:" + self.hs.hostname
        device_id = "xyz"
        # The scheduled task should be for "now" in real, wallclock time, so
        # set the test reactor to just over a week ago.
        self.reactor.advance(time.time() - 7.5 * 24 * 3600)
        # Upload some keys
        self.get_success(
            self.handler.upload_keys_for_user(
                user_id,
                device_id,
                {
                    "one_time_keys": {
                        # some keys to delete
                        "alg1:AAAAAA": "key1",
                        "alg2:AAAAAB": {"key": "key2", "signatures": {"k1": "sig1"}},
                        # A key to *not* delete
                        "alg2:AAAAAAAAAA": {"key": "key3"},
                    }
                },
            )
        )
        # A day passes
        self.reactor.advance(24 * 3600)
        # Upload some more keys
        self.get_success(
            self.handler.upload_keys_for_user(
                user_id,
                device_id,
                {
                    "one_time_keys": {
                        # some keys which match the pattern
                        "alg1:BAAAAA": "key1",
                        "alg2:BAAAAB": {"key": "key2", "signatures": {"k1": "sig1"}},
                        # A key to *not* delete
                        "alg2:BAAAAAAAAA": {"key": "key3"},
                    }
                },
            )
        )
        # The rest of the week passes, which should set the scheduled task going.
        self.reactor.advance(6.5 * 24 * 3600)
        # Check what we're left with in the database
        remaining_key_ids = {
            row[0]
            for row in self.get_success(
                self.handler.store.db_pool.simple_select_list(
                    "e2e_one_time_keys_json", None, ["key_id"]
                )
            )
        }
        self.assertEqual(
            remaining_key_ids, {"AAAAAAAAAA", "BAAAAA", "BAAAAB", "BAAAAAAAAA"}
        )
--- a/tests/media/test_media_storage.py
+++ b/tests/media/test_media_storage.py
@ -419,6 +419,11 @@ class MediaRepoTests(unittest.HomeserverTestCase):
        return channel
    @unittest.override_config(
        {
            "enable_authenticated_media": False,
        }
    )
    def test_handle_missing_content_type(self) -> None:
        channel = self._req(
            b"attachment; filename=out" + self.test_image.extension,
@ -430,6 +435,11 @@ class MediaRepoTests(unittest.HomeserverTestCase):
            headers.getRawHeaders(b"Content-Type"), [b"application/octet-stream"]
        )
    @unittest.override_config(
        {
            "enable_authenticated_media": False,
        }
    )
    def test_disposition_filename_ascii(self) -> None:
        """
        If the filename is filename=<ascii> then Synapse will decode it as an
@ -450,6 +460,11 @@ class MediaRepoTests(unittest.HomeserverTestCase):
            ],
        )
    @unittest.override_config(
        {
            "enable_authenticated_media": False,
        }
    )
    def test_disposition_filenamestar_utf8escaped(self) -> None:
        """
        If the filename is filename=*utf8''<utf8 escaped> then Synapse will
@ -475,6 +490,11 @@ class MediaRepoTests(unittest.HomeserverTestCase):
            ],
        )
    @unittest.override_config(
        {
            "enable_authenticated_media": False,
        }
    )
    def test_disposition_none(self) -> None:
        """
        If there is no filename, Content-Disposition should only
@ -491,6 +511,11 @@ class MediaRepoTests(unittest.HomeserverTestCase):
            [b"inline" if self.test_image.is_inline else b"attachment"],
        )
    @unittest.override_config(
        {
            "enable_authenticated_media": False,
        }
    )
    def test_thumbnail_crop(self) -> None:
        """Test that a cropped remote thumbnail is available."""
        self._test_thumbnail(
@ -500,6 +525,11 @@ class MediaRepoTests(unittest.HomeserverTestCase):
            unable_to_thumbnail=self.test_image.unable_to_thumbnail,
        )
    @unittest.override_config(
        {
            "enable_authenticated_media": False,
        }
    )
    def test_thumbnail_scale(self) -> None:
        """Test that a scaled remote thumbnail is available."""
        self._test_thumbnail(
@ -509,6 +539,11 @@ class MediaRepoTests(unittest.HomeserverTestCase):
            unable_to_thumbnail=self.test_image.unable_to_thumbnail,
        )
    @unittest.override_config(
        {
            "enable_authenticated_media": False,
        }
    )
    def test_invalid_type(self) -> None:
        """An invalid thumbnail type is never available."""
        self._test_thumbnail(
@ -519,7 +554,10 @@ class MediaRepoTests(unittest.HomeserverTestCase):
        )
    @unittest.override_config(
-        {"thumbnail_sizes": [{"width": 32, "height": 32, "method": "scale"}]}
+        {
            "thumbnail_sizes": [{"width": 32, "height": 32, "method": "scale"}],
            "enable_authenticated_media": False,
        },
    )
    def test_no_thumbnail_crop(self) -> None:
        """
@ -533,7 +571,10 @@ class MediaRepoTests(unittest.HomeserverTestCase):
        )
    @unittest.override_config(
-        {"thumbnail_sizes": [{"width": 32, "height": 32, "method": "crop"}]}
+        {
            "thumbnail_sizes": [{"width": 32, "height": 32, "method": "crop"}],
            "enable_authenticated_media": False,
        }
    )
    def test_no_thumbnail_scale(self) -> None:
        """
@ -546,6 +587,11 @@ class MediaRepoTests(unittest.HomeserverTestCase):
            unable_to_thumbnail=self.test_image.unable_to_thumbnail,
        )
    @unittest.override_config(
        {
            "enable_authenticated_media": False,
        }
    )
    def test_thumbnail_repeated_thumbnail(self) -> None:
        """Test that fetching the same thumbnail works, and deleting the on disk
        thumbnail regenerates it.
@ -720,6 +766,11 @@ class MediaRepoTests(unittest.HomeserverTestCase):
            )
        )
    @unittest.override_config(
        {
            "enable_authenticated_media": False,
        }
    )
    def test_x_robots_tag_header(self) -> None:
        """
        Tests that the `X-Robots-Tag` header is present, which informs web crawlers
@ -733,6 +784,11 @@ class MediaRepoTests(unittest.HomeserverTestCase):
            [b"noindex, nofollow, noarchive, noimageindex"],
        )
    @unittest.override_config(
        {
            "enable_authenticated_media": False,
        }
    )
    def test_cross_origin_resource_policy_header(self) -> None:
        """
        Test that the Cross-Origin-Resource-Policy header is set to "cross-origin"
@ -747,6 +803,11 @@ class MediaRepoTests(unittest.HomeserverTestCase):
            [b"cross-origin"],
        )
    @unittest.override_config(
        {
            "enable_authenticated_media": False,
        }
    )
    def test_unknown_v3_endpoint(self) -> None:
        """
        If the v3 endpoint fails, try the r0 one.
@ -985,6 +1046,11 @@ class RemoteDownloadLimiterTestCase(unittest.HomeserverTestCase):
        d.callback(52428800)
        return d
    @override_config(
        {
            "enable_authenticated_media": False,
        }
    )
    @patch(
        "synapse.http.matrixfederationclient.read_body_with_max_size",
        read_body_with_max_size_30MiB,
@ -1060,6 +1126,7 @@ class RemoteDownloadLimiterTestCase(unittest.HomeserverTestCase):
        {
            "remote_media_download_per_second": "50M",
            "remote_media_download_burst_count": "50M",
            "enable_authenticated_media": False,
        }
    )
    @patch(
@ -1119,7 +1186,12 @@ class RemoteDownloadLimiterTestCase(unittest.HomeserverTestCase):
        )
        assert channel.code == 200
-    @override_config({"remote_media_download_burst_count": "87M"})
+    @override_config(
        {
            "remote_media_download_burst_count": "87M",
            "enable_authenticated_media": False,
        }
    )
    @patch(
        "synapse.http.matrixfederationclient.read_body_with_max_size",
        read_body_with_max_size_30MiB,
@ -1159,7 +1231,7 @@ class RemoteDownloadLimiterTestCase(unittest.HomeserverTestCase):
        )
        assert channel2.code == 429
-    @override_config({"max_upload_size": "29M"})
+    @override_config({"max_upload_size": "29M", "enable_authenticated_media": False})
    @patch(
        "synapse.http.matrixfederationclient.read_body_with_max_size",
        read_body_with_max_size_30MiB,
--- a/tests/replication/test_multi_media_repo.py
+++ b/tests/replication/test_multi_media_repo.py
@ -40,6 +40,7 @@ from tests.http import (
 from tests.replication._base import BaseMultiWorkerStreamTestCase
 from tests.server import FakeChannel, FakeTransport, make_request
 from tests.test_utils import SMALL_PNG
 from tests.unittest import override_config
 logger = logging.getLogger(__name__)
@ -148,6 +149,7 @@ class MediaRepoShardTestCase(BaseMultiWorkerStreamTestCase):
        return channel, request
    @override_config({"enable_authenticated_media": False})
    def test_basic(self) -> None:
        """Test basic fetching of remote media from a single worker."""
        hs1 = self.make_worker_hs("synapse.app.generic_worker")
@ -164,6 +166,7 @@ class MediaRepoShardTestCase(BaseMultiWorkerStreamTestCase):
        self.assertEqual(channel.code, 200)
        self.assertEqual(channel.result["body"], b"Hello!")
    @override_config({"enable_authenticated_media": False})
    def test_download_simple_file_race(self) -> None:
        """Test that fetching remote media from two different processes at the
        same time works.
@ -203,6 +206,7 @@ class MediaRepoShardTestCase(BaseMultiWorkerStreamTestCase):
        # We expect only one new file to have been persisted.
        self.assertEqual(start_count + 1, self._count_remote_media())
    @override_config({"enable_authenticated_media": False})
    def test_download_image_race(self) -> None:
        """Test that fetching remote *images* from two different processes at
        the same time works.
--- a/tests/rest/admin/test_admin.py
+++ b/tests/rest/admin/test_admin.py
@ -30,7 +30,7 @@ from twisted.web.resource import Resource
 import synapse.rest.admin
 from synapse.http.server import JsonResource
 from synapse.rest.admin import VersionServlet
-from synapse.rest.client import login, room
+from synapse.rest.client import login, media, room
 from synapse.server import HomeServer
 from synapse.util import Clock
@ -60,6 +60,7 @@ class QuarantineMediaTestCase(unittest.HomeserverTestCase):
        synapse.rest.admin.register_servlets,
        synapse.rest.admin.register_servlets_for_media_repo,
        login.register_servlets,
        media.register_servlets,
        room.register_servlets,
    ]
@ -74,7 +75,7 @@ class QuarantineMediaTestCase(unittest.HomeserverTestCase):
        """Ensure a piece of media is quarantined when trying to access it."""
        channel = self.make_request(
            "GET",
-            f"/_matrix/media/v3/download/{server_and_media_id}",
+            f"/_matrix/client/v1/media/download/{server_and_media_id}",
            shorthand=False,
            access_token=admin_user_tok,
        )
@ -131,7 +132,7 @@ class QuarantineMediaTestCase(unittest.HomeserverTestCase):
        # Attempt to access the media
        channel = self.make_request(
            "GET",
-            f"/_matrix/media/v3/download/{server_name_and_media_id}",
+            f"/_matrix/client/v1/media/download/{server_name_and_media_id}",
            shorthand=False,
            access_token=non_admin_user_tok,
        )
@ -295,7 +296,7 @@ class QuarantineMediaTestCase(unittest.HomeserverTestCase):
        # Attempt to access each piece of media
        channel = self.make_request(
            "GET",
-            f"/_matrix/media/v3/download/{server_and_media_id_2}",
+            f"/_matrix/client/v1/media/download/{server_and_media_id_2}",
            shorthand=False,
            access_token=non_admin_user_tok,
        )
--- a/tests/rest/admin/test_media.py
+++ b/tests/rest/admin/test_media.py
@ -36,6 +36,7 @@ from synapse.util import Clock
 from tests import unittest
 from tests.test_utils import SMALL_PNG
 from tests.unittest import override_config
 VALID_TIMESTAMP = 1609459200000  # 2021-01-01 in milliseconds
 INVALID_TIMESTAMP_IN_S = 1893456000  # 2030-01-01 in seconds
@ -126,6 +127,7 @@ class DeleteMediaByIDTestCase(_AdminMediaTests):
        self.assertEqual(400, channel.code, msg=channel.json_body)
        self.assertEqual("Can only delete local media", channel.json_body["error"])
    @override_config({"enable_authenticated_media": False})
    def test_delete_media(self) -> None:
        """
        Tests that delete a media is successfully
@ -371,6 +373,7 @@ class DeleteMediaByDateSizeTestCase(_AdminMediaTests):
        self._access_media(server_and_media_id, False)
    @override_config({"enable_authenticated_media": False})
    def test_keep_media_by_date(self) -> None:
        """
        Tests that media is not deleted if it is newer than `before_ts`
@ -408,6 +411,7 @@ class DeleteMediaByDateSizeTestCase(_AdminMediaTests):
        self._access_media(server_and_media_id, False)
    @override_config({"enable_authenticated_media": False})
    def test_keep_media_by_size(self) -> None:
        """
        Tests that media is not deleted if its size is smaller than or equal
@ -443,6 +447,7 @@ class DeleteMediaByDateSizeTestCase(_AdminMediaTests):
        self._access_media(server_and_media_id, False)
    @override_config({"enable_authenticated_media": False})
    def test_keep_media_by_user_avatar(self) -> None:
        """
        Tests that we do not delete media if is used as a user avatar
@ -487,6 +492,7 @@ class DeleteMediaByDateSizeTestCase(_AdminMediaTests):
        self._access_media(server_and_media_id, False)
    @override_config({"enable_authenticated_media": False})
    def test_keep_media_by_room_avatar(self) -> None:
        """
        Tests that we do not delete media if it is used as a room avatar
--- a/tests/rest/admin/test_user.py
+++ b/tests/rest/admin/test_user.py
@ -45,6 +45,7 @@ from synapse.rest.client import (
    devices,
    login,
    logout,
    media,
    profile,
    register,
    room,
@ -3221,6 +3222,7 @@ class UserRestTestCase(unittest.HomeserverTestCase):
        self.assertIn("consent_ts", content)
        self.assertIn("external_ids", content)
        self.assertIn("last_seen_ts", content)
        self.assertIn("suspended", content)
        # This key was removed intentionally. Ensure it is not accidentally re-included.
        self.assertNotIn("password_hash", content)
@ -3517,6 +3519,7 @@ class UserMediaRestTestCase(unittest.HomeserverTestCase):
    servlets = [
        synapse.rest.admin.register_servlets,
        login.register_servlets,
        media.register_servlets,
    ]
    def prepare(self, reactor: MemoryReactor, clock: Clock, hs: HomeServer) -> None:
@ -4023,7 +4026,7 @@ class UserMediaRestTestCase(unittest.HomeserverTestCase):
        # Try to access a media and to create `last_access_ts`
        channel = self.make_request(
            "GET",
-            f"/_matrix/media/v3/download/{server_and_media_id}",
+            f"/_matrix/client/v1/media/download/{server_and_media_id}",
            shorthand=False,
            access_token=user_token,
        )
--- a/tests/rest/media/test_domain_blocking.py
+++ b/tests/rest/media/test_domain_blocking.py
@ -91,7 +91,8 @@ class MediaDomainBlockingTests(unittest.HomeserverTestCase):
        {
            # Disable downloads from a domain we won't be requesting downloads from.
            # This proves we haven't broken anything.
-            "prevent_media_downloads_from": ["not-listed.com"]
+            "prevent_media_downloads_from": ["not-listed.com"],
            "enable_authenticated_media": False,
        }
    )
    def test_remote_media_normally_unblocked(self) -> None:
@ -132,6 +133,7 @@ class MediaDomainBlockingTests(unittest.HomeserverTestCase):
            # This proves we haven't broken anything.
            "prevent_media_downloads_from": ["not-listed.com"],
            "dynamic_thumbnails": True,
            "enable_authenticated_media": False,
        }
    )
    def test_remote_media_thumbnail_normally_unblocked(self) -> None:
--- a/tests/rest/media/test_url_preview.py
+++ b/tests/rest/media/test_url_preview.py
@ -42,6 +42,7 @@ from synapse.util.stringutils import parse_and_validate_mxc_uri
 from tests import unittest
 from tests.server import FakeTransport
 from tests.test_utils import SMALL_PNG
 from tests.unittest import override_config
 try:
    import lxml
@ -1259,6 +1260,7 @@ class URLPreviewTests(unittest.HomeserverTestCase):
        self.assertIsNone(_port)
        return host, media_id
    @override_config({"enable_authenticated_media": False})
    def test_storage_providers_exclude_files(self) -> None:
        """Test that files are not stored in or fetched from storage providers."""
        host, media_id = self._download_image()
@ -1301,6 +1303,7 @@ class URLPreviewTests(unittest.HomeserverTestCase):
            "URL cache file was unexpectedly retrieved from a storage provider",
        )
    @override_config({"enable_authenticated_media": False})
    def test_storage_providers_exclude_thumbnails(self) -> None:
        """Test that thumbnails are not stored in or fetched from storage providers."""
        host, media_id = self._download_image()
		`@ -0,0 +1 @@`
							[MSC4108](https://github.com/matrix-org/matrix-spec-proposals/pull/4108): Add a `Content-Type` header on the `PUT` response to work around a faulty behavior in some caching reverse proxies.
		`@ -1 +0,0 @@`
			`Remove support for closed [MSC3886](https://github.com/matrix-org/matrix-spec-proposals/pull/3886).`
		`@ -1 +0,0 @@`
			`Addressed some typos in docs and returned error message for unknown MXC ID.`
		`@ -0,0 +1 @@`
							`Add OIDC example configuration for Forgejo (fork of Gitea).`
		`@ -1 +0,0 @@`
			Clarify the semantics of the `enable_authenticated_media` configuration option.
		`@ -1 +0,0 @@`
			`Bump macos version used to build wheels during release, as current version used is end-of-life.`
		`@ -0,0 +1 @@`
							`Fix long-standing bug where read receipts could get overly delayed being sent over federation.`
		`@ -0,0 +1 @@`
							`Raise setuptools_rust version cap to 1.10.2.`
		`@ -0,0 +1 @@`
							`Enable encrypted appservice related experimental features in the complement docker image.`
		`@ -0,0 +1 @@`
							`Return whether the user is suspended when querying the user account in the Admin API.`