mirror of
https://github.com/element-hq/synapse.git
synced 2024-12-14 11:57:44 +00:00
deploy: 1357ae869f
This commit is contained in:
DMRobertson
parent
e4d3792708
commit
5b2c5a7ad4
7 changed files with 449 additions and 117 deletions
|
|
@ -3411,7 +3411,7 @@ including _matrix/...). This is the same URL a user might enter into the
|
|||
'Custom Homeserver URL' field on their client. If you use Synapse with a
|
||||
reverse proxy, this should be the URL to reach Synapse via the proxy.
|
||||
Otherwise, it should be the URL to reach Synapse's client HTTP listener (see
|
||||
'listeners' below).</p>
|
||||
<a href="usage/configuration/config_documentation.html#listeners">'listeners'</a> below).</p>
|
||||
<p>Defaults to <code>https://<server_name>/</code>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">public_baseurl: https://example.com/
|
||||
|
|
@ -4405,7 +4405,8 @@ when Synapse is started.</p>
|
|||
<p>Config options related to logging.</p>
|
||||
<hr />
|
||||
<h3 id="log_config"><a class="header" href="#log_config"><code>log_config</code></a></h3>
|
||||
<p>This option specifies a yaml python logging config file as described <a href="https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema">here</a>.</p>
|
||||
<p>This option specifies a yaml python logging config file as described
|
||||
<a href="https://docs.python.org/3/library/logging.config.html#configuration-dictionary-schema">here</a>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">log_config: "CONFDIR/SERVERNAME.log.config"
|
||||
</code></pre>
|
||||
|
|
@ -6425,31 +6426,82 @@ mostly related to trace sampling which is documented <a href="https://www.jaeger
|
|||
false
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h2 id="workers"><a class="header" href="#workers">Workers</a></h2>
|
||||
<p>Configuration options related to workers.</p>
|
||||
<h2 id="coordinating-workers"><a class="header" href="#coordinating-workers">Coordinating workers</a></h2>
|
||||
<p>Configuration options related to workers which belong in the main config file
|
||||
(usually called <code>homeserver.yaml</code>).
|
||||
A Synapse deployment can scale horizontally by running multiple Synapse processes
|
||||
called <em>workers</em>. Incoming requests are distributed between workers to handle higher
|
||||
loads. Some workers are privileged and can accept requests from other workers.</p>
|
||||
<p>As a result, the worker configuration is divided into two parts.</p>
|
||||
<ol>
|
||||
<li>The first part (in this section of the manual) defines which shardable tasks
|
||||
are delegated to privileged workers. This allows unprivileged workers to make
|
||||
request a privileged worker to act on their behalf.</li>
|
||||
<li><a href="usage/configuration/config_documentation.html#individual-worker-configuration">The second part</a>
|
||||
controls the behaviour of individual workers in isolation.</li>
|
||||
</ol>
|
||||
<p>For guidance on setting up workers, see the <a href="usage/configuration/../../workers.html">worker documentation</a>.</p>
|
||||
<hr />
|
||||
<h3 id="worker_replication_secret"><a class="header" href="#worker_replication_secret"><code>worker_replication_secret</code></a></h3>
|
||||
<p>A shared secret used by the replication APIs on the main process to authenticate
|
||||
HTTP requests from workers.</p>
|
||||
<p>The default, this value is omitted (equivalently <code>null</code>), which means that
|
||||
traffic between the workers and the main process is not authenticated.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_replication_secret: "secret_secret"
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="start_pushers"><a class="header" href="#start_pushers"><code>start_pushers</code></a></h3>
|
||||
<p>Controls sending of push notifications on the main process. Set to <code>false</code>
|
||||
if using a <a href="usage/configuration/../../workers.html#synapseapppusher">pusher worker</a>. Defaults to <code>true</code>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">start_pushers: false
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="pusher_instances"><a class="header" href="#pusher_instances"><code>pusher_instances</code></a></h3>
|
||||
<p>It is possible to run multiple <a href="usage/configuration/../../workers.html#synapseapppusher">pusher workers</a>,
|
||||
in which case the work is balanced across them. Use this setting to list the pushers by
|
||||
<a href="usage/configuration/config_documentation.html#worker_name"><code>worker_name</code></a>. Ensure the main process and all pusher workers are
|
||||
restarted after changing this option.</p>
|
||||
<p>If no or only one pusher worker is configured, this setting is not necessary.
|
||||
The main process will send out push notifications by default if you do not disable
|
||||
it by setting <a href="usage/configuration/config_documentation.html#start_pushers"><code>start_pushers: false</code></a>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">start_pushers: false
|
||||
pusher_instances:
|
||||
- pusher_worker1
|
||||
- pusher_worker2
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="send_federation"><a class="header" href="#send_federation"><code>send_federation</code></a></h3>
|
||||
<p>Controls sending of outbound federation transactions on the main process.
|
||||
Set to false if using a federation sender worker. Defaults to true.</p>
|
||||
Set to <code>false</code> if using a <a href="usage/configuration/../../workers.html#synapseappfederation_sender">federation sender worker</a>.
|
||||
Defaults to <code>true</code>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">send_federation: false
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="federation_sender_instances"><a class="header" href="#federation_sender_instances"><code>federation_sender_instances</code></a></h3>
|
||||
<p>It is possible to run multiple federation sender workers, in which case the
|
||||
work is balanced across them. Use this setting to list the senders.</p>
|
||||
<p>It is possible to run multiple
|
||||
<a href="usage/configuration/../../workers.html#synapseappfederation_sender">federation sender worker</a>, in which
|
||||
case the work is balanced across them. Use this setting to list the senders.</p>
|
||||
<p>This configuration setting must be shared between all federation sender workers, and if
|
||||
changed all federation sender workers must be stopped at the same time and then
|
||||
started, to ensure that all instances are running with the same config (otherwise
|
||||
events may be dropped).</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">federation_sender_instances:
|
||||
<pre><code class="language-yaml">send_federation: false
|
||||
federation_sender_instances:
|
||||
- federation_sender1
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="instance_map"><a class="header" href="#instance_map"><code>instance_map</code></a></h3>
|
||||
<p>When using workers this should be a map from worker name to the
|
||||
HTTP replication listener of the worker, if configured.</p>
|
||||
<p>When using workers this should be a map from <a href="usage/configuration/config_documentation.html#worker_name"><code>worker_name</code></a> to the
|
||||
HTTP replication listener of the worker, if configured.
|
||||
Each worker declared under <a href="usage/configuration/../../workers.html#stream-writers"><code>stream_writers</code></a> needs
|
||||
a HTTP replication listener, and that listener should be included in the <code>instance_map</code>.
|
||||
(The main process also needs an HTTP replication listener, but it should not be
|
||||
listed in the <code>instance_map</code>.)</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">instance_map:
|
||||
worker1:
|
||||
|
|
@ -6459,8 +6511,10 @@ HTTP replication listener of the worker, if configured.</p>
|
|||
<hr />
|
||||
<h3 id="stream_writers"><a class="header" href="#stream_writers"><code>stream_writers</code></a></h3>
|
||||
<p>Experimental: When using workers you can define which workers should
|
||||
handle event persistence and typing notifications. Any worker
|
||||
specified here must also be in the <code>instance_map</code>.</p>
|
||||
handle writing to streams such as event persistence and typing notifications.
|
||||
Any worker specified here must also be in the <a href="usage/configuration/config_documentation.html#instance_map"><code>instance_map</code></a>.</p>
|
||||
<p>See the list of available streams in the
|
||||
<a href="usage/configuration/../../workers.html#stream-writers">worker documentation</a>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">stream_writers:
|
||||
events: worker1
|
||||
|
|
@ -6468,22 +6522,15 @@ specified here must also be in the <code>instance_map</code>.</p>
|
|||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="run_background_tasks_on"><a class="header" href="#run_background_tasks_on"><code>run_background_tasks_on</code></a></h3>
|
||||
<p>The worker that is used to run background tasks (e.g. cleaning up expired
|
||||
data). If not provided this defaults to the main process.</p>
|
||||
<p>The <a href="usage/configuration/../../workers.html#background-tasks">worker</a> that is used to run
|
||||
background tasks (e.g. cleaning up expired data). If not provided this
|
||||
defaults to the main process.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">run_background_tasks_on: worker1
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_replication_secret"><a class="header" href="#worker_replication_secret"><code>worker_replication_secret</code></a></h3>
|
||||
<p>A shared secret used by the replication APIs to authenticate HTTP requests
|
||||
from workers.</p>
|
||||
<p>By default this is unused and traffic is not authenticated.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_replication_secret: "secret_secret"
|
||||
</code></pre>
|
||||
<h3 id="redis"><a class="header" href="#redis"><code>redis</code></a></h3>
|
||||
<p>Configuration for Redis when using workers. This <em>must</em> be enabled when
|
||||
using workers (unless using old style direct TCP configuration).
|
||||
<p>Configuration for Redis when using workers. This <em>must</em> be enabled when using workers.
|
||||
This setting has the following sub-options:</p>
|
||||
<ul>
|
||||
<li><code>enabled</code>: whether to use Redis support. Defaults to false.</li>
|
||||
|
|
@ -6498,6 +6545,90 @@ localhost and 6379</li>
|
|||
port: 6379
|
||||
password: <secret_password>
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h2 id="individual-worker-configuration"><a class="header" href="#individual-worker-configuration">Individual worker configuration</a></h2>
|
||||
<p>These options configure an individual worker, in its worker configuration file.
|
||||
They should be not be provided when configuring the main process.</p>
|
||||
<p>Note also the configuration above for
|
||||
<a href="usage/configuration/config_documentation.html#coordinating-workers">coordinating a cluster of workers</a>.</p>
|
||||
<p>For guidance on setting up workers, see the <a href="usage/configuration/../../workers.html">worker documentation</a>.</p>
|
||||
<hr />
|
||||
<h3 id="worker_app"><a class="header" href="#worker_app"><code>worker_app</code></a></h3>
|
||||
<p>The type of worker. The currently available worker applications are listed
|
||||
in <a href="usage/configuration/../../workers.html#available-worker-applications">worker documentation</a>.</p>
|
||||
<p>The most common worker is the
|
||||
<a href="usage/configuration/../../workers.html#synapseappgeneric_worker"><code>synapse.app.generic_worker</code></a>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_app: synapse.app.generic_worker
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_name"><a class="header" href="#worker_name"><code>worker_name</code></a></h3>
|
||||
<p>A unique name for the worker. The worker needs a name to be addressed in
|
||||
further parameters and identification in log files. We strongly recommend
|
||||
giving each worker a unique <code>worker_name</code>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_name: generic_worker1
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_replication_host"><a class="header" href="#worker_replication_host"><code>worker_replication_host</code></a></h3>
|
||||
<p>The HTTP replication endpoint that it should talk to on the main Synapse process.
|
||||
The main Synapse process defines this with a <code>replication</code> resource in
|
||||
<a href="usage/configuration/config_documentation.html#listeners"><code>listeners</code> option</a>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_replication_host: 127.0.0.1
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_replication_http_port"><a class="header" href="#worker_replication_http_port"><code>worker_replication_http_port</code></a></h3>
|
||||
<p>The HTTP replication port that it should talk to on the main Synapse process.
|
||||
The main Synapse process defines this with a <code>replication</code> resource in
|
||||
<a href="usage/configuration/config_documentation.html#listeners"><code>listeners</code> option</a>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_replication_http_port: 9093
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_listeners"><a class="header" href="#worker_listeners"><code>worker_listeners</code></a></h3>
|
||||
<p>A worker can handle HTTP requests. To do so, a <code>worker_listeners</code> option
|
||||
must be declared, in the same way as the <a href="usage/configuration/config_documentation.html#listeners"><code>listeners</code> option</a>
|
||||
in the shared config.</p>
|
||||
<p>Workers declared in <a href="usage/configuration/config_documentation.html#stream_writers"><code>stream_writers</code></a> will need to include a
|
||||
<code>replication</code> listener here, in order to accept internal HTTP requests from
|
||||
other workers.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_listeners:
|
||||
- type: http
|
||||
port: 8083
|
||||
resources:
|
||||
- names: [client, federation]
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_daemonize"><a class="header" href="#worker_daemonize"><code>worker_daemonize</code></a></h3>
|
||||
<p>Specifies whether the worker should be started as a daemon process.
|
||||
If Synapse is being managed by <a href="usage/configuration/../../systemd-with-workers/README.html">systemd</a>, this option
|
||||
must be omitted or set to <code>false</code>.</p>
|
||||
<p>Defaults to <code>false</code>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_daemonize: true
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_pid_file"><a class="header" href="#worker_pid_file"><code>worker_pid_file</code></a></h3>
|
||||
<p>When running a worker as a daemon, we need a place to store the
|
||||
<a href="https://en.wikipedia.org/wiki/Process_identifier">PID</a> of the worker.
|
||||
This option defines the location of that "pid file".</p>
|
||||
<p>This option is required if <code>worker_daemonize</code> is <code>true</code> and ignored
|
||||
otherwise. It has no default.</p>
|
||||
<p>See also the <a href="usage/configuration/config_documentation.html#pid_file"><code>pid_file</code> option</a> option for the main Synapse process.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_pid_file: DATADIR/generic_worker1.pid
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_log_config"><a class="header" href="#worker_log_config"><code>worker_log_config</code></a></h3>
|
||||
<p>This option specifies a yaml python logging config file as described
|
||||
<a href="https://docs.python.org/3/library/logging.config.html#configuration-dictionary-schema">here</a>.
|
||||
See also the <a href="usage/configuration/config_documentation.html#log_config"><code>log_config</code> option</a> option for the main Synapse process.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_log_config: /etc/matrix-synapse/generic-worker-log.yaml
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h2 id="background-updates"><a class="header" href="#background-updates">Background Updates</a></h2>
|
||||
<p>Configuration settings related to background updates.</p>
|
||||
<hr />
|
||||
|
|
@ -6595,7 +6726,7 @@ It should be named <code><SERVERNAME>.log.config</code> by default.</p>
|
|||
# Synapse also supports structured logging for machine readable logs which can
|
||||
# be ingested by ELK stacks. See [2] for details.
|
||||
#
|
||||
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
|
||||
# [1]: https://docs.python.org/3/library/logging.config.html#configuration-dictionary-schema
|
||||
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
|
||||
|
||||
version: 1
|
||||
|
|
@ -9873,10 +10004,12 @@ need its own configuration file and can take all of its configuration from the
|
|||
shared configuration file.</p>
|
||||
<h3 id="shared-configuration"><a class="header" href="#shared-configuration">Shared configuration</a></h3>
|
||||
<p>Normally, only a couple of changes are needed to make an existing configuration
|
||||
file suitable for use with workers. First, you need to enable an "HTTP replication
|
||||
listener" for the main process; and secondly, you need to enable redis-based
|
||||
replication. Optionally, a shared secret can be used to authenticate HTTP
|
||||
traffic between workers. For example:</p>
|
||||
file suitable for use with workers. First, you need to enable an
|
||||
<a href="usage/configuration/config_documentation.html#listeners">"HTTP replication listener"</a>
|
||||
for the main process; and secondly, you need to enable
|
||||
<a href="usage/configuration/config_documentation.html#redis">redis-based replication</a>.
|
||||
Optionally, a <a href="usage/configuration/config_documentation.html#worker_replication_secret">shared secret</a>
|
||||
can be used to authenticate HTTP traffic between workers. For example:</p>
|
||||
<pre><code class="language-yaml"># extend the existing `listeners` section. This defines the ports that the
|
||||
# main process will listen on.
|
||||
listeners:
|
||||
|
|
@ -9893,23 +10026,26 @@ worker_replication_secret: ""
|
|||
redis:
|
||||
enabled: true
|
||||
</code></pre>
|
||||
<p>See the <a href="usage/configuration/config_documentation.html">configuration manual</a> for the full documentation of each option.</p>
|
||||
<p>See the <a href="usage/configuration/config_documentation.html">configuration manual</a>
|
||||
for the full documentation of each option.</p>
|
||||
<p>Under <strong>no circumstances</strong> should the replication listener be exposed to the
|
||||
public internet; replication traffic is:</p>
|
||||
<ul>
|
||||
<li>always unencrypted</li>
|
||||
<li>unauthenticated, unless <code>worker_replication_secret</code> is configured</li>
|
||||
<li>unauthenticated, unless <a href="usage/configuration/config_documentation.html#worker_replication_secret"><code>worker_replication_secret</code></a>
|
||||
is configured</li>
|
||||
</ul>
|
||||
<h3 id="worker-configuration"><a class="header" href="#worker-configuration">Worker configuration</a></h3>
|
||||
<p>In the config file for each worker, you must specify:</p>
|
||||
<ul>
|
||||
<li>The type of worker (<code>worker_app</code>). The currently available worker applications are listed below.</li>
|
||||
<li>A unique name for the worker (<code>worker_name</code>).</li>
|
||||
<li>The type of worker (<a href="usage/configuration/config_documentation.html#worker_app"><code>worker_app</code></a>).
|
||||
The currently available worker applications are listed <a href="workers.html#available-worker-applications">below</a>.</li>
|
||||
<li>A unique name for the worker (<a href="usage/configuration/config_documentation.html#worker_name"><code>worker_name</code></a>).</li>
|
||||
<li>The HTTP replication endpoint that it should talk to on the main synapse process
|
||||
(<code>worker_replication_host</code> and <code>worker_replication_http_port</code>)</li>
|
||||
<li>If handling HTTP requests, a <code>worker_listeners</code> option with an <code>http</code>
|
||||
listener, in the same way as the <a href="usage/configuration/config_documentation.html#listeners"><code>listeners</code></a>
|
||||
option in the shared config.</li>
|
||||
(<a href="usage/configuration/config_documentation.html#worker_replication_host"><code>worker_replication_host</code></a> and
|
||||
<a href="usage/configuration/config_documentation.html#worker_replication_http_port"><code>worker_replication_http_port</code></a>).</li>
|
||||
<li>If handling HTTP requests, a <a href="usage/configuration/config_documentation.html#worker_listeners"><code>worker_listeners</code></a> option
|
||||
with an <code>http</code> listener.</li>
|
||||
<li>If handling the <code>^/_matrix/client/v3/keys/upload</code> endpoint, the HTTP URI for
|
||||
the main process (<code>worker_main_http_uri</code>).</li>
|
||||
</ul>
|
||||
|
|
@ -10067,7 +10203,8 @@ For multiple workers not handling the SSO endpoints properly, see
|
|||
<a href="https://github.com/matrix-org/synapse/issues/7530">#7530</a> and
|
||||
<a href="https://github.com/matrix-org/synapse/issues/9427">#9427</a>.</p>
|
||||
<p>Note that a <a href="usage/configuration/config_documentation.html#listeners">HTTP listener</a>
|
||||
with <code>client</code> and <code>federation</code> <code>resources</code> must be configured in the <code>worker_listeners</code>
|
||||
with <code>client</code> and <code>federation</code> <code>resources</code> must be configured in the
|
||||
<a href="usage/configuration/config_documentation.html#worker_listeners"><code>worker_listeners</code></a>
|
||||
option in the worker config.</p>
|
||||
<h4 id="load-balancing"><a class="header" href="#load-balancing">Load balancing</a></h4>
|
||||
<p>It is possible to run multiple instances of this worker app, with incoming requests
|
||||
|
|
@ -10100,9 +10237,10 @@ effects of bursts of events from that bridge on events sent by normal users.</p>
|
|||
of the main process to a particular worker.</p>
|
||||
<p>To enable this, the worker must have a
|
||||
<a href="usage/configuration/config_documentation.html#listeners">HTTP <code>replication</code> listener</a> configured,
|
||||
have a <code>worker_name</code> and be listed in the <code>instance_map</code> config. The same worker
|
||||
can handle multiple streams, but unless otherwise documented, each stream can only
|
||||
have a single writer.</p>
|
||||
have a <a href="usage/configuration/config_documentation.html#worker_name"><code>worker_name</code></a>
|
||||
and be listed in the <a href="usage/configuration/config_documentation.html#instance_map"><code>instance_map</code></a>
|
||||
config. The same worker can handle multiple streams, but unless otherwise documented,
|
||||
each stream can only have a single writer.</p>
|
||||
<p>For example, to move event persistence off to a dedicated worker, the shared
|
||||
configuration would include:</p>
|
||||
<pre><code class="language-yaml">instance_map:
|
||||
|
|
@ -10142,9 +10280,24 @@ worker_log_config: /etc/matrix-synapse/event-persister-log.yaml
|
|||
be routed to the workers handling that stream. See below for the currently supported
|
||||
streams and the endpoints associated with them:</p>
|
||||
<h5 id="the-events-stream"><a class="header" href="#the-events-stream">The <code>events</code> stream</a></h5>
|
||||
<p>The <code>events</code> stream experimentally supports having multiple writers, where work
|
||||
is sharded between them by room ID. Note that you <em>must</em> restart all worker
|
||||
instances when adding or removing event persisters. An example <code>stream_writers</code>
|
||||
<p>The <code>events</code> stream experimentally supports having multiple writer workers, where load
|
||||
is sharded between them by room ID. Each writer is called an <em>event persister</em>. They are
|
||||
responsible for</p>
|
||||
<ul>
|
||||
<li>receiving new events,</li>
|
||||
<li>linking them to those already in the room <a href="development/room-dag-concepts.html">DAG</a>,</li>
|
||||
<li>persisting them to the DB, and finally</li>
|
||||
<li>updating the events stream.</li>
|
||||
</ul>
|
||||
<p>Because load is sharded in this way, you <em>must</em> restart all worker instances when
|
||||
adding or removing event persisters.</p>
|
||||
<p>An <code>event_persister</code> should not be mistaken for an <code>event_creator</code>.
|
||||
An <code>event_creator</code> listens for requests from clients to create new events and does
|
||||
so. It will then pass those events over HTTP replication to any configured event
|
||||
persisters (or the main process if none are configured).</p>
|
||||
<p>Note that <code>event_creator</code>s and <code>event_persister</code>s are implemented using the same
|
||||
<a href="workers.html#synapse.app.generic_worker"><code>synapse.app.generic_worker</code></a>.</p>
|
||||
<p>An example <a href="usage/configuration/config_documentation.html#stream_writers"><code>stream_writers</code></a>
|
||||
configuration with multiple writers:</p>
|
||||
<pre><code class="language-yaml">stream_writers:
|
||||
events:
|
||||
|
|
@ -10183,13 +10336,15 @@ the stream writer for the <code>presence</code> stream:</p>
|
|||
worker. Background tasks are run periodically or started via replication. Exactly
|
||||
which tasks are configured to run depends on your Synapse configuration (e.g. if
|
||||
stats is enabled). This worker doesn't handle any REST endpoints itself.</p>
|
||||
<p>To enable this, the worker must have a <code>worker_name</code> and can be configured to run
|
||||
background tasks. For example, to move background tasks to a dedicated worker,
|
||||
the shared configuration would include:</p>
|
||||
<p>To enable this, the worker must have a unique
|
||||
<a href="usage/configuration/config_documentation.html#worker_name"><code>worker_name</code></a>
|
||||
and can be configured to run background tasks. For example, to move background tasks
|
||||
to a dedicated worker, the shared configuration would include:</p>
|
||||
<pre><code class="language-yaml">run_background_tasks_on: background_worker
|
||||
</code></pre>
|
||||
<p>You might also wish to investigate the <code>update_user_directory_from_worker</code> and
|
||||
<code>media_instance_running_background_jobs</code> settings.</p>
|
||||
<p>You might also wish to investigate the
|
||||
<a href="workers.html#updating-the-user-directory"><code>update_user_directory_from_worker</code></a> and
|
||||
<a href="workers.html#synapseappmedia_repository"><code>media_instance_running_background_jobs</code></a> settings.</p>
|
||||
<p>An example for a dedicated background worker instance:</p>
|
||||
<pre><code class="language-yaml">worker_app: synapse.app.generic_worker
|
||||
worker_name: background_worker
|
||||
|
|
@ -10227,11 +10382,15 @@ after setting this option in the shared configuration!</p>
|
|||
worker application type.</p>
|
||||
<h3 id="synapseapppusher"><a class="header" href="#synapseapppusher"><code>synapse.app.pusher</code></a></h3>
|
||||
<p>Handles sending push notifications to sygnal and email. Doesn't handle any
|
||||
REST endpoints itself, but you should set <code>start_pushers: False</code> in the
|
||||
REST endpoints itself, but you should set
|
||||
<a href="usage/configuration/config_documentation.html#start_pushers"><code>start_pushers: false</code></a> in the
|
||||
shared configuration file to stop the main synapse sending push notifications.</p>
|
||||
<p>To run multiple instances at once the <code>pusher_instances</code> option should list all
|
||||
pusher instances by their worker name, e.g.:</p>
|
||||
<pre><code class="language-yaml">pusher_instances:
|
||||
<p>To run multiple instances at once the
|
||||
<a href="usage/configuration/config_documentation.html#pusher_instances"><code>pusher_instances</code></a>
|
||||
option should list all pusher instances by their
|
||||
<a href="usage/configuration/config_documentation.html#worker_name"><code>worker_name</code></a>, e.g.:</p>
|
||||
<pre><code class="language-yaml">start_pushers: false
|
||||
pusher_instances:
|
||||
- pusher_worker1
|
||||
- pusher_worker2
|
||||
</code></pre>
|
||||
|
|
@ -10254,13 +10413,18 @@ shared configuration file to stop the main synapse sending appservice notificati
|
|||
<p>Note this worker cannot be load-balanced: only one instance should be active.</p>
|
||||
<h3 id="synapseappfederation_sender"><a class="header" href="#synapseappfederation_sender"><code>synapse.app.federation_sender</code></a></h3>
|
||||
<p>Handles sending federation traffic to other servers. Doesn't handle any
|
||||
REST endpoints itself, but you should set <code>send_federation: False</code> in the
|
||||
shared configuration file to stop the main synapse sending this traffic.</p>
|
||||
REST endpoints itself, but you should set
|
||||
<a href="usage/configuration/config_documentation.html#send_federation"><code>send_federation: false</code></a>
|
||||
in the shared configuration file to stop the main synapse sending this traffic.</p>
|
||||
<p>If running multiple federation senders then you must list each
|
||||
instance in the <code>federation_sender_instances</code> option by their <code>worker_name</code>.
|
||||
instance in the
|
||||
<a href="usage/configuration/config_documentation.html#federation_sender_instances"><code>federation_sender_instances</code></a>
|
||||
option by their
|
||||
<a href="usage/configuration/config_documentation.html#worker_name"><code>worker_name</code></a>.
|
||||
All instances must be stopped and started when adding or removing instances.
|
||||
For example:</p>
|
||||
<pre><code class="language-yaml">federation_sender_instances:
|
||||
<pre><code class="language-yaml">send_federation: false
|
||||
federation_sender_instances:
|
||||
- federation_sender1
|
||||
- federation_sender2
|
||||
</code></pre>
|
||||
|
|
@ -10286,7 +10450,9 @@ worker_log_config: /etc/matrix-synapse/federation-sender-log.yaml
|
|||
^/_synapse/admin/v1/quarantine_media/.*$
|
||||
^/_synapse/admin/v1/users/.*/media$
|
||||
</code></pre>
|
||||
<p>You should also set <code>enable_media_repo: False</code> in the shared configuration
|
||||
<p>You should also set
|
||||
<a href="usage/configuration/config_documentation.html#enable_media_repo"><code>enable_media_repo: False</code></a>
|
||||
in the shared configuration
|
||||
file to stop the main synapse running background jobs related to managing the
|
||||
media repository. Note that doing so will prevent the main process from being
|
||||
able to handle the above endpoints.</p>
|
||||
|
|
|
|||
|
|
@ -6,7 +6,7 @@
|
|||
# Synapse also supports structured logging for machine readable logs which can
|
||||
# be ingested by ELK stacks. See [2] for details.
|
||||
#
|
||||
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
|
||||
# [1]: https://docs.python.org/3/library/logging.config.html#configuration-dictionary-schema
|
||||
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
|
||||
|
||||
version: 1
|
||||
|
|
|
|||
File diff suppressed because one or more lines are too long
File diff suppressed because one or more lines are too long
|
|
@ -275,7 +275,7 @@ including _matrix/...). This is the same URL a user might enter into the
|
|||
'Custom Homeserver URL' field on their client. If you use Synapse with a
|
||||
reverse proxy, this should be the URL to reach Synapse via the proxy.
|
||||
Otherwise, it should be the URL to reach Synapse's client HTTP listener (see
|
||||
'listeners' below).</p>
|
||||
<a href="#listeners">'listeners'</a> below).</p>
|
||||
<p>Defaults to <code>https://<server_name>/</code>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">public_baseurl: https://example.com/
|
||||
|
|
@ -1269,7 +1269,8 @@ when Synapse is started.</p>
|
|||
<p>Config options related to logging.</p>
|
||||
<hr />
|
||||
<h3 id="log_config"><a class="header" href="#log_config"><code>log_config</code></a></h3>
|
||||
<p>This option specifies a yaml python logging config file as described <a href="https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema">here</a>.</p>
|
||||
<p>This option specifies a yaml python logging config file as described
|
||||
<a href="https://docs.python.org/3/library/logging.config.html#configuration-dictionary-schema">here</a>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">log_config: "CONFDIR/SERVERNAME.log.config"
|
||||
</code></pre>
|
||||
|
|
@ -3289,31 +3290,82 @@ mostly related to trace sampling which is documented <a href="https://www.jaeger
|
|||
false
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h2 id="workers"><a class="header" href="#workers">Workers</a></h2>
|
||||
<p>Configuration options related to workers.</p>
|
||||
<h2 id="coordinating-workers"><a class="header" href="#coordinating-workers">Coordinating workers</a></h2>
|
||||
<p>Configuration options related to workers which belong in the main config file
|
||||
(usually called <code>homeserver.yaml</code>).
|
||||
A Synapse deployment can scale horizontally by running multiple Synapse processes
|
||||
called <em>workers</em>. Incoming requests are distributed between workers to handle higher
|
||||
loads. Some workers are privileged and can accept requests from other workers.</p>
|
||||
<p>As a result, the worker configuration is divided into two parts.</p>
|
||||
<ol>
|
||||
<li>The first part (in this section of the manual) defines which shardable tasks
|
||||
are delegated to privileged workers. This allows unprivileged workers to make
|
||||
request a privileged worker to act on their behalf.</li>
|
||||
<li><a href="#individual-worker-configuration">The second part</a>
|
||||
controls the behaviour of individual workers in isolation.</li>
|
||||
</ol>
|
||||
<p>For guidance on setting up workers, see the <a href="../../workers.html">worker documentation</a>.</p>
|
||||
<hr />
|
||||
<h3 id="worker_replication_secret"><a class="header" href="#worker_replication_secret"><code>worker_replication_secret</code></a></h3>
|
||||
<p>A shared secret used by the replication APIs on the main process to authenticate
|
||||
HTTP requests from workers.</p>
|
||||
<p>The default, this value is omitted (equivalently <code>null</code>), which means that
|
||||
traffic between the workers and the main process is not authenticated.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_replication_secret: "secret_secret"
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="start_pushers"><a class="header" href="#start_pushers"><code>start_pushers</code></a></h3>
|
||||
<p>Controls sending of push notifications on the main process. Set to <code>false</code>
|
||||
if using a <a href="../../workers.html#synapseapppusher">pusher worker</a>. Defaults to <code>true</code>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">start_pushers: false
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="pusher_instances"><a class="header" href="#pusher_instances"><code>pusher_instances</code></a></h3>
|
||||
<p>It is possible to run multiple <a href="../../workers.html#synapseapppusher">pusher workers</a>,
|
||||
in which case the work is balanced across them. Use this setting to list the pushers by
|
||||
<a href="#worker_name"><code>worker_name</code></a>. Ensure the main process and all pusher workers are
|
||||
restarted after changing this option.</p>
|
||||
<p>If no or only one pusher worker is configured, this setting is not necessary.
|
||||
The main process will send out push notifications by default if you do not disable
|
||||
it by setting <a href="#start_pushers"><code>start_pushers: false</code></a>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">start_pushers: false
|
||||
pusher_instances:
|
||||
- pusher_worker1
|
||||
- pusher_worker2
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="send_federation"><a class="header" href="#send_federation"><code>send_federation</code></a></h3>
|
||||
<p>Controls sending of outbound federation transactions on the main process.
|
||||
Set to false if using a federation sender worker. Defaults to true.</p>
|
||||
Set to <code>false</code> if using a <a href="../../workers.html#synapseappfederation_sender">federation sender worker</a>.
|
||||
Defaults to <code>true</code>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">send_federation: false
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="federation_sender_instances"><a class="header" href="#federation_sender_instances"><code>federation_sender_instances</code></a></h3>
|
||||
<p>It is possible to run multiple federation sender workers, in which case the
|
||||
work is balanced across them. Use this setting to list the senders.</p>
|
||||
<p>It is possible to run multiple
|
||||
<a href="../../workers.html#synapseappfederation_sender">federation sender worker</a>, in which
|
||||
case the work is balanced across them. Use this setting to list the senders.</p>
|
||||
<p>This configuration setting must be shared between all federation sender workers, and if
|
||||
changed all federation sender workers must be stopped at the same time and then
|
||||
started, to ensure that all instances are running with the same config (otherwise
|
||||
events may be dropped).</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">federation_sender_instances:
|
||||
<pre><code class="language-yaml">send_federation: false
|
||||
federation_sender_instances:
|
||||
- federation_sender1
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="instance_map"><a class="header" href="#instance_map"><code>instance_map</code></a></h3>
|
||||
<p>When using workers this should be a map from worker name to the
|
||||
HTTP replication listener of the worker, if configured.</p>
|
||||
<p>When using workers this should be a map from <a href="#worker_name"><code>worker_name</code></a> to the
|
||||
HTTP replication listener of the worker, if configured.
|
||||
Each worker declared under <a href="../../workers.html#stream-writers"><code>stream_writers</code></a> needs
|
||||
a HTTP replication listener, and that listener should be included in the <code>instance_map</code>.
|
||||
(The main process also needs an HTTP replication listener, but it should not be
|
||||
listed in the <code>instance_map</code>.)</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">instance_map:
|
||||
worker1:
|
||||
|
|
@ -3323,8 +3375,10 @@ HTTP replication listener of the worker, if configured.</p>
|
|||
<hr />
|
||||
<h3 id="stream_writers"><a class="header" href="#stream_writers"><code>stream_writers</code></a></h3>
|
||||
<p>Experimental: When using workers you can define which workers should
|
||||
handle event persistence and typing notifications. Any worker
|
||||
specified here must also be in the <code>instance_map</code>.</p>
|
||||
handle writing to streams such as event persistence and typing notifications.
|
||||
Any worker specified here must also be in the <a href="#instance_map"><code>instance_map</code></a>.</p>
|
||||
<p>See the list of available streams in the
|
||||
<a href="../../workers.html#stream-writers">worker documentation</a>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">stream_writers:
|
||||
events: worker1
|
||||
|
|
@ -3332,22 +3386,15 @@ specified here must also be in the <code>instance_map</code>.</p>
|
|||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="run_background_tasks_on"><a class="header" href="#run_background_tasks_on"><code>run_background_tasks_on</code></a></h3>
|
||||
<p>The worker that is used to run background tasks (e.g. cleaning up expired
|
||||
data). If not provided this defaults to the main process.</p>
|
||||
<p>The <a href="../../workers.html#background-tasks">worker</a> that is used to run
|
||||
background tasks (e.g. cleaning up expired data). If not provided this
|
||||
defaults to the main process.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">run_background_tasks_on: worker1
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_replication_secret"><a class="header" href="#worker_replication_secret"><code>worker_replication_secret</code></a></h3>
|
||||
<p>A shared secret used by the replication APIs to authenticate HTTP requests
|
||||
from workers.</p>
|
||||
<p>By default this is unused and traffic is not authenticated.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_replication_secret: "secret_secret"
|
||||
</code></pre>
|
||||
<h3 id="redis"><a class="header" href="#redis"><code>redis</code></a></h3>
|
||||
<p>Configuration for Redis when using workers. This <em>must</em> be enabled when
|
||||
using workers (unless using old style direct TCP configuration).
|
||||
<p>Configuration for Redis when using workers. This <em>must</em> be enabled when using workers.
|
||||
This setting has the following sub-options:</p>
|
||||
<ul>
|
||||
<li><code>enabled</code>: whether to use Redis support. Defaults to false.</li>
|
||||
|
|
@ -3362,6 +3409,90 @@ localhost and 6379</li>
|
|||
port: 6379
|
||||
password: <secret_password>
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h2 id="individual-worker-configuration"><a class="header" href="#individual-worker-configuration">Individual worker configuration</a></h2>
|
||||
<p>These options configure an individual worker, in its worker configuration file.
|
||||
They should be not be provided when configuring the main process.</p>
|
||||
<p>Note also the configuration above for
|
||||
<a href="#coordinating-workers">coordinating a cluster of workers</a>.</p>
|
||||
<p>For guidance on setting up workers, see the <a href="../../workers.html">worker documentation</a>.</p>
|
||||
<hr />
|
||||
<h3 id="worker_app"><a class="header" href="#worker_app"><code>worker_app</code></a></h3>
|
||||
<p>The type of worker. The currently available worker applications are listed
|
||||
in <a href="../../workers.html#available-worker-applications">worker documentation</a>.</p>
|
||||
<p>The most common worker is the
|
||||
<a href="../../workers.html#synapseappgeneric_worker"><code>synapse.app.generic_worker</code></a>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_app: synapse.app.generic_worker
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_name"><a class="header" href="#worker_name"><code>worker_name</code></a></h3>
|
||||
<p>A unique name for the worker. The worker needs a name to be addressed in
|
||||
further parameters and identification in log files. We strongly recommend
|
||||
giving each worker a unique <code>worker_name</code>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_name: generic_worker1
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_replication_host"><a class="header" href="#worker_replication_host"><code>worker_replication_host</code></a></h3>
|
||||
<p>The HTTP replication endpoint that it should talk to on the main Synapse process.
|
||||
The main Synapse process defines this with a <code>replication</code> resource in
|
||||
<a href="#listeners"><code>listeners</code> option</a>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_replication_host: 127.0.0.1
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_replication_http_port"><a class="header" href="#worker_replication_http_port"><code>worker_replication_http_port</code></a></h3>
|
||||
<p>The HTTP replication port that it should talk to on the main Synapse process.
|
||||
The main Synapse process defines this with a <code>replication</code> resource in
|
||||
<a href="#listeners"><code>listeners</code> option</a>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_replication_http_port: 9093
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_listeners"><a class="header" href="#worker_listeners"><code>worker_listeners</code></a></h3>
|
||||
<p>A worker can handle HTTP requests. To do so, a <code>worker_listeners</code> option
|
||||
must be declared, in the same way as the <a href="#listeners"><code>listeners</code> option</a>
|
||||
in the shared config.</p>
|
||||
<p>Workers declared in <a href="#stream_writers"><code>stream_writers</code></a> will need to include a
|
||||
<code>replication</code> listener here, in order to accept internal HTTP requests from
|
||||
other workers.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_listeners:
|
||||
- type: http
|
||||
port: 8083
|
||||
resources:
|
||||
- names: [client, federation]
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_daemonize"><a class="header" href="#worker_daemonize"><code>worker_daemonize</code></a></h3>
|
||||
<p>Specifies whether the worker should be started as a daemon process.
|
||||
If Synapse is being managed by <a href="../../systemd-with-workers/README.html">systemd</a>, this option
|
||||
must be omitted or set to <code>false</code>.</p>
|
||||
<p>Defaults to <code>false</code>.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_daemonize: true
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_pid_file"><a class="header" href="#worker_pid_file"><code>worker_pid_file</code></a></h3>
|
||||
<p>When running a worker as a daemon, we need a place to store the
|
||||
<a href="https://en.wikipedia.org/wiki/Process_identifier">PID</a> of the worker.
|
||||
This option defines the location of that "pid file".</p>
|
||||
<p>This option is required if <code>worker_daemonize</code> is <code>true</code> and ignored
|
||||
otherwise. It has no default.</p>
|
||||
<p>See also the <a href="#pid_file"><code>pid_file</code> option</a> option for the main Synapse process.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_pid_file: DATADIR/generic_worker1.pid
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h3 id="worker_log_config"><a class="header" href="#worker_log_config"><code>worker_log_config</code></a></h3>
|
||||
<p>This option specifies a yaml python logging config file as described
|
||||
<a href="https://docs.python.org/3/library/logging.config.html#configuration-dictionary-schema">here</a>.
|
||||
See also the <a href="#log_config"><code>log_config</code> option</a> option for the main Synapse process.</p>
|
||||
<p>Example configuration:</p>
|
||||
<pre><code class="language-yaml">worker_log_config: /etc/matrix-synapse/generic-worker-log.yaml
|
||||
</code></pre>
|
||||
<hr />
|
||||
<h2 id="background-updates"><a class="header" href="#background-updates">Background Updates</a></h2>
|
||||
<p>Configuration settings related to background updates.</p>
|
||||
<hr />
|
||||
|
|
|
|||
|
|
@ -162,7 +162,7 @@ It should be named <code><SERVERNAME>.log.config</code> by default.</p>
|
|||
# Synapse also supports structured logging for machine readable logs which can
|
||||
# be ingested by ELK stacks. See [2] for details.
|
||||
#
|
||||
# [1]: https://docs.python.org/3.7/library/logging.config.html#configuration-dictionary-schema
|
||||
# [1]: https://docs.python.org/3/library/logging.config.html#configuration-dictionary-schema
|
||||
# [2]: https://matrix-org.github.io/synapse/latest/structured_logging.html
|
||||
|
||||
version: 1
|
||||
|
|
|
|||
|
|
@ -210,10 +210,12 @@ need its own configuration file and can take all of its configuration from the
|
|||
shared configuration file.</p>
|
||||
<h3 id="shared-configuration"><a class="header" href="#shared-configuration">Shared configuration</a></h3>
|
||||
<p>Normally, only a couple of changes are needed to make an existing configuration
|
||||
file suitable for use with workers. First, you need to enable an "HTTP replication
|
||||
listener" for the main process; and secondly, you need to enable redis-based
|
||||
replication. Optionally, a shared secret can be used to authenticate HTTP
|
||||
traffic between workers. For example:</p>
|
||||
file suitable for use with workers. First, you need to enable an
|
||||
<a href="usage/configuration/config_documentation.html#listeners">"HTTP replication listener"</a>
|
||||
for the main process; and secondly, you need to enable
|
||||
<a href="usage/configuration/config_documentation.html#redis">redis-based replication</a>.
|
||||
Optionally, a <a href="usage/configuration/config_documentation.html#worker_replication_secret">shared secret</a>
|
||||
can be used to authenticate HTTP traffic between workers. For example:</p>
|
||||
<pre><code class="language-yaml"># extend the existing `listeners` section. This defines the ports that the
|
||||
# main process will listen on.
|
||||
listeners:
|
||||
|
|
@ -230,23 +232,26 @@ worker_replication_secret: ""
|
|||
redis:
|
||||
enabled: true
|
||||
</code></pre>
|
||||
<p>See the <a href="usage/configuration/config_documentation.html">configuration manual</a> for the full documentation of each option.</p>
|
||||
<p>See the <a href="usage/configuration/config_documentation.html">configuration manual</a>
|
||||
for the full documentation of each option.</p>
|
||||
<p>Under <strong>no circumstances</strong> should the replication listener be exposed to the
|
||||
public internet; replication traffic is:</p>
|
||||
<ul>
|
||||
<li>always unencrypted</li>
|
||||
<li>unauthenticated, unless <code>worker_replication_secret</code> is configured</li>
|
||||
<li>unauthenticated, unless <a href="usage/configuration/config_documentation.html#worker_replication_secret"><code>worker_replication_secret</code></a>
|
||||
is configured</li>
|
||||
</ul>
|
||||
<h3 id="worker-configuration"><a class="header" href="#worker-configuration">Worker configuration</a></h3>
|
||||
<p>In the config file for each worker, you must specify:</p>
|
||||
<ul>
|
||||
<li>The type of worker (<code>worker_app</code>). The currently available worker applications are listed below.</li>
|
||||
<li>A unique name for the worker (<code>worker_name</code>).</li>
|
||||
<li>The type of worker (<a href="usage/configuration/config_documentation.html#worker_app"><code>worker_app</code></a>).
|
||||
The currently available worker applications are listed <a href="#available-worker-applications">below</a>.</li>
|
||||
<li>A unique name for the worker (<a href="usage/configuration/config_documentation.html#worker_name"><code>worker_name</code></a>).</li>
|
||||
<li>The HTTP replication endpoint that it should talk to on the main synapse process
|
||||
(<code>worker_replication_host</code> and <code>worker_replication_http_port</code>)</li>
|
||||
<li>If handling HTTP requests, a <code>worker_listeners</code> option with an <code>http</code>
|
||||
listener, in the same way as the <a href="usage/configuration/config_documentation.html#listeners"><code>listeners</code></a>
|
||||
option in the shared config.</li>
|
||||
(<a href="usage/configuration/config_documentation.html#worker_replication_host"><code>worker_replication_host</code></a> and
|
||||
<a href="usage/configuration/config_documentation.html#worker_replication_http_port"><code>worker_replication_http_port</code></a>).</li>
|
||||
<li>If handling HTTP requests, a <a href="usage/configuration/config_documentation.html#worker_listeners"><code>worker_listeners</code></a> option
|
||||
with an <code>http</code> listener.</li>
|
||||
<li>If handling the <code>^/_matrix/client/v3/keys/upload</code> endpoint, the HTTP URI for
|
||||
the main process (<code>worker_main_http_uri</code>).</li>
|
||||
</ul>
|
||||
|
|
@ -404,7 +409,8 @@ For multiple workers not handling the SSO endpoints properly, see
|
|||
<a href="https://github.com/matrix-org/synapse/issues/7530">#7530</a> and
|
||||
<a href="https://github.com/matrix-org/synapse/issues/9427">#9427</a>.</p>
|
||||
<p>Note that a <a href="usage/configuration/config_documentation.html#listeners">HTTP listener</a>
|
||||
with <code>client</code> and <code>federation</code> <code>resources</code> must be configured in the <code>worker_listeners</code>
|
||||
with <code>client</code> and <code>federation</code> <code>resources</code> must be configured in the
|
||||
<a href="usage/configuration/config_documentation.html#worker_listeners"><code>worker_listeners</code></a>
|
||||
option in the worker config.</p>
|
||||
<h4 id="load-balancing"><a class="header" href="#load-balancing">Load balancing</a></h4>
|
||||
<p>It is possible to run multiple instances of this worker app, with incoming requests
|
||||
|
|
@ -437,9 +443,10 @@ effects of bursts of events from that bridge on events sent by normal users.</p>
|
|||
of the main process to a particular worker.</p>
|
||||
<p>To enable this, the worker must have a
|
||||
<a href="usage/configuration/config_documentation.html#listeners">HTTP <code>replication</code> listener</a> configured,
|
||||
have a <code>worker_name</code> and be listed in the <code>instance_map</code> config. The same worker
|
||||
can handle multiple streams, but unless otherwise documented, each stream can only
|
||||
have a single writer.</p>
|
||||
have a <a href="usage/configuration/config_documentation.html#worker_name"><code>worker_name</code></a>
|
||||
and be listed in the <a href="usage/configuration/config_documentation.html#instance_map"><code>instance_map</code></a>
|
||||
config. The same worker can handle multiple streams, but unless otherwise documented,
|
||||
each stream can only have a single writer.</p>
|
||||
<p>For example, to move event persistence off to a dedicated worker, the shared
|
||||
configuration would include:</p>
|
||||
<pre><code class="language-yaml">instance_map:
|
||||
|
|
@ -479,9 +486,24 @@ worker_log_config: /etc/matrix-synapse/event-persister-log.yaml
|
|||
be routed to the workers handling that stream. See below for the currently supported
|
||||
streams and the endpoints associated with them:</p>
|
||||
<h5 id="the-events-stream"><a class="header" href="#the-events-stream">The <code>events</code> stream</a></h5>
|
||||
<p>The <code>events</code> stream experimentally supports having multiple writers, where work
|
||||
is sharded between them by room ID. Note that you <em>must</em> restart all worker
|
||||
instances when adding or removing event persisters. An example <code>stream_writers</code>
|
||||
<p>The <code>events</code> stream experimentally supports having multiple writer workers, where load
|
||||
is sharded between them by room ID. Each writer is called an <em>event persister</em>. They are
|
||||
responsible for</p>
|
||||
<ul>
|
||||
<li>receiving new events,</li>
|
||||
<li>linking them to those already in the room <a href="development/room-dag-concepts.html">DAG</a>,</li>
|
||||
<li>persisting them to the DB, and finally</li>
|
||||
<li>updating the events stream.</li>
|
||||
</ul>
|
||||
<p>Because load is sharded in this way, you <em>must</em> restart all worker instances when
|
||||
adding or removing event persisters.</p>
|
||||
<p>An <code>event_persister</code> should not be mistaken for an <code>event_creator</code>.
|
||||
An <code>event_creator</code> listens for requests from clients to create new events and does
|
||||
so. It will then pass those events over HTTP replication to any configured event
|
||||
persisters (or the main process if none are configured).</p>
|
||||
<p>Note that <code>event_creator</code>s and <code>event_persister</code>s are implemented using the same
|
||||
<a href="#synapse.app.generic_worker"><code>synapse.app.generic_worker</code></a>.</p>
|
||||
<p>An example <a href="usage/configuration/config_documentation.html#stream_writers"><code>stream_writers</code></a>
|
||||
configuration with multiple writers:</p>
|
||||
<pre><code class="language-yaml">stream_writers:
|
||||
events:
|
||||
|
|
@ -520,13 +542,15 @@ the stream writer for the <code>presence</code> stream:</p>
|
|||
worker. Background tasks are run periodically or started via replication. Exactly
|
||||
which tasks are configured to run depends on your Synapse configuration (e.g. if
|
||||
stats is enabled). This worker doesn't handle any REST endpoints itself.</p>
|
||||
<p>To enable this, the worker must have a <code>worker_name</code> and can be configured to run
|
||||
background tasks. For example, to move background tasks to a dedicated worker,
|
||||
the shared configuration would include:</p>
|
||||
<p>To enable this, the worker must have a unique
|
||||
<a href="usage/configuration/config_documentation.html#worker_name"><code>worker_name</code></a>
|
||||
and can be configured to run background tasks. For example, to move background tasks
|
||||
to a dedicated worker, the shared configuration would include:</p>
|
||||
<pre><code class="language-yaml">run_background_tasks_on: background_worker
|
||||
</code></pre>
|
||||
<p>You might also wish to investigate the <code>update_user_directory_from_worker</code> and
|
||||
<code>media_instance_running_background_jobs</code> settings.</p>
|
||||
<p>You might also wish to investigate the
|
||||
<a href="#updating-the-user-directory"><code>update_user_directory_from_worker</code></a> and
|
||||
<a href="#synapseappmedia_repository"><code>media_instance_running_background_jobs</code></a> settings.</p>
|
||||
<p>An example for a dedicated background worker instance:</p>
|
||||
<pre><code class="language-yaml">worker_app: synapse.app.generic_worker
|
||||
worker_name: background_worker
|
||||
|
|
@ -564,11 +588,15 @@ after setting this option in the shared configuration!</p>
|
|||
worker application type.</p>
|
||||
<h3 id="synapseapppusher"><a class="header" href="#synapseapppusher"><code>synapse.app.pusher</code></a></h3>
|
||||
<p>Handles sending push notifications to sygnal and email. Doesn't handle any
|
||||
REST endpoints itself, but you should set <code>start_pushers: False</code> in the
|
||||
REST endpoints itself, but you should set
|
||||
<a href="usage/configuration/config_documentation.html#start_pushers"><code>start_pushers: false</code></a> in the
|
||||
shared configuration file to stop the main synapse sending push notifications.</p>
|
||||
<p>To run multiple instances at once the <code>pusher_instances</code> option should list all
|
||||
pusher instances by their worker name, e.g.:</p>
|
||||
<pre><code class="language-yaml">pusher_instances:
|
||||
<p>To run multiple instances at once the
|
||||
<a href="usage/configuration/config_documentation.html#pusher_instances"><code>pusher_instances</code></a>
|
||||
option should list all pusher instances by their
|
||||
<a href="usage/configuration/config_documentation.html#worker_name"><code>worker_name</code></a>, e.g.:</p>
|
||||
<pre><code class="language-yaml">start_pushers: false
|
||||
pusher_instances:
|
||||
- pusher_worker1
|
||||
- pusher_worker2
|
||||
</code></pre>
|
||||
|
|
@ -591,13 +619,18 @@ shared configuration file to stop the main synapse sending appservice notificati
|
|||
<p>Note this worker cannot be load-balanced: only one instance should be active.</p>
|
||||
<h3 id="synapseappfederation_sender"><a class="header" href="#synapseappfederation_sender"><code>synapse.app.federation_sender</code></a></h3>
|
||||
<p>Handles sending federation traffic to other servers. Doesn't handle any
|
||||
REST endpoints itself, but you should set <code>send_federation: False</code> in the
|
||||
shared configuration file to stop the main synapse sending this traffic.</p>
|
||||
REST endpoints itself, but you should set
|
||||
<a href="usage/configuration/config_documentation.html#send_federation"><code>send_federation: false</code></a>
|
||||
in the shared configuration file to stop the main synapse sending this traffic.</p>
|
||||
<p>If running multiple federation senders then you must list each
|
||||
instance in the <code>federation_sender_instances</code> option by their <code>worker_name</code>.
|
||||
instance in the
|
||||
<a href="usage/configuration/config_documentation.html#federation_sender_instances"><code>federation_sender_instances</code></a>
|
||||
option by their
|
||||
<a href="usage/configuration/config_documentation.html#worker_name"><code>worker_name</code></a>.
|
||||
All instances must be stopped and started when adding or removing instances.
|
||||
For example:</p>
|
||||
<pre><code class="language-yaml">federation_sender_instances:
|
||||
<pre><code class="language-yaml">send_federation: false
|
||||
federation_sender_instances:
|
||||
- federation_sender1
|
||||
- federation_sender2
|
||||
</code></pre>
|
||||
|
|
@ -623,7 +656,9 @@ worker_log_config: /etc/matrix-synapse/federation-sender-log.yaml
|
|||
^/_synapse/admin/v1/quarantine_media/.*$
|
||||
^/_synapse/admin/v1/users/.*/media$
|
||||
</code></pre>
|
||||
<p>You should also set <code>enable_media_repo: False</code> in the shared configuration
|
||||
<p>You should also set
|
||||
<a href="usage/configuration/config_documentation.html#enable_media_repo"><code>enable_media_repo: False</code></a>
|
||||
in the shared configuration
|
||||
file to stop the main synapse running background jobs related to managing the
|
||||
media repository. Note that doing so will prevent the main process from being
|
||||
able to handle the above endpoints.</p>
|
||||
|
|
|
|||
Loading…
Reference in a new issue