thoughts/data/matrix.md

We have all been there during security operations. One of the parties involved in an incident or daily routine is not prepared for thinking they could be compromised.

Communications and information sharing is one of the fundamental things that you need to get right during a crisis.

As now-retired FBI director James Comey put it to 60 minutes [1]:

There are two kinds of big companies in the United States. There are those who've been hacked by the Chinese and those who don't know they've been hacked by the Chinese.

The following question always arises: How do we maintain operational security while still being able to communicate with all parties involved?

In practical terms this requires a communications platform to:

• Be independent of the service infrastructure
• Provide traceability
• Be resistant to resourceful threat actors
• Have simple and secure identity management
• Have cross-platform compability
• Provide file-sharing capabilities and ability to give the user an opportunity to express himself
• Support video and audio exchanges
• Be under the control of the team using it (the smallest circle of trust)
• Provide both end-to-end and transport layer encryption
• Disposable server infrastructure

This could have been a bit too much to ask for a couple of years ago, but today there are at least two alternatives satisfying the above requirements: Mattermost and the Matrix ecosystem. For the remainder of this post I will focus on how to establish an ad-hoc system with the tools provided by the Matrix project.

Setting Up An Out-of-Band Channel for Incident Handling with Matrix

Getting started takes three steps:

1. Establish a back-end server on Digital Ocean
2. Serve the Riot front-end website
3. Establish a recording capability with Matrix Recorder [2]

For the two first points, it is clever to use an approach that can be easily reproduced and that provides exactly the same, secure-by-default configuration each time. Due to this the preferred method in this case is to manage the VPS that can be established on anything with Debian or CentOS with Ansible. There is a script available on Github, known as matrix-docker-ansible-deploy [3]. The latter have also been endorsed by the Matrix project [4]. Both 1 and 2 can be accomplished with matrix-docker-ansible-deploy.

So let's get started.

Basic DNS-service

For this example I created a domain on namesilo.com and pointed that to (ns1|ns2|ns3).digitalocean.com. It would be ufortunate for the continuity of the service if a domain was taken offline or redirected somewhere, but due to the end to end encryption in Matrix it would not compromise the content of the conversations. Now that Digital Ocean has control of the primary domain, make sure to add the following before continuing:

Type	Hostname	          Value	                       TTL	
A	    <domain>               <ip>                        600 
A	    riot.<domain>          <ip>                        600 
A	    matrix.<domain>        <ip>                        600 
SRV	    _matrix._tcp.<domain>  10 0 8448 matrix.<domain>   600 

This can take some time to propagate, so make sure that the DNS-infrastructure is readily resolvable before you continue deploying the services.

Configure

Make sure to grab a copy of the current matrix-docker-ansible-deploy by running:

git clone https://github.com/spantaleev/matrix-docker-ansible-deploy.git

Create the following files:

inventory/host_vars/matrix.<domain>/vars.yml
inventory/hosts

vars.yml should look like this:

host_specific_matrix_ssl_support_email: <your-contact-email>
host_specific_hostname_identity: <domain>
matrix_coturn_turn_static_auth_secret: "<run pwgen -s 64 1>"
matrix_synapse_macaroon_secret_key: "<run pwgen -s 64 1>"

The Ansible hosts file should be formatted like the following:

all:
  children:
    matrix-servers:
      hosts:
        matrix.<domain>:
          ansible_user: root

Deploy and Execute

Now that your configuration files and server are ready, you can start deploying the Matrix Synapse server and start serving the Riot HTML/JS client.

First deploy the services (Riot and Matrix Synapse) by running:

ansible-playbook -i inventory/hosts setup.yml --tags=setup-main

When that completes successfully, you can start the services by:

ansible-playbook -i inventory/hosts setup.yml --tags=start

After starting the services, the Riot web interface is available on https://riot.<domain> where metadata is protected by a Let's Encrypt certificate.

The two primary endpoints you now have exposed to the WWW is:

• The Matrix API which runs at https://matrix.
• The Riot UI which runs at https://riot.

Going to https://riot.<domain> brings you to the Riot logon-screen

Adding Users

Registration is disabled by default on the server, so new users can be added by the following command:

ansible-playbook -i inventory/hosts setup.yml
                 --tags=register-user
	     --extra-vars='username=<first user>
	                   password=<some password>
			   admin=(yes|no)'

It is better to use pseudonyms on such a platform to make sure no information can be traced to a specific individual not involved in the case. Each user needs to verify his private key fingerprint with the other participants.

Vital Steps to Take as an Administrator

When using multiple servers, it is necessary to create an #control channel that is a fallback if a server hosting a room goes down.

Setup Matrix Recorder

To make sure that all communications is stored for traceability make sure to install the Matrix Recorded (MR). MR should be installed locally and not on the Matrix server.

git clone https://gitlab.com/argit/matrix-recorder.git
cd matrix-recorder/
npm install

To execute the recorder, run the following. The first time you will be asked to enter the login credentials of the user.

$ node matrix-recorder.js <case-folder>
Loading olm...
Your homeserver (give full URL): https://matrix.<domain>
Your username at the homeserver: <username>
Your password at the homeserver: <password>
No of items to retrieve for initial sync: 1000
[...]

View messages as HTML by running the Matrix Recorder conversion script:

node recorder-to-html.js <case-folder>

Controlling Logins

Access monitoring can be done in the console by e.g. tail -f /matrix/synapse/run/homeserver.log.

The Power of Disposability

At some point you have finished the information exchange. The beauty of this setup is that is can now be safely deleted from the Digital Ocean droplet console.

[1] James Comey and 60 minutes: https://www.cbsnews.com/news/fbi-director-james-comey-on-threat-of-isis-cybercrime/

[2] Matrix Recorder: https://matrix.org/docs/projects/other/matrix-recorder.html

[3] matrix-docker-ansible-deploy: https://github.com/spantaleev/matrix-docker-ansible-deploy

[4] Matrix project endorsement: https://matrix.org/blog/2018/06/01/this-week-in-matrix-2018-06-01/