Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added Proxmox Support #10

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
192 changes: 103 additions & 89 deletions matrix-synapse-riot/README.md
Original file line number Diff line number Diff line change
@@ -1,13 +1,14 @@
Matrix Synapse + Riot
===================
=====================

## Provision Matrix homeserver with Riot frontend

We will be using the following tools and services:

* [Digital Ocean](https://www.digitalocean.com) as the virtual machine provider
* [Digital Ocean](https://www.digitalocean.com) as the virtual machine and or DNS provider
* [Terraform](https://www.terraform.io) to provision the cloud servers
* [Let's Encrypt](https://letsencrypt.org) to get SSL certificates for HTTPS
* [Proxmox](https://www.proxmox.com) as self-hosted virtual machine option

The following steps assume you have a Digital Ocean account.

Expand All @@ -26,48 +27,103 @@ The following steps assume you have a Digital Ocean account.

1. Then store the domain name in your local environment:

echo -n YOUR_DOMAIN_NAME > .keys/domain_name
echo -n YOUR_DOMAIN_NAME > .keys/domain_name

1. Obtain a read-write access token from your Digital Ocean account's `API` tab, then store
it in your local environment:

echo -n YOUR_DIGITAL_OCEAN_ACCESS_TOKEN > .keys/do_token
echo -n YOUR_DIGITAL_OCEAN_ACCESS_TOKEN > .keys/do_token

1. Generate RSA keys to access your Digital Ocean VMs:
1. Generate RSA keys to access your Matrix VM:

ssh-keygen -t rsa -f .keys/id_rsa
ssh-keygen -t rsa -f .keys/id_rsa

Add the SSH key to your Digital Ocean account under `Settings > Security`, then copy the
## Digital Ocean Specific Steps
1. Rename main-digital-ocean.tf.txt to main.tf

mv main-digital-ocean.tf.txt main.tf

1. Add the SSH key to your Digital Ocean account under `Settings > Security`, then copy the
SSH fingerprint to your local environment:

echo -n YOUR_SSH_FINGERPRINT > .keys/ssh_fingerprint

1. [Download Terraform](https://www.terraform.io/intro/getting-started/install.html), add it to
your path. On Linux it would look something like this:

https://releases.hashicorp.com/terraform/0.11.10/terraform_0.11.10_linux_amd64.zip
unzip terraform_0.11.10_linux_amd64.zip
mv terraform /usr/bin
wget https://releases.hashicorp.com/terraform/0.11.13/terraform_0.11.13_linux_amd64.zip
unzip terraform_0.11.13_linux_amd64.zip
mv terraform /usr/local/bin

Then run initialization from our `terraform` working directory:

terraform init

## Proxmox Specific Steps
1. Follow Proxmox steps to set up Debian Cloud-Init image [here](https://pve.proxmox.com/wiki/Cloud-Init_Support)
Use Debian image from [here](https://cdimage.debian.org/cdimage/openstack/current-9/)
Name the template `Deb9Cloud-InitTemplate`

1. Rename main-digital-ocean.tf.txt to main.tf

mv main-proxmox.tf.txt main.tf

1. Set your host IP information

echo -n YOUR_IPV4_ADDRESS > .keys/interface_ip
echo -n YOUR_IPV4_GATEWAY > .keys/interface_gw
echo -n YOUR_IPV4_NETMASK > .keys/interface_ip_netmask (for example 24)
echo -n YOUR_IPV6_ADDRESS > .keys/interface_ip6
echo -n YOUR_IPV6_GATEWAY > .keys/interface_gw6
echo -n YOUR_IPV6_NETMASK > .keys/interface_ip6_netmask (for example 64)

1. Set your DNS resolver

echo -n YOUR_RESOLVER_IP_ADDRESS > .keys/nameserver

1. [Download Terraform](https://www.terraform.io/intro/getting-started/install.html), add it to
your path. On Linux it would look something like this:

wget https://releases.hashicorp.com/terraform/0.11.13/terraform_0.11.13_linux_amd64.zip
unzip terraform_0.11.13_linux_amd64.zip
mv terraform /usr/local/bin

Install the Proxmox plug-in

go get github.com/Telmate/terraform-provider-proxmox/cmd/terraform-provider-proxmox
go get github.com/Telmate/terraform-provider-proxmox/cmd/terraform-provisioner-proxmox
go install github.com/Telmate/terraform-provider-proxmox/cmd/terraform-provider-proxmox
go install github.com/Telmate/terraform-provider-proxmox/cmd/terraform-provisioner-proxmox
mkdir -p ~/.terraform.d/plugins
cp ~/go/bin/terraform-provider-proxmox ~/go/bin/terraform-provisioner-proxmox ~/.terraform.d/plugins/

To prevent timeout errors change TaskStatusCheckInterval to 60 seconds in `~/go/src/github.com/Telmate/proxmox-api-go/proxmox/client.go`




Then run initialization from our `terraform` working directory:

terraform init
terraform init

## Provision the server
1. Provision the server by running:

terraform apply
terraform apply

By default, this will set up the frontend to be accessible from the internet and cjdns
By default, this will set up the frontend to be accessible from the internet, cjdns, and yggdrasil

You may also choose to not install cjdns by changing the `cjdns` variable to `false`, for example:
You may also choose to not install cjdns and or Yggdrasil by changing their variables to `false`, for example:

terraform apply -var "cjdns=false"
terraform apply -var "cjdns=false" -var "yggdrasil=false"

1. From your browser, login to your Digital Ocean dashboard and find your new VMs tagged with
`matrix-synapse-riot`. When it is done you will see a temporary password. At your first login you will
1. From your browser, login to your dashboard and find your new VM.
When it is done you will see a temporary password. At your first login you will
be prompted to change your password. We recommmend that you do not delete your access token as it is
needed to renew Let's Encrypt certificates

1. Add peers to your cjdns and Yggdrasil config files.

## Maintaining and updating

For the below instructions we will be using the `tomesh.net` as an example. Please substitute `tomesh.net` with
Expand All @@ -77,19 +133,9 @@ the domain name you are setting up.

1. SSH into **matrix.tomesh.net**

1. Enter the `virtualenv` as the `synapse` user:

sudo -i -u synapse
cd ~/.synapse
source ./bin/activate

1. Stop the Synapse server with `synctl stop`
1. Update Synapse using Debian's apt command

1. Update with the following command where `VERSION` can be a branch like `master` or `develop`, or a release tag like `v0.34.0`, or a commit hash:

pip install --upgrade --process-dependency-links https://github.com/matrix-org/synapse/tarball/VERSION

1. Start the Synapse server again with `synctl start`
sudo apt update && sudo apt dist-upgrade -y

### Updating Riot Web client

Expand All @@ -99,54 +145,22 @@ the domain name you are setting up.

1. Download the pre-compiled [Riot Web release](https://github.com/vector-im/riot-web/releases):

wget https://github.com/vector-im/riot-web/releases/download/v0.17.8/riot-v0.17.8.tar.gz
wget https://github.com/vector-im/riot-web/releases/download/v1.3.0/riot-v1.3.0.tar.gz

1. Backup config file
cp /var/www/chat.tomesh.net/public/config.json /root/riot-config.json

1. Remove old Riot client:

rm -r /var/www/chat.tomesh.net/public/*

1. Extract **riot-v0.17.8.tar.gz** into **/var/www/chat.tomesh.net/public**:

tar xf riot-v0.17.8.tar.gz -C /var/www/chat.tomesh.net/public --strip-components 1

1. Create **config.json** in /var/www/chat.tomesh.net/public/ with the following lines, so it is used in place of the default **config.sample.json**:

{
"default_hs_url": "https://matrix.tomesh.net",
"default_is_url": "https://vector.im",
"disable_custom_urls": false,
"disable_guests": false,
"disable_login_language_selector": false,
"disable_3pid_login": false,
"brand": "Riot",
"integrations_ui_url": "https://scalar.vector.im/",
"integrations_rest_url": "https://scalar.vector.im/api",
"integrations_jitsi_widget_url": "https://scalar.vector.im/api/widgets/jitsi.html",
"bug_report_endpoint_url": "https://riot.im/bugreports/submit",
"features": {
"feature_groups": "labs",
"feature_pinning": "labs"
},
"default_federate": true,
"welcomePageUrl": "home.html",
"default_theme": "light",
"roomDirectory": {
"servers": [
"tomesh.net",
"matrix.org"
]
},
"welcomeUserId": "@riot-bot:matrix.org",
"piwik": {
"url": "https://piwik.riot.im/",
"whitelistedHSUrls": ["https://matrix.org"],
"whitelistedISUrls": ["https://vector.im", "https://matrix.org"],
"siteId": 1
},
"enable_presence_by_hs_url": {
"https://matrix.org": false
}
}
rm -r /var/www/chat.tomesh.net/public/*

1. Extract **riot-v1.3.0.tar.gz** into **/var/www/chat.tomesh.net/public**:

tar xf riot-v1.3.0.tar.gz -C /var/www/chat.tomesh.net/public --strip-components 1

1. Restore config file

cp /root/riot-config.json /var/www/chat.tomesh.net/public/config.json

1. Run `chown -R www-data:www-data /var/www/` to ensure that www-data have full access

Expand All @@ -158,42 +172,42 @@ the domain name you are setting up.

1. Run the query to make the user an admin replace USERNAME with the username of the user:

UPDATE users SET admin=1 WHERE name LIKE '@USERNAME:tomesh.net';
UPDATE users SET admin=1 WHERE name LIKE '@USERNAME:tomesh.net';

### Purging old posts and media files from one year ago
### Purging old posts and media files from one year ago to reclaim disk space

1. Login as an admin user at https://matrix.tomesh.net and copy your `Access token`
1. Login as an admin user at https://chat.tomesh.net and copy your `Access token`

1. SSH into **matrix.tomesh.net**

1. Switch to the synapse user `sudo -i -u synapse`

1. Enter the `.synapse` directory `cd ~/.synapse/`

1. Put your `Access token` into a variable called `access_token`:

access_token=ABCD1234...
access_token=ABCD1234...

1. Run the API call to purge old posts (e.g. `#tomesh:tomesh.net` channel with the `Internal room ID:` `!FsFLbKGMcUXEMBxZdu:tomesh.net`).
To purge another room, replace the ID with that room's ID:

curl -XPOST -d '{"delete_local_events": true, "purge_up_to_ts": '$(echo $(($(date --date="1 year ago" -u +%s%N)/1000000)))' }' 'http://localhost:8008/_matrix/client/r0/admin/purge_history/!FsFLbKGMcUXEMBxZdu:tomesh.net?access_token='$access_token
curl -XPOST -d '{"delete_local_events": true, "purge_up_to_ts": '$(echo $(($(date --date="1 year ago" -u +%s%N)/1000000)))' }' 'http://localhost:8008/_matrix/client/r0/admin/purge_history/!FsFLbKGMcUXEMBxZdu:tomesh.net?access_token='$access_token

1. Optionally you can remove all remote content by running:

curl -XPOST -d '{}' "http://localhost:8008/_matrix/client/r0/admin/purge_media_cache?before_ts=$(echo $(($(date -u +%s%N)/1000000)))&access_token=$access_token"`

1. Logout of the synapse user
curl -XPOST -d '{}' "http://localhost:8008/_matrix/client/r0/admin/purge_media_cache?before_ts=$(echo $(($(date -u +%s%N)/1000000)))&access_token=$access_token"

1. Switch to Postgres user `sudo -i -u postgres`

1. Load CLI and connect to Synapse database `psql -d synapse`

1. Run the command `VACUUM;`

1. Logout of the database and the Postgres user and return back to Synapse shell
1. Logout of the database and the Postgres user and return back to your shell

1. Switch to the root user `sudo -i`

1. Go into Synapse's media storage directory

cd /var/lib/matrix-synapse/media/local_content/

1. Delete old media files by running the following commands:

cd ~/.synapse/media_store/local_content
find * -mindepth 1 -mtime +365 -delete
cd /var/lib/matrix-synapse/media/local_content/
find * -mindepth 1 -mtime +365 -delete
Empty file.
Empty file.
Empty file.
Empty file.
1 change: 1 addition & 0 deletions matrix-synapse-riot/terraform/.keys/interface_ip6_netmask
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
64
1 change: 1 addition & 0 deletions matrix-synapse-riot/terraform/.keys/interface_ip_netmask
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
24
1 change: 1 addition & 0 deletions matrix-synapse-riot/terraform/.keys/ipv6_yggdrasil
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
null
1 change: 1 addition & 0 deletions matrix-synapse-riot/terraform/.keys/nameserver
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
1.1.1.1
Loading