blender-open-data/playbooks

Ansible playbooks located in this directory are used by Blender Foundation staff to manage installation and continuous deployment of this project.

While the playbooks can be used as reference for another production or staging installation (e.g. you can find all the required packages in install.yaml, templates of web server configuration under templates/ and variables such as domain names or paths where back-end code is located in vars_common.yaml), they will not provide you with a working installation if you run them "as is".

It should be possible, however, to adjust the playbooks by copying a directory under environments/ and adjusting variables in that directory. Refer to Ansible documentation for details about inventory variables.

Deployment playbooks

The target system is assumed to be Ubuntu 20.04 LTS. The playbooks have not been tested with other distros or releases, and will most likely fail due to differences in configuration paths and so on.

To avoid adding more dependencies to the project itself, ansible uses its own virtualenv. To set it up use the following commands:

virtualenv .venv -p python
source .venv/bin/activate
pip install -r requirements.txt

First time install

./ansible.sh -i environments/production install.yaml --vault-id production@prompt
./ansible.sh -i environments/production setup_certificate.yaml

Note that currently playbook install.yaml does not setup production-ready settings.py and config.py. These must be created after at the following path:

/opt/blender-open-data-<env>/website/opendata/settings.py
/opt/blender-open-data-<env>/launcher_authenticator/config.py

Encrypting variables

Let's say one of the config templates used by install.yaml refers to a variable named sentry_dsn, and for production we want this variable to have the following value: https://foo@bar.example.com/1234. To encrypt this value, use the following command:

echo -n 'https://foo@bar.example.com/1234' | ansible-vault encrypt_string --vault-id production@prompt --stdin-name 'sentry_dsn'

Store the ouput of the above command in environments/production/group_vars/all/99_vault.yaml (not tracked by this repository):

# environments/production/group_vars/all/99_vault.yaml
...
sentry_dsn: !vault |
      $ANSIBLE_VAULT;1.2;AES256;production
      foo5643bbar56563663265653430636530deadbeef65353534643361616238346264343763356362
      ..
      6439356237386bar303062393861626639613531326363380a653266646534383831666364663964
...

Any playbook that uses this variable will need to be able to decrypt it, so use --vault-id production@prompt: this will make Ansible prompt for a Vault password.

If a playbook you are running and its templates don't use any encrypted variables, --vault-id parameter doesn't need to be added to the command.

Deploy

Except for error page templates, which are part of the playbooks, the playbooks do not deploy local uncommitted changes. When you need to deploy something, make sure to commit and push your changes both to main and production:

1. commit and push your changes to main;
2. push the same exact changes to production using the following:

git fetch origin main:production && git push origin production

3. navigate to the playbooks and run deploy.yaml

./ansible.sh -i environments/production deploy.yaml

Periodic tasks

OpenData is using crontab for periodic tasks such as making daily snapshots.

To install or update these, use the following playbook:

./ansible.sh -i environments/production configure_crontab.yaml

To see current state of production crontab, use the following commands at the target server:

sudo -Hu opendata-production crontab -l