Update and Maintenance

This guide helps to update and monitor your AURA installation.

Docker commands can be executed from any location. Docker Compose commands are to be called from the docker-compose.yml folder.

To learn about additional commands and arguments, consult the official Docker Compose command reference.

Upgrade to a new release

Update Containers

Before doing an update, check the Release Notes page, to learn what has been changed. For the case of an failed update, ensure you have backups of your data.

Follow these steps to update your services.

1. Switch to the new release

Pull the aura repository:

git pull

list available releases:

git tag
   v1.0.0
   v1.1.0

and check out the new release:

git checkout tags/<release-version>

Replace <release-version> with any of the releases shown by git tag.

2. Update the configuration

Compare your .env file with the (potentially changed) sample.env and change your .env file accordingly. Take special note of new versions of the services. Carefully check the release notes for changes you might need to take into account. For aura-playout, you also want to compare the config/services/sample-config/engine-core.yaml with your existing config/services/engine-core.yaml.

When the release notes inform you about new permissions, carefully follow the instructions and review how permissions can be re-added and updated manually.

3. Pull the new images from Docker Hub

docker compose pull

4. Re-create the containers

docker compose up -d

Logging and Monitoring

List running services

To get a list of services currently running use:

docker ps

Alternatively you can also run, giving additional details on the container composition:

docker compose ps

Logs

To continuously watch the logs of all services included in the current Docker Compose, type:

docker compose logs -f

To see the current logs of individual containers running with Docker Compose, execute:

docker compose logs -f --tail=100 <service-name>

This command prints the last hundred log lines, but also keeps watching for new entries.

If you prefer to access the logs directly, go to logs. Here you can find the log files for all services. Use tail to watch the log file:

tail -f -n100 <file>

The files in logs can be integrated to your favorite third-party monitoring software.

Prometheus Monitoring

In some future release, we will provide a native Prometheus Monitoring integration.

Sentry Monitoring

If you have a Sentry API compatible service running, you can send logs to it by setting the respective AURA_*_SENTRY_DSN environment variable in your .env file.

These services provide a Sentry integration:

Steering: AURA_STEERING_SENTRY_DSN
Battery: AURA_BATTERY_SENTRY_DSN
Can: AURA_CAN_SENTRY_DSN
Dashboard: AURA_DASHBOARD_SENTRY_DSN
Dashboard Clock: AURA_DASHBOARD_CLOCK_SENTRY_DSN

If you do not have a Sentry API compatible server running, you can get in touch with us to get access to our development Sentry service. This allows us to provide you with better support in case of issues.

Resetting, shutting down and cleaning up

Enforce creation of new containers

docker compose up -d --force-recreate

This recreates containers even if their configuration and image have not changed.

Stop and remove containers

If you wish to remove your deployment, all you need to do is shutting it down:

docker compose down

If you also wish to delete all data, including the Docker Volumes, you can run the following command:

docker compose down -v

Delete unused images

Since Docker does not automatically delete old images, they may pile up and occupy to much space. So you should delete them once in a while.

To delete images of containers not currently running, use:

docker system prune --all

This will not delete the Docker volumes. Volume are the place, where the databases and the persistent data lives.

The --all option also removes unused images, not just dangling ones.

Delete all data

If you want a complete fresh installation, you may need to delete all data, avoiding any conflicts.

Be aware this deletes all volume data, including the ones not part of the AURA installation.

docker system prune --all --volumes

Backup Strategy

Include following filesystem locations to your backup system. Some of these are only available in certain deployment bundles, such as aura-web, aura-playout or aura-recorder.

Filesystem location	Deployment bundle	Description
aura/config	all	All custom and sample configuration files
aura/log	all	The reported log files
aura/audio	all	The Audio Store, holding all the media files. The location may vary based on your specific configuration.

In addition to the above mentioned locations, data is also stored in Docker Volumes.

To get a list of all available Docker Volumes, run:

$ docker volume ls

local     aura-playout_aura_engine_socket
local     aura-playout_engine_api_db_data
local     aura-playout_pipewire_socket
local     aura-web_battery_data
local     aura-web_battery_db_data
local     aura-web_nats_data
local     aura-web_nginx_config
local     aura-web_steering_db_data
local     aura-web_steering_media
local     aura-web_steering_static
local     aura-web_tank_db_data

Every Docker Volume is prefixed with the specific deployment bundle. Below you find commands for backing up all Docker Volumes.

For Database specific backups you can use the description from Upgrade the database version

Backup & Restore Docker Volumes

To backup all AURA related Docker Volumes, commands for backup and restore are provided.

To create a backup run:

sudo make aura-volume.backup BACKUP_PATH=/your/backup/path

This will create a tar.gz archive for each volume added with a timestamp, to your desired path.

To restore a volume backup run:

sudo make aura-volume.backup BACKUP_FILE=/your/backup.tar.gz VOLUME_NAME=aura-steering

Avoid data corruption, ensure volume is not in use

Before running the restore procedure, make sure the specific volume is not in use, otherwise you may end up with corrupted data!

Use the provided backup and restore scripts

Do not simply copy the volume data under /var/lib/docker/volumes to another location. This can cause data corruption. Use the provided backup and restore scripts instead.

Special Operations

Log into a container

To bash into an already running container execute:

docker compose exec -it steering bash

Do not change container data

Avoid installing package inside the container or modifying files other than documented mounts. Otherwise you may end up with corrupted data!

Database container operations

To log into the database of a PostgreSQL container you can run:

docker compose exec steering-postgres psql -U steering

docker compose exec steering-postgres psql -U steering
select id, short_description, description from program_show;

Or you combine login and query in a one-liner:

echo 'select id, short_description, description from program_show;' | docker exec -i steering-postgres psql -h 127.0.0.1 -U steering

Change the way timeslots for schedules are created

By default, steering generates timeslots from the first date of a schedule, until either the last date of the schedule, the end of the year or defined number of days after the first date.

If you prefer to generate timeslots for a number of days after the first date of a schedule, instead of until the end of the year, you can change ADD_TIMESLOTS_UNTIL_END_OF_THE_CURRENT_YEAR and ADD_TIMESLOTS_FOR_DAYS_AFTER_THE_FIRST_DATE.

For example, setting the following environment variables will create timeslots for a schedule 365 days after the schedule’s first date:

ADD_TIMESLOTS_UNTIL_END_OF_THE_CURRENT_YEAR = 365
ADD_TIMESLOTS_FOR_DAYS_AFTER_THE_FIRST_DATE = False

Display group permissions differences

Every AURA Web release comes with a set of default groups permissions assigned.

Learn more about about the permission behaviour and the provided default group permissions in the roles & permissions chapter of the user guide.

The default group permissions are automatically added during the initialization step of AURA Web.

To see the differences between the current set and the default set of permissions, you can execute:

docker compose exec steering steering diffpermissions

If you haven’t changed any permissions after the initialization, you will see:

Group 'Programme Manager' is not missing any default permission
Group 'Programme Manager' has no extra permissions
Group 'Host' is not missing any default permission
Group 'Host' has no extra permissions
Group 'Host+' is not missing any default permission
Group 'Host+' has no extra permissions

But, for example, if you removed the “Program | episode | Can add episode” (add_episode) permission from the “Host” group and added the “Program | cba | Can add cba” (add_cba) permission to the “Host+” group, you will see:

Group 'Programme Manager' is not missing any default permission
Group 'Programme Manager' has no extra permissions
Group 'Host' is missing default permission(s): add_episode
Group 'Host' has no extra permissions
Group 'Host+' is not missing any default permission
Group 'Host+' has extra permission(s): add_cba

Load the default permissions to groups

You can load the default set of permission after updating from a previous release, by executing:

docker compose exec steering steering loaddefaultpermissions

The output will show something like "Installed 3 object(s) from 1 fixture(s)".

Double-check expected group permissions after loading

Keep in mind that any extra permissions you may have added to any group(s) will remain, while any missing permissions you’ve removed as part of your configuration will be added back.

Upgrade the database version

The postgres-version is saved in the POSTGRES_VERSION variable. When upgrading Postgres, it is not sufficient to change this version though. New major versions of Postgres cannot read the databases created by older major versions. The data has to be exported from a running instance of the old version and imported by the new version.

Thankfully, there is a Docker container available to automate this process. You can use the following snippet to upgrade your database in the volume aura-web_steering_db_data, keeping a backup of the old version in aura-web_steering_db_data_old:

export OLD=14
export NEW=16
export POSTGRES_USER="user"
export POSTGRES_PASSWORD="password"

# stop and remove the containers
docker compose down

# create a new volume for the upgraded data
docker volume create aura-web_steering_db_data_new

# upgrade the data volumes
docker run --rm \
  -e PGUSER="$POSTGRES_USER" \
  -e POSTGRES_INITDB_ARGS="-U $POSTGRES_USER" \
  -e POSTGRES_PASSWORD="$POSTGRES_PASSWORD" \
  -v aura-web_steering_db_data:/var/lib/postgresql/${OLD}/data \
  -v aura-web_steering_db_data_new:/var/lib/postgresql/${NEW}/data \
  tianon/postgres-upgrade:${OLD}-to-${NEW}

# append the access rule to the new data volume
docker run --rm -it \
  -v aura-web_steering_db_data_new:/data \
  alpine ash -c "echo 'host all all all scram-sha-256' | tee -a /data/pg_hba.conf"

# create a backup volume
docker volume create aura-web_steering_db_data_old

# copy the data from the current volume to the backup one
docker run --rm -it \
  -v aura-web_steering_db_data:/from \
  -v aura-web_steering_db_data_old:/to \
  alpine ash -c "cd /from ; cp -av . /to"

# copy the data from the new volume into the current one
docker run --rm -it \
  -v aura-web_steering_db_data_new:/from \
  -v aura-web_steering_db_data:/to \
  alpine ash -c "cd /from ; cp -av . /to"

# delete the new volume
docker volume rm aura-web_steering_db_data_new

You need to repeat the procedure for the other database volumes:

aura-web_battery_db_data
aura-playout_engine_api_db_data

Then, you can set POSTGRES_VERSION=16 in your .env file and restart the services, this should pull the new Postgres images if needed.

docker compose up -d

Overriding the docker-compose.yml

If you need to make changes to the docker-compose.yml you can create a docker-compose.override.yml and make the necessary adjustments in there. That way, your adjustments won’t create conflicts.

Deploy other Docker Images

If you prefer some individual deployment scenario, you can also run single Docker Images.

These Docker images are hosted on https://hub.docker.com/u/autoradio.

Work in progress

These images are not yet fully documented. We will update the documentation on Docker Hub as we move along. If you need such image please consult the documentation and Makefiles in the relevant repositories.