# Messaging System
The AURA Web deployment comes with a messaging system based on [NATS](https://nats.io/).
Services in AURA produce and consume messages for different purposes.
Even though the REST APIs provided by AURA are the primary way of
interacting with the system, messages can be used to complement certain
use cases. The primary motivations for the messaging system in AURA are:
- Reduce the number of poll-to-refresh calls to the REST API,
especially for endpoints where clients can expect frequent changes
or where they have to react to changes quickly.
- Make it easier to inform clients of complex changes in the state
of AURA that could only be derived through a series of REST API calls.
## Nomenclature & Concepts
Some of the concepts described here have more general meanings
or come with other expectations in other messaging systems.
Most of them should be familiar to people who have worked
with messaging systems before.
However, some are specific to NATS, because that is the
messaging system we are using or AURA itself, because we
follow our own conventions whenever NATS is unopinionated.
### Messages
Messages in AURA consist of a subject, a payload and metadata.
A subject is a string that identifies a type of message.
They are often called topics in other messaging systems.
With regard to NATS, subjects often also encode
the most essential details of a message.
Dots separate a subject into a hierarchy.
An example for a subject in AURA is `resource.steering.show.123.updated`,
which encodes its source, resource, resource id and effect on the resource.
Their shape is not fixed and can vary depending on the context and
only follows general syntactic rules like allowed characters and length.
The payload is just bytes.
AURA specifically usually uses a UTF-8 encoded string containing JSON.
Metadata are some additional data fragments.
AURA’s messages for instance usually all have a unique UUID.
### Producers
A producer is a service that publishes messages.
It does so because the message _might_ be interesting to other services.
In almost all cases it should not concern itself with the question of
if or how other services actually react to the message sent by it.
In that sense, messages are usually “fire and forget.”
### Consumers
A consumer is a service that subscribes to messages.
Consumers usually consume messages to react to them
in some fashion. How they react to them is entirely up to them.
Compared to traditional subscribers in other messaging systems,
consumers in NATS/AURA are usually stateful. That means that any
service that consumes messages can crash or fail but is able to
resume message processing at a later point in time.
If and how this works is up to the consumer’s own configuration.
See the [NATS documentation on consumers](https://docs.nats.io/nats-concepts/jetstream/consumers)
for more information.
### Stream
A stream is a persistent sequence of messages stored in NATS.
Streams have a name and usually store messages that have a
similar shape or are related to each other.
## Core Messaging Components
AURA Web provides two major components for the messaging system.
1. The NATS message broker, and
2. [_can_](https://gitlab.servus.at/aura/can), an HTTP SSE proxy for NATS.
NATS itself can be used to publish and subscribe to messages.
Whenever possible, NATS should be used directly, as the ecosystem
around NATS has mature client libraries for many languages that
give extensive control of NATS’ feature set.
In contrast, _can_ should be used if connecting directly to NATS
is either impossible or non-desirable. The primary use case
for _can_ are browsers, which don’t support arbitrary TCP connections.
_can_ provides a streaming endpoint based on
[HTTP Server-Sent Events (SSE)](https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events).
SSE is supported in all major browsers and is fairly easy to consume
with the help of the [EventSource](https://developer.mozilla.org/en-US/docs/Web/API/EventSource) API.
_can_ is only forwarding messages from the time the client connects.
## Message producers & consumers
All backend services in AURA can publish messages.
However, not all services do nor are they required to do so.
Some services will always be consumers,
some services will always be producers,
most will likely consume and produce messages,
but often they will do one thing more than the other.
The messaging infrastructure is fairly new in AURA, and not all
services that should be producers or consumers, or would profit from it
have added support for it yet.
A future AURA Web deployment will likely look like this,
with regard to the messaging system:
```mermaid
flowchart TB
steering[steering]
battery[battery]
playout[playout]
nats[NATS
Message Broker]
can[can
HTTP SSE Proxy]
dashboard[dashboard]
dashboard-clock[dashboard-clock]
steering <-->|pushes/receives messages| nats
battery -->|pushes/receives messages| nats
playout <-->|pushes/receives messages| nats
nats -->|pushes messages| can
can -->|pushes messages| dashboard
can -->|pushes messages| dashboard-clock
```
Currently, only steering is producing messages and only the
dashboard is consuming them.
## Streams & Messages in AURA
### Common message format
All messages in AURA share a similar message payload in the form
of a UTF-8 encoded string containing a JSON object.
The JSON will look like this:
```yaml
{ "version": 1, "createdAt": "2021-08-23T12:00:00.000Z", "data": ... }
```
With `version` being a version specifier for the message format and
`createdAt` being the time the message was created by the producer
(compared to the time the message was received by NATS or the consumer).
### Streams
AURA currently features two streams of messages.
#### _resource_ Stream
The `resource` stream emits messages that represent changes
in the state of a specific resource like a show or an episode.
`resource` is a reference to the REST nomenclature for API endpoints,
meaning that each message on the `resource` stream represents a change
in the state of a specific entity on that endpoint.
Continuing with the example of a show, the following message
subjects could be emitted by the `resource` stream:
- `resource.steering.show.123.created`
- `resource.steering.show.123.updated`
- `resource.steering.show.123.deleted`
As you can see, the subject encodes:
- the source of the message, or more accurately: the location of the resource (`steering`),
- the resource type (`show`),
- the resource id (`123`),
- and the effect of the change on the resource (`created`, `updated` or `deleted`).
and the resource type.
Identical events are emitted for almost all other resources as well.
The message payload will look like this:
```json
{
"version": 1,
"createdAt": "2021-08-23T12:00:00.000Z",
"data": {
"id": 123,
"url": "http://aura.local/steering/api/v1/shows/123/",
"updatedAt": "2021-08-23T11:59:59.519Z"
}
}
```
Note that `updatedAt` will only be present if that field is also
available on the resource itself.
Clients can use these messages to invalidate local caches of resources.
### _program_ stream
The `program` stream emits messages that signal a change in the
planned program of the radio station.
The following message subjects could be emitted by the `program` stream:
- `program.planned.date.2025-11-13.updated`
- `program.planned.week.2025-W46.updated`
- `program.planned.month.2025-11.updated`
- `program.planned.upcoming.now.updated`
- `program.planned.upcoming.24h.updated`
These represent changes to the planned program in a specific timeframe:
for a date, a week, a month, the current program or program of the
next 24 hours. Here, _a change in the planned program_ refers to
either a change of the media or the scheduled times of the program.
A single change in the planned program in steering will always trigger
more than one message because the change might have ripple effects throughout
the program and because messages are sent or multiple relevant timeframes.
Each change to a timeslot in steering will produce at least three messages
(date, week, month). It might produce more if the timeslot spans
multiple days or if it affects the upcoming program.
Additionally, indirect changes to timeslots are also taken into account.
If a timeslot inherits media from a schedule or a show, changes to these
resources are reflected as well.
The message payload follows the common message format defined above,
with `data` being undefined for `date`, `week`, and `month` messages.
For `upcoming` messages, `data` will contain the program
of the referenced timeframe in the format returned by the
[`/program/playout/` endpoint](https://api.aura.radio/steering/#/program/program_playout_list).
Clients can use these messages to refresh calendar displays,
change the actual played program, or display the upcoming program.
## Development
### NATS client libraries
The NATS ecosystem features client libraries for common languages.
You can find a list of them
[in the NATS documentation](https://docs.nats.io/using-nats/developer).
### Consumer
If you want to create a consumer, you need to decide if you can
connect directly to NATS or if you need to use _can_.
Whenever possible, use NATS directly.
#### With NATS client libraries
If you want to see an example of a consumer, _can_ actually
is a good example with its
[`NatsManager` class](https://gitlab.servus.at/aura/can/-/blob/d7e29b0c03b50375444f6d2b84a0c77ccefb57b8/can/utils.py#L20).
The code describes the instantiation of a client and the logic to
create a consumer as well as actually consuming messages.
Consumers in NATS are stateful, so that you can resume messages
if at one point you stopped processing them.
Additionally, messages in NATS can be filtered by subject, with
the dots creating a tree-like structure.
Going with the `resource` stream as an example, you can
filter all messages for shows by using the
`resource.steering.show.>` subject.
See the
[NATS documentation on filtering](https://docs.nats.io/nats-concepts/subjects#matching-a-single-token)
for more information.
#### With _can_
The [_can_ README](https://gitlab.servus.at/aura/can#usage) contains
a complete example for a basic SSE subscriber.