Messaging System

The AURA Web deployment comes with a messaging system based on NATS. Services in AURA produce and consume messages for different purposes.

Even though the REST APIs provided by AURA are the primary way of interacting with the system, messages can be used to complement certain use cases. The primary motivations for the messaging system in AURA are:

Reduce the number of poll-to-refresh calls to the REST API, especially for endpoints where clients can expect frequent changes or where they have to react to changes quickly.
Make it easier to inform clients of complex changes in the state of AURA that could only be derived through a series of REST API calls.

Nomenclature & Concepts

Some of the concepts described here have more general meanings or come with other expectations in other messaging systems. Most of them should be familiar to people who have worked with messaging systems before. However, some are specific to NATS, because that is the messaging system we are using or AURA itself, because we follow our own conventions whenever NATS is unopinionated.

Messages

Messages in AURA consist of a subject, a payload and metadata.

A subject is a string that identifies a type of message. They are often called topics in other messaging systems. With regard to NATS, subjects often also encode the most essential details of a message. Dots separate a subject into a hierarchy. An example for a subject in AURA is resource.steering.show.123.updated, which encodes its source, resource, resource id and effect on the resource. Their shape is not fixed and can vary depending on the context and only follows general syntactic rules like allowed characters and length.

The payload is just bytes. AURA specifically usually uses a UTF-8 encoded string containing JSON.

Metadata are some additional data fragments. AURA’s messages for instance usually all have a unique UUID.

Producers

A producer is a service that publishes messages. It does so because the message might be interesting to other services. In almost all cases it should not concern itself with the question of if or how other services actually react to the message sent by it. In that sense, messages are usually “fire and forget.”

Consumers

A consumer is a service that subscribes to messages. Consumers usually consume messages to react to them in some fashion. How they react to them is entirely up to them.

Compared to traditional subscribers in other messaging systems, consumers in NATS/AURA are usually stateful. That means that any service that consumes messages can crash or fail but is able to resume message processing at a later point in time. If and how this works is up to the consumer’s own configuration.

See the NATS documentation on consumers for more information.

Stream

A stream is a persistent sequence of messages stored in NATS. Streams have a name and usually store messages that have a similar shape or are related to each other.

Core Messaging Components

AURA Web provides two major components for the messaging system.

The NATS message broker, and
can, an HTTP SSE proxy for NATS.

NATS itself can be used to publish and subscribe to messages. Whenever possible, NATS should be used directly, as the ecosystem around NATS has mature client libraries for many languages that give extensive control of NATS’ feature set.

In contrast, can should be used if connecting directly to NATS is either impossible or non-desirable. The primary use case for can are browsers, which don’t support arbitrary TCP connections. can provides a streaming endpoint based on HTTP Server-Sent Events (SSE). SSE is supported in all major browsers and is fairly easy to consume with the help of the EventSource API. can is only forwarding messages from the time the client connects.

Message producers & consumers

All backend services in AURA can publish messages. However, not all services do nor are they required to do so. Some services will always be consumers, some services will always be producers, most will likely consume and produce messages, but often they will do one thing more than the other.

The messaging infrastructure is fairly new in AURA, and not all services that should be producers or consumers, or would profit from it have added support for it yet. A future AURA Web deployment will likely look like this, with regard to the messaging system:

        flowchart TB
    steering[steering]
    battery[battery]
    playout[playout]
    nats[NATS<br/>Message Broker]
    can[can<br/>HTTP SSE Proxy]
    dashboard[dashboard]
    dashboard-clock[dashboard-clock]

    steering <-->|pushes/receives messages| nats
    battery -->|pushes/receives messages| nats
    playout <-->|pushes/receives messages| nats

    nats -->|pushes messages| can

    can -->|pushes messages| dashboard
    can -->|pushes messages| dashboard-clock

Currently, only steering is producing messages and only the dashboard is consuming them.

Streams & Messages in AURA

Common message format

All messages in AURA share a similar message payload in the form of a UTF-8 encoded string containing a JSON object.

The JSON will look like this:

{ "version": 1, "createdAt": "2021-08-23T12:00:00.000Z", "data": ... }

With version being a version specifier for the message format and createdAt being the time the message was created by the producer (compared to the time the message was received by NATS or the consumer).

Streams

AURA currently features two streams of messages.

resource Stream

The resource stream emits messages that represent changes in the state of a specific resource like a show or an episode.

resource is a reference to the REST nomenclature for API endpoints, meaning that each message on the resource stream represents a change in the state of a specific entity on that endpoint.

Continuing with the example of a show, the following message subjects could be emitted by the resource stream:

resource.steering.show.123.created
resource.steering.show.123.updated
resource.steering.show.123.deleted

As you can see, the subject encodes:

the source of the message, or more accurately: the location of the resource (steering),
the resource type (show),
the resource id (123),
and the effect of the change on the resource (created, updated or deleted). and the resource type.

Identical events are emitted for almost all other resources as well.

The message payload will look like this:

{
  "version": 1,
  "createdAt": "2021-08-23T12:00:00.000Z",
  "data": {
    "id": 123,
    "url": "http://aura.local/steering/api/v1/shows/123/",
    "updatedAt": "2021-08-23T11:59:59.519Z"
  }
}

Note that updatedAt will only be present if that field is also available on the resource itself.

Clients can use these messages to invalidate local caches of resources.

program stream

The program stream emits messages that signal a change in the planned program of the radio station.

The following message subjects could be emitted by the program stream:

program.planned.date.2025-11-13.updated
program.planned.week.2025-W46.updated
program.planned.month.2025-11.updated
program.planned.upcoming.now.updated
program.planned.upcoming.24h.updated

These represent changes to the planned program in a specific timeframe: for a date, a week, a month, the current program or program of the next 24 hours. Here, a change in the planned program refers to either a change of the media or the scheduled times of the program.

A single change in the planned program in steering will always trigger more than one message because the change might have ripple effects throughout the program and because messages are sent or multiple relevant timeframes.

Each change to a timeslot in steering will produce at least three messages (date, week, month). It might produce more if the timeslot spans multiple days or if it affects the upcoming program.

Additionally, indirect changes to timeslots are also taken into account. If a timeslot inherits media from a schedule or a show, changes to these resources are reflected as well.

The message payload follows the common message format defined above, with data being undefined for date, week, and month messages. For upcoming messages, data will contain the program of the referenced timeframe in the format returned by the /program/playout/ endpoint.

Clients can use these messages to refresh calendar displays, change the actual played program, or display the upcoming program.

Development

NATS client libraries

The NATS ecosystem features client libraries for common languages. You can find a list of them in the NATS documentation.

Consumer

If you want to create a consumer, you need to decide if you can connect directly to NATS or if you need to use can. Whenever possible, use NATS directly.

With NATS client libraries

If you want to see an example of a consumer, can actually is a good example with its NatsManager class. The code describes the instantiation of a client and the logic to create a consumer as well as actually consuming messages.

Consumers in NATS are stateful, so that you can resume messages if at one point you stopped processing them.

Additionally, messages in NATS can be filtered by subject, with the dots creating a tree-like structure. Going with the resource stream as an example, you can filter all messages for shows by using the resource.steering.show.> subject.

See the NATS documentation on filtering for more information.

With can

The can README contains a complete example for a basic SSE subscriber.