In the first part we saw how it is possible to make real-time information available to a set of customers through the Pub/Sub concept.

In this part, we will see how Kuzzle solves some of Pub/Sub limitations with its new generation real time engine

This article is part of a series on real-time application development:

Automatic notifications from the database

To overcome the limitations of the Pub/Sub, we implemented our own real-time engine, Koncorde, and connected it directly to our database.

With Kuzzle, customers subscribe to database changes and not to communication channels.

Thus, it is possible to receive notifications when a document is created, modified or deleted without the need to create a dedicated communication channel.

When a customer wishes to subscribe to real-time notifications, he must provide the name of the index and the collection containing the documents. It will then be notified when changes (addition, modification, deletion) occur on this collection.

Example: real-time asset tracking on a map

Let's take an example to understand the difference with Pub/Sub.

Imagine a freight container tracking application, each container has an IoT GPS beacon connected to Kuzzle with MQTT and records its new position in Kuzzle every 10 seconds to keep a history.

On the operator's side, a web application allows the position of the containers to be visualized on a real-time map.

With Pub/Sub, the necessary actions are the following:

the web client subscribes to a predefined communication channel,
the IoT GPS beacon records its position in the backend,
the backend publishes the position in the communication channel,
the web client is notified of the new position

With Kuzzle the principle is essentially the same except that there is no need to manually publish the new position:

the web client subscribes to changes in the database (realtime:subscribe),
the IoT GPS tag records its position in Kuzzle (document:create),
the web client is notified of the new position

The use of the MQTT protocol for the IoT GPS beacon and the Javascript SDK on the web application side reduces the amount of code required on the client side, but above all there is no line of code on the server side!

The real-time provision of information is implemented only on the client side without the need of expensive development on the server.

Filter your notifications from client side

I mentioned the possibility for customers to fill in filters when subscribing to Kuzzle's real-time notifications.

These will allow us to filter finely the information received in real time by customers, always without intervening on the server side.

Example: track only one asset position

Let's take again our example of a container tracking application.

All containers record their new positions in the same collection by specifying their unique identifier.

On the web application side, I want to display the position of a single container.

With a filter, I will be able to tell Kuzzle that I only want to receive notifications about the container of my choice.

const filters = {
  equals: { containerId: 'fret-4221' }
};

await kuzzle.realtime.subscribe('containers', 'positions', filters, notification => {
  // Only notifications about the 'fret-4221' container
});

It is possible to receive only the information I really need and always without writing an additional line on the server side.

If you know GraphQL, it's the same principle. The client asks the server only for the notifications it needs

We have developed a large number of different filters and it is possible to filter everything you want by combining them with our different operators.

Real-time geofencing

We will talk about one type of filter in particular: geofencing filters.

This type of filter will allow us to define a specific geographical area and receive notifications when documents located by a GPS position enter or leave the area.

These filters can take different shapes, circles, rectangles or even the polygon of your choice.

Example: track only assets in a predefined area

Let us return to our container monitoring app.

Since each operator is responsible for a particular area, the web application must be able to show only the containers in that area.

Simply apply a geofencing filter corresponding to the chosen area and Kuzzle will only send the new positions of the containers inside that area.

const filters = {
  geoPolygon: {  
    location: {
      points: [
        [2.35107421875, 51.0275763378024],  
        [-4.89990234375, 48.472921272487824],
        [-1.6259765625, 43.48481212891603],
        [3.01025390625, 42.48830197960227],
        [7.62451171875, 43.77109381775651],
        [8.173828125, 48.980216985374994],
        [2.35107421875, 51.0275763378024]
      ]
    }
  }
};
  
await kuzzle.realtime.subscribe('containers', 'positions', filters, notification => {
  // Only notifications about containers in France
});

To define your polygons easily, you can use a site like http://geojson.io/.

With Kuzzle, it is very easy to develop real-time applications that include monitoring or alerting concepts based on geographical positions.

If you are interested in real-time geofencing, you can watch our Youtube channel with a video on real-time geomarketing.

What about performance?

As you may already know, here at Kuzzle we attach great importance to the performance of our products.

Our real-time engine, Koncorde, is also open-source and available on Github.

You will notice that particular attention has been paid to the whole code, in particular through specific optimizations for Node.js engine or by native modules compiled in C.

Benchmarks on real-time notification filters display very high performance with tens of thousands of operations per second, even on complex filters such as geofencing filters.

For example, on an i5-7300U CPU laptop @ 2.60GHz and 16GB of RAM, it is possible to raise up to 64,000 notifications/sec with 10,000 connected clients and a total of 100,000 different geofencing filters.

In the next article, we will discuss application performance and scaling issues involving a large number of real-time notification users.

See you next week!