What's hot in Kuzzle : Equinox Release đŸ”„

Hello everyone!

We hope you had good holidays!

Here in Kuzzle the whole team took the opportunity to recharge their batteries and enjoy the sun.

However, we also have a lot of new features to introduce you !

Here is a summary of the new features on our different projects.

đŸ‡«đŸ‡· See this article in French https://blog.kuzzle.io/fr/equinox-release

Kuzzle 1.5

Version 1.5 is a new minor version of Kuzzle with two new features and many improvements.

Users, profiles and roles provisioning

It is now possible to define users, profiles and roles that will be loaded when Kuzzle starts.

The operation is the same as for loading mappings and data.

These users, profiles and rights must be defined in a JSON file with the following structure:

 

{
  "roles": {
	"role-id": {
  	/* role definition */
	}
  },
  "profiles": {
	"profile-id": {
  	/* profile definition */
	}
  },
  "users": {
	"user-id": {
  	/* user definition */
	}
  }

 

This file must then be passed as a parameter to the command line to start Kuzzle:

 

./bin/kuzzle start --securities /path/to/files.json

For more information, you can consult the associated documentation or for the more technical ones directly the pull request.

Plugins

Many new features on plugins with the integration of the SDK JS 6 in the plugin context, a redesign of the manifest file, node 8 usable by default and additional information in requests.

SDK JS 6 integration in the plugin context

Usually, developers should use context.accessors.execute to execute queries against Kuzzle API.

This low level method is more complicated because it requires instantiating a new query and filling in all the necessary fields manually.

After many discussions, we decided to expose the SDK JS 6 inside the plugins so that plugin developers could harness the power of the SDK.

Now plugin developers and SDK users will use the same syntax to develop with Kuzzle !

An example with the creation of a new document in Kuzzle:

 

const request = new Request({
  controller: 'document'
  action: 'create',
  index: 'my-index',
  collection: 'my-collection',
  body: {
	falut: 'world',
	foo: 42
  }
});

context.accessors.execute(request)

 

Now, with the SDK JS 6:

 

context.accessors.sdk.document.create('my-index', 'my-collection', null, { falut: 'world', foo: 42 });

 

Note that the SDK JS 6 is still in beta. Instructions for using it are available on Github.

It has been designed to follow exactly the Kuzzle API. Its documentation is being written, you can consult the controllers already documented here.

The core-plugin-boilerplate has also been updated with an example of using the SDK JS 6 in a controller action.

See the pull request on Github.

The Manifest file

Plugins must now have a manifest.json file in addition to the package.json file.

This file contains information about the plugin loaded by Kuzzle:

 - name : the name of the plugin

 - kuzzleVersion: Kuzzle versions supported in the format of a semver range

 

It is possible to see an example of a manifest file in the core-plugin-boilerplate.

The use of the package.json file to define the plugin name is deprecated and will be removed in Kuzzle v2.

See the documentation and the associated pull request.

Other changes

Node.js 8 for plugins

A new Docker image is available to plugin developers: kuzzleio/plugin-dev.

This new image contains Node.js 8 to allow you to exploit all the new features of the language, including the keywords async and await.

This is the new default image of the core-plugin-boilerplate that we recommend to use as the basis for developing a new plugin

Request context

The Request  object now contains information about the connection used.

We can find the type of protocol used request.context.connection.protocol or the ips addresses of the client request.context.connection.ips.

Each protocol can expose additional information in the request.context.connection.misc field. For example, the HTTP protocol exposes the url and headers of the original request.

See the pull request on Github.

Node.js 8 for Kuzzle

The official image of Kuzzle (kuzzleio/kuzzle) now uses Node.js 8 by default. 

Kuzzle support with Node.js 8 will end at the same time as Node's LTS support, i.e. in December 2019 (See https://github.com/nodejs/Release).

Node.js 6 will be supported in the same way until April 2019. 

MacOS

Kuzzle is officially supported on macOS!

Now macOS users can use the Kuzzle automatic installation script to start using it on their machine.

Just open your favorite terminal and type:

 

bash -c "$(curl http://get.kuzzle.io/)"

 

At Kuzzle we take the comfort of our developers very seriously and we do everything we can to ensure that our products are accessible and work well for everyone.

It is with this in mind that we have set up a series of automated tests to verify the proper functioning of our installation script with different operating systems. These tests are integrated into our continuous integration processes and are subject to rigorous monitoring.

For the most curious of you, you can check out how they work on Github

Koncorde

For those of you who don't know it yet, Koncorde is the real time engine developed especially for Kuzzle. It was designed from the beginning to offer unbeatable performance compared to existing open-source alternatives.

Today we are releasing version 1.2 of Koncorde which includes new features on matching keywords and a very important performance gain.

Exists and missing

The exists keywords and its opposite missing are now usable with a new simplified syntax. As a reminder, the keyword exists tests the existence of a property in a document and the keyword missing tests its absence.

The old syntax has of course been kept to ensure the backwards compatibility of Koncorde, however its use is deprecated and it will be deleted in the next major release.

Let's imagine that I want to subscribe in real time to documents containing a name property but not the nested driver.license property.

 

Example with the old syntax:

 

{
  “and”: [
	“exists”: { “field”: “name” },
	“missing”: { “field”: “driver.license” }
  ]
}

 

 And with the new one:

 

{
  “and”: [
	“exists”: “name”,
	“missing”: “driver.license”
  ]
}

 

The use of exists and missing has also been extended to array values. It is now possible to test the presence of a scalar (int, string, boolean or null) inside an array value property.

For example, to test the existence of a string in an array value property:

 

{
  “and”: [
	“exists”: “aliases[\”aschen\”]”  
  ]
}

 

You can also use this filter with nested properties:

 

{
  “and”: [
	“exists”: “school.ages[42]”  
  ]
}

 

You can consult the Koncorde documentation which has been entirely revised for the occasion or the associated pull request on Github.

Performance

During the implementation of this new feature, performance optimizations were realized on the core of Koncorde.

Historically Koncorde was written with Node.js 4, since then many optimizations have been made on performances and data structures. These performance gains are the result of the application of the new Node.js 8 standards, including the use of Map instead of standard javascript objects.

These modifications improve Koncorde's performance by an average of 4000%. It sounds huge, but it's the truth. This is a sample of the new performance benchmarks:

 

# Before

Filter count per tested keyword: 10000

> Benchmarking keyword: equals
  Registration: time = 0.484s, mem = +41MB
  Matching x 106,992 ops/sec ±2.46% (70 runs sampled)
  Filters removal: time = 0.02s

# After

Filter count per tested keyword: 10000

> Benchmarking keyword: equals
  Indexation: time = 0.435s, mem = +41MB
  Matching x 4,006,895 ops/sec ±0.35% (97 runs sampled)
  Filters removal: time = 0.02s

 

The most curious of you can check out the associated pull requests on Github: here and here.

Kuzzle Admin Console

The Admin Console is upgraded to version 2.2.0 with new features and a major code redesign!

New document views

The Admin Console is enhanced with two new views in addition to the list view.  

The first is a block view where each document is presented as a card.

 

image 1

 

The second is a map that is available when the collection contains documents with at least one geo_point property.

A geo_point is a special type of data supported by Elasticsearch, it represents a GPS coordinate and allows Kuzzle to do real time geofencing.

Each document is placed on the map according to its GPS coordinates. If the collection has several geo_points then it is possible to choose the one used by the map view.

 

image 2

 

To see the associated requests it's here and here.

Search filters

We have also worked hard on search filters to make them more user-friendly.

These filters are essential when you want to correctly view certain documents in collections that contain several tens of thousands of them.

Persistence of search filters across navigation

Filters are now saved by the Admin Console even when reloading a page or navigate through different collection. Each set of filters used with a collection will be saved.

They are present in the URL of your browser, which makes it easy to share or store them in the form of bookmarks, for example.

See the pull request on Github

Real time search filters

Most search engines offer real-time updating of results without the need to validate your search.

The Admin Console now offers the same functionality for simple search. This is done at the same time as the user inputs and filters the results of the collection in real time.

See the pull request on Github

Autocomplete properties names in complex filters

A collection can potentially contain several hundred different properties at several levels of nesting.

When using complex filters on certain properties, it is more convenient to have auto completion on the name of the property rather than having to remember their exact name.

 

 

For more information it's on Github.

Kuzzle SDKs

For several months now, we have been working on the release of our brand new SDKs.

These SDKs have been developed by sticking as much as possible to the architecture of our API.

SDKs documentation

We are rewriting the documentation to include all these new SDKs but it is a long and expensive process.

Indeed, always in a concern for quality, we have decided to test each example of code in our documentation individually in order to ensure that it always works properly.

You can see the reports of these tests in our CI: https://travis-ci.org/kuzzleio/documentation-V2/builds/431052204

We are also working on an article dedicated entirely to our process of testing code examples from our documentation to share our experience, stay tuned !

Javascript

I already told you about the SDK JS 6 which is used by plugin developers, it is currently in beta version.

To use it in a project, you can use the following command :

 

npm i https://github.com/kuzzleio/sdk-javascript#6-beta --save

 

The documentation is not finished yet but you can already start by reading the README on Github. You can also consult the documentation that is in the process at https://docs-v2.kuzzle.io/sdk-reference/kuzzle/constructor/.

For controllers that are not yet documented, you can simply rely on the Kuzzle API doc for the names of the methods and expected parameters.

PHP

The SDK PHP 4  is also available in beta version.

You can test it in your projects by installing it with the following command:

 

composer require kuzzleio/kuzzle-sdk:1-dev

 

Documentation is not yet available but as with the SDK JS 6 , the class names and methods are the same as the API controllers and actions.

Don't forget to report any bug encountered on Github, we are counting on you!

Go

The SDK GO is now officially available in version 1 on Github !

This SDK is the key to our SDK creation process in all languages. It then allows us, thanks to SWIG, to generate SDKs in the languages of our choice at a much lower cost.

An entire article will soon be dedicated to the functioning of SWIG and how we used it in Kuzzle to industrialize our SDK creation and maintenance process.

You can start using it in your Go projects with the following command:

 

go get github.com/kuzzleio/sdk-go

 

As with the other new SDKs, partial documentation is available here but the structure of the SDK remains the same as that of the API.

Wrapping things up

As you can see, at Kuzzle we like to ensure the quality and accessibility of our products for everyone. We do our best to provide quality tools to our customers and our community.

Once again, I would like to thank the entire Kuzzle team who give their best to fulfill their mission and provide a backend to accelerate the development of mobile, web and IoT applications.

If you have any technical questions about Kuzzle or any of our products, you can contact us on Discord or by email.

 

Alexandre Bouthinon

Related posts