As you know, Kuzzle is an open source product whose entire code is freely available on Github.
To continue this open approach, we have decided to make our working methods "open source" so that everyone can understand the processes we use every day to move our projects forward.
Throughout this article we will follow the steps of developing a real feature!
There is no shortage of ideas. In order not to overload Kuzzle with features that are rarely or never used, each idea must be accompanied by a concrete use case.
Everyone is free to offer a feature, whether it is a team member, a customer or the community.
It all starts with a request from one of our customers for Slack
that wanted a direct access to ES through Kuzzle.
All these ideas are then added to the backlog of our Product Workshop.
Study the need
Once every two weeks, the team meets to discuss the ideas submitted to the product workshop for 1 hour.
We use Trello to list ideas and a script to randomly draw a card.
Proposal of the feature in our Product Workshop.
The person who submitted the idea must then explain the potential functioning of the feature and the use case related to it.
The purpose of this workshop is to ensure two things: Utility and Feasibility.
- If the feature meets these two criteria, the card is moved to the "Processed" column and a User Story is added to the Product Backlog in Jira.
- Otherwise the card is moved to the "Won't Do" column.
A feature refused because of its non-feasibility in a cluster environment.
Archiving is important in order to keep track of which features have been approved or not!
The third step in the life of a feature is its prioritization during Backlog Refinement.
Every two weeks, we analyze our Kanban board in Jira to prioritize the tasks to be performed in the next two weeks.
View of the Product Backlog. On the right, the task accepted from the Product Workshop
This prioritization is carried out in a collegial manner, in consultation with the entire team.
We take into account the needs of our customers but also those of the community through our public roadmap.
The tasks thus prioritized are moved to the "To do" column.
Develop the feature
Once a task has been prioritized in the "To Do" column, everyone is free to assign it to themselves to complete it.
This is part of our team's self-management approach, where everyone organizes their work as they wish, according to the needs around the product.
Create a feature branch
We use Gitflow for all our developments, which means that each feature is developed on a new branch created from the current development branch.
Architecture of a Git repository with Gitflow
Test Driven Development
The first step is to write the functional tests using Cucumber's Gherkin syntax.
Thus it will be possible to test the functioning of the code during the development phase without having to perform the actions by hand or via a "disposable" script”.
Write the code and the unit tests
Then, it is the writing of the code of the feature itself.
Special care must be taken when processing user input to prevent security breaches and also when modifying critical code areas that can potentially be executed several hundred times per second.
Finally, unit tests must be written to ensure maximum future maintainability.
Write the documentation
The documentation for each project is located within the repository and must therefore be updated at the same time as the feature is added.
If the new documentation includes code examples, they should also be tested with our documentation testing framework.
In addition to the ease of reviewing a pull request including code and documentation, this allows us to ensure that the deployment of a feature is always accompanied by the associated documentation.
Pull Request with Continuous Integration & Deployment
Once all the necessary work has been done, a pull request is created.
Pull Request of the feature on the Kuzzle repository.
It must follow the standard format to facilitate its review by team members.
Each pull request must be approved by 3 team members to ensure the best possible quality of the new product code.
Finally, each step of our CI must be validated in order for the pull request to be definitively validated.
Our CI works with Travis and includes the following steps for pull requests:
- Build and deploy documentation on Netlify
- Code quality analysis with ESlint, SonarQube and LGTM
- Unit tests and code coverage measurement with Mocha and Codecov
- Functional tests with Cucumber
Once the CI has passed and the team has approved, the pull request is merged into the development branch with a squash merge, as for description a summary of the feature which is used for the Git history.
Deploy to pre-production
The features delivered on the development branch are then deployed in pre-production.
This automatic deployment includes the following steps:
- Build and push images on Dockerhub (kuzzleio/kuzzle:2-dev and kuzzleio/plugin-dev:2-dev)
- Deployment of documentation on https://next-docs.kuzzle.io/
Release the new version
We regularly prepare the delivery of a new version including the latest features developed.
To do this, we use an internal tool that will automatically perform the different steps:
- Generation of the changelog of the new version
- Creation of a pull request from the development branch to the stable branch
- Creating a tag with the new version and the associated Github release
Changelog of the release including the new feature.
Then, as during the pre-production deployment, our CI will take care of deploying the production images on Dockerhub as well as the documentation and NPM packages.
What changes with Open Source?
You can tell yourself that these steps are the standard steps in the development life cycle of a feature and you're right!
What changes with Open Source is above all the way these steps are carried out by involving the community in each of them.
You can then help us prioritize features by voting on our public roadmap.
Finally, everyone is free to come and contribute or audit the development phases on Github.