What's new

Kaa 1.4

Find below high-level descriptions of some of the major release highlights.

Other highlights of Kaa 1.4

  • [DCX] supports handling the plain text data like 21, 55%, 2061m, etc. on /plain/${metric-name} resource path. See details on plain data handling overview.

Kaa 1.3 (February 17-th, 2021)

Find below high-level descriptions of some of the major release highlights.

Branding

Kaa 1.3 supports branding customization for the platform and per tenant. branding overview It is possible to set up a company’s unique color scheme, add logo and icons, and change styling to match the branded palette. The following customization options are now available:

  • Company name
  • Logo logo and company name configuration
  • Favicon favicon configuration
  • Login page styling as well as theme colors used within the application login page configuration
  • Color schema login page configuration

File management

Kaa 1.3 supports the sharing of uploaded or downloaded files with other users and devices. These file management capabilities are enabled by the open-source storage called MinIO. All binary and file blobs used in WD configuration, such as branding, dashboards, and widgets, are stored in a bucket named by the tenant id. Each tenant has two buckets, one of which is configured to allow public read access. Minio UI and its file management functions can be accessed from the WD admin sidebar, which features a direct link.

Minio UI

Disaster recovery plan

Kaa 1.3 introduces a Disaster Recovery Plan (DRP) by implementing backup and restore procedures. By default, Kaa automatically backs up itself on a daily basis and uploads snapshots to an AWS S3 bucket related to the particular deployment. Using the snapshots it is possible to restore the platform to a specific state in time.

Find more info on how Disaster Recovery Plan works here.

Migration to Helm 3

In the Kaa 1.3 release, all Kaa services were migrated to Helm 3. You can use the improvements and features of this version, such as support of library charts, the improved Upgrade strategy, validation of chart values with JSONSchema and others. More information about Helm 3 features and changes you can find at changes since Helm 2.

Solution templates

Kaa 1.3 provides 3 templates with preconfigured services, configurations and dashboards to provision a fully featured, end-to-end solution. Each template provides a simulator example that describes main protocols to communicate with the platform features, such as data collection, metadata management, command execution, and device configuration.

Run solution

Request Status Extension

Kaa 1.3 has a new service called Request Status Extension that allows connected clients to retrieve the status of 1/KP-based message by its Request ID. Suppose the client published telemetry data with the request ID 67 and immediately restarted, thus having no time to handle the platform’s response for the message with that request ID. After the restart, the client should somehow know whether it should republish the telemetry data or not. It can be done by requesting the last known status code and reason phrase for the given request ID 67 using the extension protocol implemented by RSX.

Find more info on how to use RSX here.

Other highlights of Kaa 1.3

  • [CMX] In previous Kaa versions, CMX would send configuration applied messages to the configuration repository, e.g., ECR, regardless of whether the config was applied by an endpoint or not. Starting with Kaa 1.3, CMX no longer automatically sends configuration applied messages. Instead, CMX sends only those apply messages that were explicitly initiated by an endpoint.
  • [WD] added form control to upload files.
  • [WD] added unit converting fields for Label, Single Number widgets.
  • [WD] responsive layout for mobile and tablet devices.
  • [WD] added Single number digital view.
  • [WD] fixed security vulnerabilities in Raw HTML widget, Dashboard titles and description fields.
  • [WD] In Kaa 1.3, platform administrators can configure announcement and maintenance banners directly from UI. A maintenance banner can be used to notify other users about planned maintenance. It cannot be closed by the user, so everyone will know about the upcoming downtime in advance. A marketing banner can be used for commercial announcements, discounts, or any other marketing campaigns. KaaIoT Cloud users can enable advanced security configuration that allows to change all tenant specific properties.
  • [Data analytics] In [[Open Distro for Elasticsearch]] added automatic index pattern creation for [Kibana][open distro kibana.
  • [EPTS] Deprecated REST API for updating endpoint time series data under an application is dropped. Using the application version-specific API instead is recommended going forward.
  • [EPTS] now permits time series names to contain upper-case Latin letters, making the effectively allowed charset as follows: Latin letters (a-z, A-Z), digits (0-9), dashes (-) and underscores (_). Time series value names must not include any of: backslash (\), circumflex accent (^), dollar sign ($), single quotation mark ('), double quotation mark ("), equal sign (=), or comma (,). Also, names time, kaaEndpointID, kaaEmpty, _field, and _measurement are reserved and cannot be used.
  • [EPTS] Data sample keys eligible for auto-extraction are now limited to Latin letters (a-z, A-Z), digits (0-9), underscores (_), and dashes (-) to conform to the existing restrictions on EPTS time series names.
  • [DCX] Support processing non-batched, individual data samples in addition to batches defined in 2/DCP. Now an endpoint can upload a single, non-batched data sample as a JSON record. This is useful for devices that do not support data batching and can only send single data sample JSON record per data collection message to the platform.
  • [DCX] Processes messages with an extension-specific resource path that starts with /json without requiring the full path match as defined in the 2/DCP. As a result, the devices that automatically append random values to the MQTT topic can communicate with DCX without restrictions. E.g. the resource path /json/random_string is handled as the /json. See the application configuration for details.
  • KPC now supports MQTT over TLS 1.3.
  • BCX supports retrieving the most recent binary data blob payload using the REST API.
  • BCX supports deleting binary data blob by its ID using the REST API.
  • BCX data persistence interface configuration parameter kaa.postgresql.url is no longer supported. See data persistence interface configuration and Environment variables as well for the new parameters.
  • CCM data persistence interface configuration parameter kaa.postgresql.url is no longer supported. See data persistence interface configuration and Environment variables as well for the new parameters.
  • [CEX] supports deleting the command using the REST API.
  • [OTAO] Now endpoint can get software specification on software creation or update without making explicit requests.
  • [Helmfile] Starting with Kaa 1.3, the platform is deployed by Helmfile. With Helmfile each Kaa component has its own Helm release lifecycle. So if you want to upgrade or downgrade a specific Kaa component, you don’t need to upgrade or downgrade the whole platform but instead only this specific component. Note, that the platform does not have a Kaa Helm meta-chart anymore.

Kaa 1.2-mr1 (October 2-nd, 2020)

Kaa 1.2-mr1 is a maintenance release for Kaa 1.2 with the following changes:

  • [Bug fix] Incomplete TLS handshake prevents KPC from accepting other MQTT/TLS connections. The condition occurs when an outstanding client TLS handshake hangs prior to completion, and clears when the handshake times out.

Kaa 1.2 (July 6-th, 2020)

Find below high-level descriptions of some of the major release highlights.

Multi-tenancy

Kaa 1.2 implements advanced multi-tenancy so that every platform tenant gets a dedicated authentication and authorization KeyCloak realm. As a result, each tenant has a fully isolated space and can manage their own:

  • users
  • permissions
  • applications and their versions
  • endpoints and their related data
  • client credentials
  • solutions
  • dashboards, etc.

Tenants are also able to configure their own external Identity Providers (IDPs: e.g. corporate LDAP, Active Directory, various OAuth or SAML providers).

To learn more about multi-tenancy, see the corresponding documentation.

Client credentials management

Client Credentials Management service (CCM) is a new Kaa service for the client authentication, which takes over these responsibilities from the Credentials Management service (CM). CCM supports authentication using basic credentials, such as username/password, and SSL/TLS certificates, based on X.509 technology.

Basic authentication is currently supported by all MQTT-based transports available in the Kaa Protocol Communication service (KPC): plain MQTT, MQTT/TLS, MQTT/WebSocket. X.509 authentication is supported by the MQTT/TLS KPC transport.

Both basic and X.509 authentication are now enforcable on a per-tenant basic, separately for each compatible transport. You can toggle between them in your tenant right from the Kaa Web Dashboard interface.

Basic auth toggle on plain MQTT transport

Kaa WD now also provides interface for managing basic and X.509 credentials. Credentials of both types are a propery of a given tenant and can be used to connect clients to exchange data on behalf of any endpoints that belong to applications of that tenant.

Basic credentials management.

create basic credentials

X.509 credentials management.

create x509 credentials

HTTP transport support

Kaa Protocol Communication service (KPC) now supports HTTP transport that implements 1/KP protocol over plain HTTP. Unlike 1/KP over MQTT, HTTP binding is synchronous: it follows the request-response communication pattern and does not support server message push.

Learn more about integrating clients using the HTTP transport here.

Binary data collection

Kaa 1.2 supports collection of binary data blobs (still images, video segments, audio recordings, etc.) from connected devices. This is enabled by a new microservice: Binary data Collection Extension (BCX). The supported data storage backend is AWS S3.

To upload a binary data blob, client must first retrieve a temporary authorization token on behalf of an endpoint from the BCX service using an existing communication channel (MQTT- or HTTP-based). Once in possession of a temporary token, client may upload binary data blobs related to that endpoint using the RESTful data upload API. BCX also provides REST API for managing and accessing already uploaded binary blobs.

The submitted binary data blobs can be viewed and downloaded using the corresponding Kaa Web Dashboard (WD) widgets:

Binary data blobs in Kaa WD

Endpoint configuration schema management

In Kaa 1.0 and 1.1, endpoint configuration was a free-form JSON document. With Kaa 1.2 we introduce an ability to configure endpoint configuration schema in the Endpoint Configuration Registry service (ECR). The endpoint configuration schemas are associated with Kaa applications and appversions. The appversion-specific schema takes precedence over the corresponding application-specific schema.

When schema validation is enabled in ECR, it rejects provisioning endpoint configs that do not match the expected schema.

Endpoint configuration schemas can be configured for ECR in the Kaa Web Dashboard, either in the schema view:

Endpoint configuration schema management schema view

or in the JSON view:

Endpoint configuration schema management JSON view

Data samples enrichment with endpoint metadata

Kaa Data Collection Extension service (DCX) now allows enriching data samples received from connected endpoints with their metadata attributes. When this feature is enabled in the DCX configuration, it appends endpoint metadata key-value pairs to each data sample events using the specified path (~ep-metadata by default). Doing so makes it possible to feed downstream data processing services, such as EPTS or KDCA with additional endpoint-related state information. Note that only data samples that are JSON objects can be enriched with endpoint metadata. The data samples enrichment is disabled by default for backward compatibility.

See the DCX documentation for more details.

Data analytics

Kaa 1.2 is now pre-integrated out of the box with the Open Distro for Elasticsearch. Each tenant’s data is isolated in Elasticsearch and Kibana, and the security access policies are seamlessly integrated with Kaa. This integration enables various IoT data analytics functionality, including collection, analysis, querying and visualizing device data.

Data analytics

Flexible triggers and alerts can be configured to send notifications to preferred destinations.

Trigger and alerts

Find out more about the data analytics in Kaa here.

Other highlights of Kaa 1.2

  • [TEKTON] Tekton now restricts the application version name suffix to match ^[a-z0-9]+$ regex pattern when you create a new application version using the REST API. In addition, application names will be automatically generated for newly created applications when kaa.tekton.app-names.auto-generation.enabled configuration variable is set to true. It is recommended that you enable the auto-generation to prevent the possible application name conflicts.
  • [TEKTON] Tekton now supports bulk REST API for Bulk operations on tenants and their applications.
  • [CEX] commandRetentionTtl time unit changed in REST API from hours to milliseconds.
  • [CEX] commandRetentionTtl renamed in REST API to commandTtl.
  • [EPTS] EPTS now supports updating time series data for specified endpoints under application version in its REST API. Just like with DSTP and TSTP interfaces, the data points published to this API yield time series events on the TSTP interface.
  • [EPTS] REST API for updating endpoint time series data under an application is deprecated and will be dropped in the next release. Using the application version-specific API instead is recommended going forward.
  • [EPTS] EPTS now supports defining which of the fromDate and toDate are inclusive when retrieving historical time series data.
  • [EPTS] EPTS now supports data points filtering using the beforeDate query parameter in its REST API.
  • [EPR] In previous Kaa versions, EPR provided endpoint metadata and endpoint filter management only via its REST API. Now, in addition to REST API it is possible to manage endpoint metadata and endpoint filters via NATS using the 19/EPMMP and 20/EFMP protocols. These interfaces improve overall performance and give more flexibility for platform expansion and customization.
  • [CEX] now supports getting the list of existing command resources per endpoint or application name via the REST API.
  • [CEX] now uses PostgresSQL database instead of Redis.
  • [ECR] configuration response format changed in default config API and per-endpoint config API.
  • [ECR] /endpoints/{endpointId}/app-versions/{appVersionName}/current REST endpoint removed. Use [per-endpoint config API][CEX REST API GET per-endpoint config] instead.
  • [RCI] service is now fully deprecated and removed. CEX service provides a super-set of the original RCI functionality.
  • [KDCA] now adds tenant ID and the endpoint application version fields to Kafka events.
  • [CMX] now supports configuration applied messages from endpoints per 7/CMP.
  • [WD] JSON schema editor integrated with Configuration form, Endpoint list, Software OTA related widgets.
  • [WD] New widgets added: “Device orientation” and “Luminance”.
  • [WD] Added dynamic variables for “Gauge” widgets. It is possible to populate values from endpoint or application version configuration to scale, max, min and threshold fields.
  • [WD] “Multi series chart” now can be configured to show mixed line styles like bar, step and line. It is now possible to show data from multiple endpoints.
  • [WD] now has a pre-configured device management page for manage endpoint metadata, commands, data blobs, etc.
  • [WD] now has a “User management” link that redirects to the KeyCloak user administration page.
  • [WD] now has a “Help” page.
  • [WD] various UX improvements, performance optimizations, and bugfixes.
  • [Bug fix] KPC sets SubAck MQTT packet QoS.
  • [Bug fix] KDCA does not recover after losing a Kafka connection.
  • [Bug fix] Java services don’t fetch Tekton configs at boot time.

Kaa 1.1-mr3 (November 17-th, 2020)

Kaa 1.1-mr3 is a maintenance release for Kaa 1.1 with the following changes:

  • [Bug fix] commandRetentionTtl zero value in REST API was not supported. Now commands with zero commandRetentionTtl are pushed to the device only once. Also, commandRetentionTtl can be configured with kaa.cex.commands.command-retention-ttl-hours service configuration property.

Kaa 1.1-mr2 (October 2-nd, 2020)

Kaa 1.1-mr2 is a maintenance release for Kaa 1.1 with the following changes:

  • [Bug fix] Incomplete TLS handshake prevents KPC from accepting other MQTT/TLS connections. The condition occurs when an outstanding client TLS handshake hangs prior to completion, and clears when the handshake times out.

Kaa 1.1-mr1 (March 25-th, 2020)

Kaa 1.1-mr1 is a maintenance release for Kaa 1.1. In scope of this release the following changes were made:

  • [Bug fix] Creating a new application or application version overwrites configuration of an existing one. Reproducible with Java-based services when the new name is a sub-string of an existing one.
  • [Bug fix] The WD permits creating endpoints with an empty token.
  • [Bug fix] The WD does not render variables in dashboard titles.
  • [Bug fix] Wrong WD documentation link anchors.

Known issues in Kaa 1.1-mr1

Application and application version names conflict in Java-based services

Affected Kaa versions: 1.1-mr1, 1.1, and 1.0.

Issue summary: Application or application version names that differ only by dashes (-) or underscores (_) cause naming conflict in Java-based services.

Consider an example of a Kaa application with name smart_kettle. Creating applications with names like smart-kettle, smartkettle, or similar, will cause a conflict when loading such configuration in Java-based services.

To prevent this issue, starting from Kaa 1.2 we introduced the option of application name auto-generation on creation via the Tekton REST API. Additionally, the application version name suffix is restricted to lowercase Latin letters (a-z) and digits (0-9) when you create new application versions.

Known workaround: To prevent the issue from happening in affected Kaa versions, refrain from creating applications or application versions with names that only differ by dashes (-) or underscores (_). Starting with Kaa 1.2 it is recommended to enable the application name auto-generation feature.

Kaa 1.1 (November 8-th, 2019)

Find below high-level descriptions of some of the major release highlights.

Kaa Tekton

Kaa Tekton is a new Kaa infrastructure component that takes ownership of the Kaa applications, application versions, and the application-specific configurations for Kaa service instances. In previous Kaa versions applications and their configurations were defined in the configuration files of each Kaa microservice. Such configuration is still supported for backward compatibility and simple deployments. However, Tekton now offers a more convenient management mechanism.

Tekton exposes REST API for managing applications, versions, and the associated configurations. The Web Dashboard (WD) leverages this API and provides a convenient management UI. Applications are represented as protected resources in the auth server, so you can configure users’ level of access by granting corresponding OAuth 2.0 scopes.

Application configuration

Whenever the list of applications, versions, or service configs changes, Tekton generates an appropriate 17/SCMP notification message over NATS that gets delivered to all affected service replicas. In turn, they reload updated configurations from Tekton and apply changes immediately without a restart. It is no longer necessary to update all service instance configuration files or reboot service replicas.

You can use this script to convert your Kaa 1.0 blueprint configuration files into a JSON suitable for Tekton bulk configuration load REST API. Note that due to the various compatibility reasons the application and application version names must be limited to lowercase latin letters (a-z), digits (0-9), dashes (-) and underscores (_).

Unified resource naming convention

The Kaa platform uses OAuth 2.0 and User-Managed Access (UMA) for API calls authentication and authorization. Starting with Kaa 1.1 all Kaa components follow a unified resource naming convention that is designed to prevent naming conflicts. Users upgrading from Kaa 1.0 must rename resources in their auth servers according to the new convention.

Web Dashboard theme customization

Previously, the Web Dashboard theme customization was possible via loading custom CSS. In Kaa 1.1 the primary UI colors were revised and consolidated, and the theme customization is now possible right from the Administration page.

Theme customization

Other highlights of Kaa 1.1

  • [CEX] CEX now supports synchronous and asynchronous REST API calls for command invocation. Due to this, the [RCI][RCI] microservice is now deprecated and will be retired in a future Kaa version.
  • [CEX] In previous Kaa versions CEX would send new commands asynchronously to any connected endpoint. Starting with Kaa 1.1, according to 11/CEP protocol, CEX no longer would send new commands asynchronously after the endpoint sends a command request message with observe flag explicitly set to false.
  • [CMX] In previous Kaa versions CMX would send new configurations asynchronously to any connected endpoint. Starting with Kaa 1.1, according to 7/CMP protocol, CMX no longer would send new configurations asynchronously after the endpoint sends a configuration request message with observe flag explicitly set to false.
  • [EPTS] Auto-extracted time series now have auto~ prefix to prevent name conflicts with user configured time series.
  • [EPTS] Support for saving time series data via the EPTS REST API.
  • [EPTS] Support for storing and retrieving time series data with explicit null values.
  • [EPTS] User configured time series names are now limited to a combination of lowercase latin letters (a-z), digits (0-9), dashes (-) and underscores (_).
  • [CM] REST API OAuth 2.0 scopes for client credential and certificate management changed to kaa:client-credentials:read, kaa:client-credentials:create, kaa:client-credentials:update, kaa:client-certificates:read, kaa:client-certificates:create, and kaa:client-certificates:update.
  • [CM] CM now only accepts x.509 cert serial number in base 10 encoding on provisioning and validation. Also, a specific list of x.509 cert issuer fields processed by CM is restricted and documented here. Active x.509 client certificates must be re-provisioned into CM.
  • [EPR] EPR no longer supports GET /applications and GET /applications/{applicationName} REST API calls. The API for retreving application information is now available from Tekton.
  • [EPR] EPR now supports creating endpoints with user-defined endpoint IDs. Such IDs must match ^[a-zA-Z0-9._~-]+$ regex pattern. This is useful for matching entities between Kaa and external systems.
  • [KPC] KPC now always observes the MQTT keepalive interval set by the client in the CONNECT packet.
  • [KPC] Support for endpoint roaming across client MQTT connections to different KPC replicas. When multiple gateways (Kaa clients) represent a single endpoint, they may communicate on behalf of such endpoint in turns. Whenever such communication switch (roaming) occurs, even across KPC replicas, they are now able to identify that and correctly remove endpoint from the routing table of the previously used KPC replica.
  • [WD] Default “Device management” and “Software management” pages are now available in WD to administrators with application:update OAuth 2.0 scope granted for a given Kaa application.
  • [WD] Added the ability to specify the default sort key and direction in endpoint list and time series table widgets.
  • Documentation restructuring: previously Kaa components documentation was hosted separately from the general documentation. For the readers’ convenience, all component docs are now relocated to the “Features and components” section. This documentation now captures the exact state of all platform components at the moment of the general platform release. You can refer to the “Components” subsection of each feature page to find out the specific service versions included in a given release. For example, see the “Device management” feature page.
  • Multiple performance optimizations, usability and visual improvements, and bug fixes.

Kaa 1.0 (June 10-th, 2019)

Kaa 1.0 is the initial general release of the Kaa Enterprise IoT platform.

Prior to the 1.0 version, every Kaa component was versioned independently. Such independent versioning still exists for each of the Kaa microservices, while the Kaa 1.0 release is a “meta-package” that includes a set of component versions. All of the microservices in Kaa 1.0 have been tested for interoperation and can be installed in one shot to a Kubernetes cluster of your choice with a new Kaa installer.