Overview

Kafka Data Collection Adapter service (KDCA) is a Kaa service that receives endpoint data samples and publishes them into Kafka. Each data sample is published as a separate record.

Endpoint data sample is a piece of endpoint-related data of an arbitrary structure.

The record’s key is an endpoint ID that produced a data sample.

The record’s value is a JSON with the next payload structure:

  • serverTimestamp - time of a data sample message arrival to KDCA;
  • appVersionName - application version name of a device that produced a data sample;
  • dataSample - device data sample without any modification.

Ingestion

KDCA receives endpoint data samples through the data sample transmission interface. For more information on how you can configure the service instance name of the data transmitter, see Data transmitter configuration.

Broadcasting

As a result of ingestion phase, KDCA publishes endpoint data samples on specified Kafka topic, see Kafka configuration.

Interfaces

KDCA supports a number of interfaces to perform its functional role. The key supported interfaces are summarized in the following diagram.

KDCA interfaces diagram

For inter-service communication, Kaa services mainly use REST APIs and messaging protocols that run over NATS messaging system.

Data sample transmission

KDCA supports 13/DSTP and acts as a data sample receiver service by subscribing to data transmitters. See the configuration page for information on how to configure this interface.

Service configuration

Like all Kaa services, KDCA can fetch its configuration from Service Configuration Repository (SCR) (or other compatible implementation of Spring Cloud Config Server) using Spring Cloud Config REST API. KDCA fetches the configuration on startup as well as on any configuration updates. Configuration update events are broadcasted by SCR using Service Configuration Protocol (17/SCP).