Kafka Data Collection Adapter service (KDCA) is a Kaa service that receives endpoint data samples and publishes them into Kafka. Each data sample is published as a separate record.
Endpoint data sample is a piece of endpoint-related data of an arbitrary structure.
The record’s key is an endpoint ID that produced a data sample.
The record’s value is a JSON with the next payload structure:
serverTimestamp- time of a data sample message arrival to KDCA;
appVersionName- application version name of a device that produced a data sample;
dataSample- device data sample without any modification.
KDCA receives endpoint data samples through the data sample transmission interface. For more information on how you can configure the service instance name of the data transmitter, see Data transmitter configuration.
As a result of ingestion phase, KDCA publishes endpoint data samples on specified Kafka topic, see Kafka configuration.
KDCA supports a number of interfaces to perform its functional role. The key supported interfaces are summarized in the following diagram.
For inter-service communication, Kaa services mainly use REST APIs and messaging protocols that run over NATS messaging system.
Data sample transmission
Like all Kaa services, KDCA can fetch its configuration from Service Configuration Repository (SCR) (or other compatible implementation of Spring Cloud Config Server) using Spring Cloud Config REST API. KDCA fetches the configuration on startup as well as on any configuration updates. Configuration update events are broadcasted by SCR using Service Configuration Protocol (17/SCP).