Deployment
All Kaa services, including CM, are distributed as Helm charts. You can run these charts using Kubernetes.
Installing CM chart on Kubernetes
Preparation
These steps should be done once for your entire Kaa cluster in Kubernetes.
-
Install Helm client and Tiller server.
-
Create a Kaa license secret (remember to put in your Kaa license key file contents and password):
export HISTCONTROL=ignorespace # Prevent saving your key password in the shell history; note the leading space in the next line cat << EOF > /tmp/kaa-licence.yaml apiVersion: v1 data: file: < your licence key file contents, base64-encoded > password: < your licence key password > kind: Secret metadata: name: license type: Opaque EOF kubectl create -f /tmp/kaa-licence.yaml
-
Specify the image pull secret for the official KaaIoT docker registry. To define this secret, use your KaaID credentials:
export HISTCONTROL=ignorespace # Prevent saving your credentials in the shell history; note the leading space in the next line export KAAID_EMAIL=<your KaaID email, eg. bob@example.com> KAAID_PASSWORD=<your KaaID password> kubectl create secret docker-registry kaaid --docker-server=dev-hub.kaaiot.net --docker-username=$KAAID_EMAIL --docker-email=$KAAID_EMAIL --docker-password=$KAAID_PASSWORD
-
If you want to enable API security in your Kaa cluster (recommended), you should create an auth server backend secret for the service to be able to request PAT tokens.
export HISTCONTROL=ignorespace # Prevent saving your client ID amd secret in the shell history; note the leading space in the next line cat << EOF > /tmp/keycloak-backend-secret.yaml apiVersion: v1 data: client-id: < your client-id, base64-encoded > client-secret: < your client-secret, base64-encoded > kind: Secret metadata: name: "keycloak-backend" labels: app.kubernetes.io/name: "kaa-name" helm.sh/chart: "kaa-chart-version" app.kubernetes.io/instance: "kaa-instance-name" app.kubernetes.io/managed-by: "release-service-name" EOF kubectl create -f /tmp/keycloak-backend-secret.yaml
-
Add the KaaIoT Helm repository:
helm repo add kaa-museum https://museum.kaaiot.net/
Installation
Once you have completed the preparation steps, everything is ready for deploying CM on your Kubernetes cluster. To deploy the service, run the following command (observe the reference to the previously created license secret):
helm install --set global.license.secretName=license kaa-museum/cm --name kaa-cm
Check that pods are running:
kubectl get pods
Once the service initialization is complete, you should observe the output similar to the below:
NAME READY STATUS RESTARTS AGE
...
kaa-cm 1/1 Running 0 2m
...
Chart Requirements
Repository | Name | Version |
---|---|---|
@bitnami-pre-2022 | mariadb | 9.0.1 |
@kaa | service-chart | 0.0.88 |
Chart Values
Key | Type | Default | Description |
---|---|---|---|
affinity | object | {} |
|
annotations.deployment | object | {} |
|
annotations.pod | object | {} |
|
app.ports.http | int | 80 |
|
app.ports.management | int | 8080 |
|
config | string | "" |
Content for the service config map, automatically mounted as a config file into the pod. |
env | object | {"JAVA_OPTS":{"value":"-XX:MaxRAMPercentage=80.0 -XX:MinRAMPercentage=50.0 -Xverify:none -XX:TieredStopAtLevel=1"}} |
Defines the environment variables that Kubernetes passes to the service replica. |
extraPodSpecs.automountServiceAccountToken | bool | false |
|
extraVolumeMounts | list | [] |
|
extraVolumes | list | [] |
|
fullnameOverride | string | "" |
|
global.iamcore.enabled | bool | false |
|
global.image.pullSecrets | list | [] |
List of image pull secret names. Each must be defined as a record with the name field. Overrides image.pullSecrets . |
global.kaaIngress.tls.issuerKind | string | "Issuer" |
|
global.kaaIngress.tls.issuerName | string | "letsencrypt-stage" |
|
global.kaaIngress.tls.selfSigned | bool | false |
|
global.keycloak.backend.existingSecret | string | "-keycloak-backend" |
|
global.keycloak.enabled | string | "" |
Enables API security using the auth provider. Overrides keycloak.enabled . |
global.keycloak.privateUrl | string | "" |
|
global.keycloak.publicUrl | string | "" |
|
global.keycloak.realm | string | "" |
Auth provider realm. Required. |
global.license.existingSecret | string | "" |
Name of the license secret, which must contain two base64-encoded fields: file (license file contents in PKCS #12) and password . Overrides license.secretName . |
global.mariadb.existingSecret | string | "" |
Name of an existing Kubernetes secret containing MariaDB admin and user passwords (mariadb-root-password and mariadb-password secret keys, respectively). |
global.mariadb.url | string | "" |
Mariadb URL. |
global.monitoring.enabled | bool | false |
|
global.nats.url | string | "" |
NATS URL. |
global.tekton.enabled | string | "" |
Enables Tekton integration. When disabled, the service will expect Kaa application configs to be defined in the config map. Overrides tekton.enabled . |
global.tekton.url | string | "" |
Tekton URL. Overrides tekton.url . |
global.tenantManager.baseUrl | string | "" |
Tenant manager base URL. |
global.tenantManager.enabled | string | "" |
Enables multitenancy using the Kaa Tenant Manager. Overrides tenantManager.enabled . |
image.pullPolicy | string | "IfNotPresent" |
Docker image pull policy. |
image.pullSecrets | list | [] |
List of image pull secret names. Each must be defined as a record with the name field. |
image.repository | string | "dev-hub.kaaiot.net/core/service//" |
Docker image repository image URL. |
image.tag | string | "" |
Docker image tag version to pull and run. |
ingress.annotations | object | {} |
|
ingress.enabled | bool | false |
|
ingress.hosts[0] | string | "chart-example.local" |
|
ingress.paths | list | [] |
|
ingress.tls | list | [] |
|
mariadb.auth.database | string | "cm" |
|
mariadb.auth.existingSecret | string | "true" |
|
mariadb.auth.username | string | "kaa" |
|
mariadb.enabled | bool | true |
Enables Mariadb dependency. For detailed info please look in https://github.com/helm/charts/tree/master/stable/mariadb |
mariadb.primary.persistence.enabled | bool | true |
|
mariadb.url | string | "mariadb://-:3306/" |
Mariadb URL. |
mariadb.urlOverride | string | "" |
Mariadb URL. Overrides global.mariadb.url and mariadb.url . |
metadata.component | string | "backend" |
|
metadata.partOf | string | "kaa" |
|
monitoring.metrics.export.path | string | "/prometheus" |
Specifies REST resource path that exposes metrics. |
monitoring.metrics.type | string | "java" |
|
monitoring.rules[0].alert | string | "CM connection pool has totally more than 3 leased connections" |
|
monitoring.rules[0].annotations.message | string | "Got more then 3 leased pool's connections error on the pod [/]." |
|
monitoring.rules[0].annotations.runbook | string | "" |
|
monitoring.rules[0].expr | string | "sum(httpcomponents_httpclient_pool_total_connections{httpclient=\"rest-client-connection-pool\", service=~\".*-cm\", state=\"leased\"}) > 3" |
|
monitoring.rules[0].for | string | "1m" |
|
monitoring.rules[0].labels.severity | string | "error" |
|
monitoring.rules[1].alert | string | "CM connection pool has more than 1 pending connection" |
|
monitoring.rules[1].annotations.message | string | "Got more then 1 pending pool's connection error on the pod [/]." |
|
monitoring.rules[1].annotations.runbook | string | "" |
|
monitoring.rules[1].expr | string | "sum(httpcomponents_httpclient_pool_total_pending{httpclient=\"rest-client-connection-pool\", service=~\".*-cm\"}) > 1" |
|
monitoring.rules[1].for | string | "1m" |
|
monitoring.rules[1].labels.severity | string | "error" |
|
monitoring.rules[2].alert | string | "CM connection pool used all available connections" |
|
monitoring.rules[2].annotations.message | string | "Got more then 1 pending pool's connection error on the pod [/]." |
|
monitoring.rules[2].annotations.runbook | string | "" |
|
monitoring.rules[2].expr | string | "httpclient_pool_connections_per_route_available{host=~\".*keycloak.*\", service=~\".*-cm\"} >= httpcomponents_httpclient_pool_route_max_default{httpclient=\"rest-client-connection-pool\", service=~\".*-cm\"}" |
|
monitoring.rules[2].for | string | "1m" |
|
monitoring.rules[2].labels.severity | string | "error" |
|
monitoring.rules[3].alert | string | "CM connection pool has totally more than 75% available connections" |
|
monitoring.rules[3].annotations.message | string | "Got more then 75% available pool's connections error on the pod [/]." |
|
monitoring.rules[3].annotations.runbook | string | "" |
|
monitoring.rules[3].expr | string | "sum(httpcomponents_httpclient_pool_total_connections{httpclient=\"rest-client-connection-pool\", service=~\".*-cm\", state=\"available\"}) > (0.75 * sum(httpcomponents_httpclient_pool_total_max{httpclient=\"rest-client-connection-pool\", service=~\".*-cm\"}))" |
|
monitoring.rules[3].for | string | "1m" |
|
monitoring.rules[3].labels.severity | string | "error" |
|
monitoring.rules[4].alert | string | "CM connection pool has routes with half leased connections" |
|
monitoring.rules[4].annotations.message | string | "Got warning - appeared connection pool's routes with half leased connections on the pod [/]." |
|
monitoring.rules[4].annotations.runbook | string | "" |
|
monitoring.rules[4].expr | string | "sum(httpclient_pool_routes_with_half_leased_connections{service=~\".*-cm\"}) > 0" |
|
monitoring.rules[4].for | string | "1m" |
|
monitoring.rules[4].labels.severity | string | "warning" |
|
monitoring.rules[5].alert | string | "CM connection pool has routes with all leased connections" |
|
monitoring.rules[5].annotations.message | string | "Got warning - appeared connection pool's routes with all leased connections on the pod [/]." |
|
monitoring.rules[5].annotations.runbook | string | "" |
|
monitoring.rules[5].expr | string | "sum(httpclient_pool_routes_with_max_leased_connections{service=~\".*-cm\"}) > 0" |
|
monitoring.rules[5].for | string | "1m" |
|
monitoring.rules[5].labels.severity | string | "error" |
|
monitoring.rules[6].alert | string | "CM connection pool has routes with pending connections" |
|
monitoring.rules[6].annotations.message | string | "Got warning - appeared connection pool's routes with pending connections on the pod [/]." |
|
monitoring.rules[6].annotations.runbook | string | "" |
|
monitoring.rules[6].expr | string | "sum(httpclient_pool_routes_with_pending_connections{service=~\".*-cm\"}) > 0" |
|
monitoring.rules[6].for | string | "1m" |
|
monitoring.rules[6].labels.severity | string | "error" |
|
nameOverride | string | "" |
|
nodeSelector | object | {} |
|
probes.enabled | bool | true |
Enables liveness, readiness, and startup probes for containers. |
probes.liveness.initialDelaySeconds | int | 600 |
|
probes.liveness.periodSeconds | int | 3 |
|
probes.readiness.initialDelaySeconds | int | 10 |
|
probes.readiness.periodSeconds | int | 5 |
|
replicaCount | int | 1 |
The number of service instance replicas to run. |
resources.limits.cpu | int | 1 |
|
resources.limits.memory | string | "700Mi" |
|
resources.requests.cpu | string | "100m" |
|
resources.requests.memory | string | "700Mi" |
|
runbookUrl | string | "" |
|
securityContext | object | {} |
|
service.externalIPs | list | [] |
|
service.loadBalancerIP | string | "" |
|
service.port | int | 80 |
|
service.type | string | "ClusterIP" |
|
terminationMessagePolicy | string | "FallbackToLogsOnError" |
Kubernetes termination message policy. |
tolerations | list | [] |
|
updateStrategy.type | string | "RollingUpdate" |
Deployment update strategy. |
waitContainers.curl.image | string | "dev-hub.kaaiot.net/devops/docker/alpine-curl-jq/alpine-curl-jq" |
|
waitContainers.curl.tag | string | "0.0.3" |
|
waitContainers.enabled | bool | true |
Wait for dependency services. |
waitContainers.timeout | int | 300 |
Wait timeout for dependency services in seconds. |
The keys with no description are standard Kubernetes values. Refer to the official Kubernetes documentation for more information on these.
Environment variables
The table below summarizes the variables supported by the CM Docker image and provides default values along with descriptions.
Variable name | Default value | Description |
---|---|---|
INSTANCE_NAME |
cm | Service instance name. |
APP_CONFIG_PATH |
/srv/cm/service-config.yml | Path to the service configuration YAML file inside container. In case of running in Kubernetes, consider using K8s Volumes for externalization. |
KAA_TEKTON_ENABLED |
false | Enables Tekton integration. |
KAA_TEKTON_URL |
http://tekton | URL of the Tekton service. |
KAA_SECURITY_ENABLED |
false |
Enables authentication and authorization on REST API endpoints (inbound and outbound API calls). |
KAA_SECURITY_ISSUER_PUBLIC_URL |
OAuth 2.0 issuer public URL for the system tenant (“kaa”). | |
KAA_SECURITY_ISSUER_PRIVATE_URL |
OAuth 2.0 issuer private URL for the system tenant (“kaa”). | |
KAA_SECURITY_CLIENT_ID |
Client ID for making requests in the system tenant scope. | |
KAA_SECURITY_CLIENT_SECRET |
Client secret for making requests in the system tenant scope. | |
KAA_SECURITY_IAMCORE_ENABLED |
false |
Enables iamcore integration. |
KAA_SECURITY_IAMCORE_API_KEY |
iamcore API KEY. | |
KAA_SECURITY_IAMCORE_URL |
"https://cloud.iamcore.io" |
iamcore server URL. |
KAA_SECURITY_IAMCORE_APPLICATION |
"kaa" |
iamcore application. |
KAA_SECURITY_MULTITENANCY_ENABLED |
false |
Enables multitenancy via integration with the Kaa Tenant Manager. Only effective when kaa.security.enabled is set to true . |
KAA_SECURITY_MULTITENANCY_TENANT_MANAGER_URL |
"http://tenant-manager" |
URL of the Kaa Tenant Manager that provides security configurations for tenants. |
JMX_ENABLE |
false | Enables JMX monitoring. |
JMX_PORT |
10500 | JMX service port. |
JMX_MONITOR_USER_PASSWORD |
JMX monitor user password. Required when JMX_ENABLE=true . |
|
JAVA_OPTIONS |
-Xmx700m | Additional parameters for Java process launch. |
NATS_USERNAME |
Username for connecting to NATS message broker. | |
NATS_PASSWORD |
Password for connecting to NATS message broker. | |
KAA_LICENSE_CERT_PATH |
/run/license/license.p12 | Path to the Kaa platform license certificate file in PKCS #12 format. |
KAA_LICENSE_CERT_PASSWORD |
License certificate password. Required. |