Deployment

All Kaa services, including ECR, are distributed as Helm charts. You can run these charts using Kubernetes.

Installing ECR chart on Kubernetes

Preparation

These steps should be done once for your entire Kaa cluster in Kubernetes.

  1. Install Kubernetes.

  2. Install Helm client and Tiller server.

  3. Create a Kaa license secret (remember to put in your Kaa license key file contents and password):

     export HISTCONTROL=ignorespace # Prevent saving your key password in the shell history; note the leading space in the next line
      cat << EOF > /tmp/kaa-licence.yaml
     apiVersion: v1
     data:
       file: < your licence key file contents, base64-encoded >
       password: < your licence key password >
     kind: Secret
     metadata:
       name: license
       type: Opaque
     EOF
     kubectl create -f /tmp/kaa-licence.yaml
    
  4. Specify the image pull secret for the official KaaIoT docker registry. To define this secret, use your KaaID credentials:

     export HISTCONTROL=ignorespace  # Prevent saving your credentials in the shell history; note the leading space in the next line
      export KAAID_EMAIL=<your KaaID email, eg. bob@example.com> KAAID_PASSWORD=<your KaaID password>
     kubectl create secret docker-registry kaaid --docker-server=dev-hub.kaaiot.net --docker-username=$KAAID_EMAIL --docker-email=$KAAID_EMAIL --docker-password=$KAAID_PASSWORD
    
  5. If you want to enable API security in your Kaa cluster (recommended), you should create an auth server backend secret for the service to be able to request PAT tokens.

     export HISTCONTROL=ignorespace # Prevent saving your client ID amd secret in the shell history; note the leading space in the next line
      cat << EOF > /tmp/keycloak-backend-secret.yaml
     apiVersion: v1
     data:
       client-id: < your client-id, base64-encoded >
       client-secret: < your client-secret, base64-encoded >
     kind: Secret
     metadata:
       name: "keycloak-backend"
       labels:
         app.kubernetes.io/name: "kaa-name"
         helm.sh/chart: "kaa-chart-version"
         app.kubernetes.io/instance: "kaa-instance-name"
         app.kubernetes.io/managed-by: "release-service-name"
     EOF
     kubectl create -f /tmp/keycloak-backend-secret.yaml
    
  6. Add the KaaIoT Helm repository:

     helm repo add kaa-museum https://museum.kaaiot.net/
    

Installation

Once you have completed the preparation steps, everything is ready for deploying ECR on your Kubernetes cluster. To deploy the service, run the following command (observe the reference to the previously created license secret):

helm install --set global.license.secretName=license kaa-museum/ecr --name kaa-ecr

Check that pods are running:

kubectl get pods

Once the service initialization is complete, you should observe the output similar to the below:

NAME                                          READY   STATUS             RESTARTS   AGE
...
kaa-ecr                       1/1     Running            0          2m
...

Chart Requirements

Repository Name Version
@bitnami-pre-2022 mongodb 10.1.1
@kaa service-chart 0.0.88

Chart Values

Key Type Default Description
affinity object {}  
annotations.deployment object {}  
annotations.pod object {}  
config string ""  
env.JAVA_OPTS.value string "-XX:MaxRAMPercentage=80.0 -XX:MinRAMPercentage=50.0 -Xverify:none -XX:TieredStopAtLevel=1"  
exports.dependencies.mongodb.enabled bool true  
extraPodSpecs.automountServiceAccountToken bool false  
fullnameOverride string ""  
global.epr.baseUrl string "" EPR base URL.
global.iamcore.enabled bool false  
global.image.pullSecrets list []  
global.kaaIngress.tls.issuerKind string "Issuer"  
global.kaaIngress.tls.issuerName string "letsencrypt-stage"  
global.kaaIngress.tls.selfSigned bool false  
global.keycloak.backend.existingSecret string "-keycloak-backend"  
global.keycloak.enabled string ""  
global.keycloak.privateUrl string ""  
global.keycloak.publicUrl string ""  
global.keycloak.realm string ""  
global.license.secretName string ""  
global.mongodb.url string ""  
global.monitoring.enabled bool false  
global.nats.url string ""  
global.tekton.enabled string ""  
global.tekton.url string ""  
global.tenantManager.baseUrl string "" Tenant manager base URL.
global.tenantManager.enabled string "" Enables multitenancy using the Kaa Tenant Manager. Overrides tenantManager.enabled.
image.pullPolicy string "IfNotPresent"  
image.pullSecrets list []  
image.repository string "dev-hub.kaaiot.net/core/service//"  
image.tag string ""  
ingress.annotations object {}  
ingress.enabled bool false  
ingress.hosts[0] string "chart-example.local"  
ingress.paths list []  
ingress.tls list []  
metadata.component string "backend"  
metadata.partOf string "kaa"  
mongodb.auth.enabled bool false  
mongodb.enabled bool true  
mongodb.image.pullPolicy string "IfNotPresent"  
mongodb.image.tag string "3.6.10"  
mongodb.persistence.enabled bool true  
mongodb.url string "mongodb://-:27017"  
mongodb.urlOverride string ""  
mongodb.useStatefulSet bool true  
monitoring.metrics.export.path string "/prometheus"  
monitoring.metrics.type string "java"  
monitoring.rules[0].alert string "ECR connection pool has totally more than 3 leased connections"  
monitoring.rules[0].annotations.message string "Got more then 3 leased pool's connections error on the pod [/]."  
monitoring.rules[0].annotations.runbook string ""  
monitoring.rules[0].expr string "sum(httpcomponents_httpclient_pool_total_connections{httpclient=\"rest-client-connection-pool\", service=~\".*-ecr\", state=\"leased\"}) > 3"  
monitoring.rules[0].for string "1m"  
monitoring.rules[0].labels.severity string "error"  
monitoring.rules[1].alert string "ECR connection pool has more than 1 pending connection"  
monitoring.rules[1].annotations.message string "Got more then 1 pending pool's connection error on the pod [/]."  
monitoring.rules[1].annotations.runbook string ""  
monitoring.rules[1].expr string "sum(httpcomponents_httpclient_pool_total_pending{httpclient=\"rest-client-connection-pool\", service=~\".*-ecr\"}) > 1"  
monitoring.rules[1].for string "1m"  
monitoring.rules[1].labels.severity string "error"  
monitoring.rules[2].alert string "ECR connection pool has totally more than 75% available connections"  
monitoring.rules[2].annotations.message string "Got more then 75% available pool's connections error on the pod [/]."  
monitoring.rules[2].annotations.runbook string ""  
monitoring.rules[2].expr string "sum(httpcomponents_httpclient_pool_total_connections{httpclient=\"rest-client-connection-pool\", service=~\".*-ecr\", state=\"available\"}) > (0.75 * sum(httpclient_pool_total_max{httpclient=\"rest-client-connection-pool\", service=~\".*-ecr\"}))"  
monitoring.rules[2].for string "1m"  
monitoring.rules[2].labels.severity string "error"  
monitoring.rules[3].alert string "ECR connection pool has routes with half leased connections"  
monitoring.rules[3].annotations.message string "Got warning - appeared connection pool's routes with half leased connections on the pod [/]."  
monitoring.rules[3].annotations.runbook string ""  
monitoring.rules[3].expr string "sum(httpclient_pool_routes_with_half_leased_connections{service=~\".*-ecr\"}) > 0"  
monitoring.rules[3].for string "1m"  
monitoring.rules[3].labels.severity string "warning"  
monitoring.rules[4].alert string "ECR connection pool has routes with all leased connections"  
monitoring.rules[4].annotations.message string "Got warning - appeared connection pool's routes with all leased connections on the pod [/]."  
monitoring.rules[4].annotations.runbook string ""  
monitoring.rules[4].expr string "sum(httpclient_pool_routes_with_max_leased_connections{service=~\".*-ecr\"}) > 0"  
monitoring.rules[4].for string "1m"  
monitoring.rules[4].labels.severity string "error"  
monitoring.rules[5].alert string "ECR connection pool has routes with pending connections"  
monitoring.rules[5].annotations.message string "Got warning - appeared connection pool's routes with pending connections on the pod [/]."  
monitoring.rules[5].annotations.runbook string ""  
monitoring.rules[5].expr string "sum(httpclient_pool_routes_with_pending_connections{service=~\".*-ecr\"}) > 0"  
monitoring.rules[5].for string "1m"  
monitoring.rules[5].labels.severity string "error"  
nameOverride string ""  
nodeSelector object {}  
probes.enabled bool true Enables liveness, readiness, and startup probes for containers.
probes.liveness.initialDelaySeconds int 600  
probes.liveness.periodSeconds int 3  
probes.readiness.initialDelaySeconds int 10  
probes.readiness.periodSeconds int 5  
replicaCount int 1  
resources.limits.cpu int 1  
resources.limits.memory string "700Mi"  
resources.requests.cpu string "100m"  
resources.requests.memory string "700Mi"  
runbookUrl string ""  
securityContext object {}  
service.externalIPs list []  
service.loadBalancerIP string ""  
service.port int 80  
service.type string "ClusterIP"  
terminationMessagePolicy string "FallbackToLogsOnError"  
tolerations list []  
updateStrategy.type string "RollingUpdate"  
waitContainers.curl.image string "dev-hub.kaaiot.net/devops/docker/alpine-curl-jq/alpine-curl-jq"  
waitContainers.curl.tag string "0.0.3"  
waitContainers.enabled bool true  
waitContainers.timeout int 300  

The keys with no description are standard Kubernetes values. Refer to the official Kubernetes documentation for more information on these.

Environment variables

The table below summarizes the variables supported by the ECR Docker image and provides default values along with descriptions.

Variable name Default value Description
INSTANCE_NAME ecr Service instance name.
APP_CONFIG_PATH /srv/ecr/service-config.yml Path to the service configuration YAML file inside container. In case of running in Kubernetes, consider using K8s Volumes for externalization.
KAA_TEKTON_ENABLED false Enables Tekton integration.
KAA_TEKTON_URL http://tekton URL of the Tekton service.
KAA_SECURITY_ENABLED false Enables authentication and authorization on REST API endpoints (inbound and outbound API calls).
KAA_SECURITY_ISSUER_PUBLIC_URL   OAuth 2.0 issuer public URL for the system tenant (“kaa”).
KAA_SECURITY_ISSUER_PRIVATE_URL   OAuth 2.0 issuer private URL for the system tenant (“kaa”).
KAA_SECURITY_CLIENT_ID   Client ID for making requests in the system tenant scope.
KAA_SECURITY_CLIENT_SECRET   Client secret for making requests in the system tenant scope.
KAA_SECURITY_IAMCORE_ENABLED false Enables iamcore integration.
KAA_SECURITY_IAMCORE_API_KEY   iamcore API KEY.
KAA_SECURITY_IAMCORE_URL "https://cloud.iamcore.io" iamcore server URL.
KAA_SECURITY_IAMCORE_APPLICATION "kaa" iamcore application.
KAA_SECURITY_MULTITENANCY_ENABLED false Enables multitenancy via integration with the Kaa Tenant Manager. Only effective when kaa.security.enabled is set to true.
KAA_SECURITY_MULTITENANCY_TENANT_MANAGER_URL "http://tenant-manager" URL of the Kaa Tenant Manager that provides security configurations for tenants.
JMX_ENABLE false Enables JMX monitoring.
JMX_PORT 10500 JMX service port.
JMX_MONITOR_USER_PASSWORD   JMX monitor user password. Required when JMX_ENABLE=true.
JAVA_OPTIONS -Xmx700m Additional parameters for Java process launch.
NATS_USERNAME   Username for connecting to NATS message broker.
NATS_PASSWORD   Password for connecting to NATS message broker.
MONGODB_USERNAME   Username for connecting to MongoDB.
MONGODB_PASSWORD   Password for connecting to MongoDB.
KAA_LICENSE_CERT_PATH /run/license/license.p12 Path to the Kaa platform license certificate file in PKCS #12 format.
KAA_LICENSE_CERT_PASSWORD   License certificate password. Required.