KAFKA_CLUSTER

The KAFKA_CLUSTER sharedConfig type defines the details of Kafka cluster to connect to and common Kafka client configuration.

Once this shared config is defined, it can be referred to in any configuration of the publishing or subscription services .

This means that any common Kafka cluster configuration can be defined as a KAFKA_CLUSTER shared config, and reused in multiple services.

Example 1. Sample configuration for sharedConfig of type KAFKA_CLUSTER
    {
      "sharedConfigName": "localKafkaCluster",
      "sharedConfigType": "KAFKA_CLUSTER",
      "description": "Kafka sharedConfig for local Kafka cluster",
      "config": {
        "application": {
          "bootstrapServers": [
            "localhost:9092",
            "localhost:9093"
          ],
          "configuration": {
            "metadata.max.age.ms": 1000
          }
        }
      }
    }

The supported application configuration parameters for this sharedConfig type are defined in the table below:

KAFKA_CLUSTER sharedConfig type
Name Type Description Mandatory Default value

bootstrapServers

list of strings

The list of host and port pairs of the Kafka brokers in the Kafka cluster, For example: localhost:9092, localhost:9093

Yes

n\a

configuration

map

The map of Kafka client’s configuration keys and value pairs

No

empty map

sslDetails

SslDetails

The SSL connection details for Kafka clients. More details below.

No

n/a

saslDetails

SaslDetails

The SASL authentication configuration details. More details below.

No

n/a

securityProtocol

SecurityProtocol

Protocol used to communicate with brokers. More details below.

No

PLAINTEXT

SslDetails
Name Type Description Mandatory Default value

trustStoreLocation

string

The Kafka client’s JKS truststore file location

true

n/a

trustStorePassword

string

The Kafka client’s JKS truststore password

false

n/a

keystoreLocation

string

The Kafka client’s keystore file location

false

n/a

keystorePassword

string

The Kafka client’s keystore password

false

n/a

SaslDetails
Name Type Description Mandatory Default value

saslMechanism

string

The SASL Authentication mechanism. Accepted values are: PLAIN, GSSAPI, SCRAM_SHA_256, SCRAM_SHA_512

true

n/a

userName

string

The username for PLAIN, SCRAM_SHA_256 or SCRAM_SHA_512 SASL mechanism

mandatory if saslMechanism is PLAIN, SCRAM_SHA_256 or SCRAM_SHA_512

n/a

password

string

The password for PLAIN, SCRAM_SHA_256 or SCRAM_SHA_512 SASL mechanism

mandatory if saslMechanism is PLAIN, SCRAM_SHA_256 or SCRAM_SHA_512

n/a

kerberosServiceName

string

The kerberos service name for GSSAPI SASL

mandatory if saslMechanism is GSSAPI

n/a

securityProtocol

The securityProtocol parameter is an enum and encompasses all security protocols supported by Kafka clients. They include:

SecurityProtocol
Name

PLAINTEXT

SSL

SASL_PLAINTEXT

SASL_SSL

All configuration parameters for the KAFKA_CLUSTER sharedConfig type can also be defined within the configuration for SUBSCRIPTION_SERVICE or PUBLISHING_SERVICE services.
Any configuration defined in the sharedConfig can be overridden by defining it in service specific configuration. If the configuration is of type Map, they will be merged. The entry in service configuration will take precedence.

See Configuring the adapter for a complete example of the configuration for the adapter which specifies an instance of KAFKA_CLUSTER sharedConfig and refers to it in the service configurations.