Kafka Connector

Thanh Nguyen

Introduction

• The Anypoint Connector for Apache Kafka allows you to interact with the Apache Kafka messaging system

• Enabling seamless integration between your Mule applications and an Apache Kafka cluster, using Mule runtime

Compatibility

Application/Service VersionMule Runtime 3.7.0 and higherApache Kafka 0.9.0.0

Configure the Kafka Connector Global Element

Global Element Properties for Kafka Connector

Field Description

Name Enter a name for this connector configuration to be able to reference it later.

Bootstrap Servers Comma-separated host-port pairs for establishing the initial connection to the Kafka cluster — same as bootstrap.servers you provide to Kafka clients (producer/consumer). If provided with Producer/Consumer Properties files this value is ignored and the one from the Properties file is used.

Global Element Properties for Kafka Connector

Consumer Properties File Path to properties file where you can set the Consumer — similar to what you provide to Kafka command line tools. If you do not specify a value for bootstrap.servers within properties file, the value provided with Bootstrap Servers is going to be used. Also if you do not specify a value for key.serializerand value.serializer they will be set to org.apache.kafka.common.serialization.StringDeserializer.

Global Element Properties for Kafka Connector

Producer Properties File Path to properties file where you can set the producer — similar to what you provide to Kafka command line tools. If you do not specify a value for bootstrap.servers within properties file, the value provided with Bootstrap Servers is going to be used. Also if you do not specify a value for key.serializerand value.serializer they will be set to org.apache.kafka.common.serialization.StringSerializer

Connector Namespace and Schema

• Namespace:– xmlns:apachekafka=http://

www.mulesoft.org/schema/mule/apachekafka• Schema Location:– xsi:schemaLocation="http://www.mulesoft.org/

schema/mule/apachekafka http://www.mulesoft.org/schema/mule/sfdc-composite/current/mule-apachekafka.xsd"

Connector Namespace and Schema

Using the Connector in a Mavenized Mule App

<dependency> <groupId>org.mule.modules</groupId> <artifactId>mule-module-kafka</artifactId> <version>1.0.0</version></dependency>

Kafka Connector Example Use Cases

• use the connector to consume messages from a topic and log each consumed message to console in the following format: "New message arrived: <message>"

Kafka Connector Example Use Cases

1. Create a new Mule Project by clicking on File > New > Mule Project.

2. With your project open, search the Studio palette for the Kafka connector you should have already installed. Drag and drop a new Apache Kafka connector onto the canvas.

Kafka Connector Example Use Cases

3. Drag and drop a Logger after the Apache Kafka element to log incoming messages in the console

Kafka Connector Example Use Cases

4. Double click on the flow’s header and rename it consumer-flow

Kafka Connector Example Use Cases

5. Double click on the Apache Kafka connector element, and configure its properties as below

Kafka Connector Example Use Cases

5. Double click on the Apache Kafka connector element, and configure its properties as below

Kafka Connector Example Use Cases

6. Select the logger and set its fields like so:

Kafka Connector Example Use Cases

7. Enter your valid Apache Kafka properties in /src/main/app/mule-app.properties and identify them there using property placeholders

8. deploy the app on Studio’s embedded Mule runtime (Run As > Mule Application). When a new message is pushed into the topic you set consumer.topic to, you should see it logged in the console

Question and answer