Kafka connector

Kafka Connector

Thanh Nguyen

Introduction

• The Anypoint Connector for Apache Kafka allows you to interact with the Apache Kafka messaging system

• Enabling seamless integration between your Mule applications and an Apache Kafka cluster, using Mule runtime

Compatibility

Application/Service VersionMule Runtime 3.7.0 and higherApache Kafka 0.9.0.0

Configure the Kafka Connector Global Element

Global Element Properties for Kafka Connector

Field Description

Name Enter a name for this connector configuration to be able to reference it later.

Bootstrap Servers Comma-separated host-port pairs for establishing the initial connection to the Kafka cluster — same as bootstrap.servers you provide to Kafka clients (producer/consumer). If provided with Producer/Consumer Properties files this value is ignored and the one from the Properties file is used.


Consumer Properties File Path to properties file where you can set the Consumer — similar to what you provide to Kafka command line tools. If you do not specify a value for bootstrap.servers within properties file, the value provided with Bootstrap Servers is going to be used. Also if you do not specify a value for key.serializerand value.serializer they will be set to org.apache.kafka.common.serialization.StringDeserializer.


Producer Properties File Path to properties file where you can set the producer — similar to what you provide to Kafka command line tools. If you do not specify a value for bootstrap.servers within properties file, the value provided with Bootstrap Servers is going to be used. Also if you do not specify a value for key.serializerand value.serializer they will be set to org.apache.kafka.common.serialization.StringSerializer

Connector Namespace and Schema

• Namespace:– xmlns:apachekafka=http://

www.mulesoft.org/schema/mule/apachekafka• Schema Location:– xsi:schemaLocation="http://www.mulesoft.org/

schema/mule/apachekafka http://www.mulesoft.org/schema/mule/sfdc-composite/current/mule-apachekafka.xsd"

http://www.mulesoft.org/schema/mule/apachekafka



Connector Namespace and Schema

Using the Connector in a Mavenized Mule App

<dependency> <groupId>org.mule.modules</groupId> <artifactId>mule-module-kafka</artifactId> <version>1.0.0</version></dependency>

Kafka Connector Example Use Cases

• use the connector to consume messages from a topic and log each consumed message to console in the following format: "New message arrived: <message>"


1. Create a new Mule Project by clicking on File > New > Mule Project.

2. With your project open, search the Studio palette for the Kafka connector you should have already installed. Drag and drop a new Apache Kafka connector onto the canvas.


3. Drag and drop a Logger after the Apache Kafka element to log incoming messages in the console


4. Double click on the flow’s header and rename it consumer-flow


5. Double click on the Apache Kafka connector element, and configure its properties as below


6. Select the logger and set its fields like so:


7. Enter your valid Apache Kafka properties in /src/main/app/mule-app.properties and identify them there using property placeholders

8. deploy the app on Studio’s embedded Mule runtime (Run As > Mule Application). When a new message is pushed into the topic you set consumer.topic to, you should see it logged in the console

Question and answer

Kafka connector

Technology

Transcript of Kafka connector