Kafkawize Documentation€¦ · Apache Kafka is widely being adapted in organizations irrespective...
Transcript of Kafkawize Documentation€¦ · Apache Kafka is widely being adapted in organizations irrespective...
Kafkawize DocumentationRelease 41
Read the Docs
Apr 26 2020
Contents
1 Introduction 311 What is Kafkawize 312 Background 313 Key features 414 41 glimpse 415 How is it developed 416 Architecture 417 Git Repositories 518 Developer 5
2 Getting Started 721 Demo 722 Prerequisites 723 1 Download Kafkawize 724 2 Download Metastore 725 3 License Key 826 4 Build KW Cluster Api Application 827 5 Configure Kafkawize UI Api Application 828 6 Metastore setup 929 7 Run KW and KWClusterApi 10210 8 Kafka Connectivity 11211 9 Final Check 11
3 Migration Kafkawize from 35 to 40 13
4 Features 1541 Security 1542 Teams and Role Management 1543 Login 1644 Dashboard 1645 Browse 1746 Requests 1847 Clusters 1948 Audit 2049 Approve Or Decline 22410 Users 23411 Teams 24
i
412 Synchronize Metadata 25413 Admin 26414 Profile 26415 Logout 26
5 Functionalities 27
6 Customize 29
7 Trouble shooting 31
8 Release Notes 33
9 Roadmap 35
10 Contact Me 37
11 Submit a Review 39
12 Indices and tables 41
ii
Kafkawize Documentation Release 41
Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization Read more
Contents
Contents 1
Kafkawize Documentation Release 41
2 Contents
CHAPTER 1
Introduction
11 What is Kafkawize
Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization
12 Background
Let me give some background Apache Kafka is widely being adapted in organizations irrespective of the scale Itrsquosunique features like scalability retention and reliability unlike the traditional messaging platforms makes it stand out
While adapting Kafka you would notice there are few manual activities like creating topics acls updating configu-ration etc There have been few topic management tools in the market which provide automation to certain extenthowever its not complete Assume there is a team requesting for a kafka topic with in an organization They wouldeither request for it through email with a template or any user interface The actual kafka team in the organization ex-ecutes the commands to create topics With Development Test Acceptance and Production environments managingthese configurations by the Kafka team and handling all the requests from various teams would become a hassle asyou see the growth of teams using Kafka It is just not manual activity but Audit time and effort the team has to putin
Apart from that this meta data (Kafka topics acls) which is stored in ZK could be easily stored in a good readableformat and can be used as source of truth
Hence I saw the need of Kafkawize ndash a self service topic management tool to keep ZERO interaction by other Teamswith Kafka team for topics acls or schemas creation by introducing roles and authorizations to users of teams It isdeveloped with a simple front end and backend with Spring boot It saves time and effort governs information avoidsrisk in kafka config loss avoids manual activities and mistakes maintains up to date SOT (source of truth)
3
Kafkawize Documentation Release 41
13 Key features
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
14 41 glimpse
15 How is it developed
Simple technologies have been used to develop Kafkawize Front end with Angular HTML CSS MaterialViewbootstrap theme and Javascript Backend with Java Kafka Admin client libraries Spring boot and Spring securityframeworks First version of Kafkawize has been released in Sep 2018 with basic features and gradually enhanced it
16 Architecture
Kafkawize contains two main Apis (User Interface API and Cluster management API) and a Front end
User Interface Api directly communicates between Frontend UI and Cluster API
Front end is built with AngularJs HTML and Java script
Cluster API acts as middle layer between Kafka brokers and UserInterface API
4 Chapter 1 Introduction
Kafkawize Documentation Release 41
Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client
Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth
Requests from users are directed to cluster api and also data is stored metastore
On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application
17 Git Repositories
UI Api httpsgithubcommuralibasanikafkawize
Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi
18 Developer
Muralidhar Basani
LinkedIn httpswwwlinkedincominmuralibasani
Web httpskafkawizecom
17 Git Repositories 5
Kafkawize Documentation Release 41
6 Chapter 1 Introduction
CHAPTER 2
Getting Started
21 Demo
Lets start with setting up Kafkawize on your WindowsLinuxUnix systems
22 Prerequisites
bull Any IDE like IntelliJNetbeans or Eclipse to build the applications
bull Java installed machines to run the applications
bull Rdbms (like OracleMySQL) or Apache Cassandra
23 1 Download Kafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi
OR
Download Kafkawize 41 bundle here Kafkawize_41
24 2 Download Metastore
If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload
If Rdbms is used as metastore download
Mysql httpsdevmysqlcomdownloads
7
Kafkawize Documentation Release 41
Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml
Or any other Rdbms is ok
25 3 License Key
License key is not required from 41 version
26 4 Build KW Cluster Api Application
This Api does the below
bull Receives and responds to calls from UI Api
bull Connects to Kafka Brokers with Kafka AdminClient Api
bull There is no connection to any metastore
bull Swagger enabled
Unzip kafkawizeclusterapizip and open in any IDE
Configure the application properties (srcmainresources) if port has to be changed else default port is 9343
and run maven command to create a runnable jar
mvn clean package
This should create a jar (kafkawizeclusterapi-41jar) in target dir
Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even
27 5 Configure Kafkawize UI Api Application
This Api does the below
bull Users interact with interface with this Api
bull All the end points in this application either connect to Metastore or Cluster Api or both
Unzip kafkawize-(version)zip and open in any IDE
Configure application properties
In kafkawizesrcmainresourcesapplicationproperties
default port is 9097 if port has to be changed else
serverport9097
If metastore is cassandra (from step 4)
setstore type as cassandra
customdbstoretype=cassandra
8 Chapter 2 Getting Started
Kafkawize Documentation Release 41
configure cassandra host port and keyspace
customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042
If metastore is rdbms (from step 4)
setstore type as rdbms
customdbstoretype=rdbms
bull Install and run Rdbms (like MysqlOracle) and create a db schema or database
configure db properties like below
Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver
271 Configure Cluster Api
configure cluster api host and port details
customclusterapiurlhttplocalhost9343
bull ignore userpwd of cluster api properties
28 6 Metastore setup
281 Metastore Cassandra
In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo
bull Create tables and run insert scripts in Cassandra
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql
28 6 Metastore setup 9
Kafkawize Documentation Release 41
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
282 Metastore Rdbms
In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run MysqlOracle and create a db schema or database
bull Create tables and run insert scripts in Database
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
Run maven command to create a runnable jar
mvn clean package
This should create a jar in target dir (kafkawizetargetkafkawize-41jar)
29 7 Run KW and KWClusterApi
Run
java -jar kafkawizeclusterapi-41jar
Run
java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar
If application is running you can access UI from http[host][port]kafkawize
10 Chapter 2 Getting Started
Kafkawize Documentation Release 41
210 8 Kafka Connectivity
Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed
bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)
Examples SSL Based Acl (Note of double quotes in the below command if copied properly)
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster
Examples IP Based Acl
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster
bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application
211 9 Final Check
bull Cluster Api is running
bull Metastore (Cassandra or Rdbms) is running and has tables and data
bull UI Api is running
bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)
bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize
Default users passwords and roles
uiuser1user USER
uiuser2user USER
uiuser3user USER
uiuser4user ADMIN
uiuser5user ADMIN
superuseruser SUPERUSER
210 8 Kafka Connectivity 11
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Contents
1 Introduction 311 What is Kafkawize 312 Background 313 Key features 414 41 glimpse 415 How is it developed 416 Architecture 417 Git Repositories 518 Developer 5
2 Getting Started 721 Demo 722 Prerequisites 723 1 Download Kafkawize 724 2 Download Metastore 725 3 License Key 826 4 Build KW Cluster Api Application 827 5 Configure Kafkawize UI Api Application 828 6 Metastore setup 929 7 Run KW and KWClusterApi 10210 8 Kafka Connectivity 11211 9 Final Check 11
3 Migration Kafkawize from 35 to 40 13
4 Features 1541 Security 1542 Teams and Role Management 1543 Login 1644 Dashboard 1645 Browse 1746 Requests 1847 Clusters 1948 Audit 2049 Approve Or Decline 22410 Users 23411 Teams 24
i
412 Synchronize Metadata 25413 Admin 26414 Profile 26415 Logout 26
5 Functionalities 27
6 Customize 29
7 Trouble shooting 31
8 Release Notes 33
9 Roadmap 35
10 Contact Me 37
11 Submit a Review 39
12 Indices and tables 41
ii
Kafkawize Documentation Release 41
Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization Read more
Contents
Contents 1
Kafkawize Documentation Release 41
2 Contents
CHAPTER 1
Introduction
11 What is Kafkawize
Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization
12 Background
Let me give some background Apache Kafka is widely being adapted in organizations irrespective of the scale Itrsquosunique features like scalability retention and reliability unlike the traditional messaging platforms makes it stand out
While adapting Kafka you would notice there are few manual activities like creating topics acls updating configu-ration etc There have been few topic management tools in the market which provide automation to certain extenthowever its not complete Assume there is a team requesting for a kafka topic with in an organization They wouldeither request for it through email with a template or any user interface The actual kafka team in the organization ex-ecutes the commands to create topics With Development Test Acceptance and Production environments managingthese configurations by the Kafka team and handling all the requests from various teams would become a hassle asyou see the growth of teams using Kafka It is just not manual activity but Audit time and effort the team has to putin
Apart from that this meta data (Kafka topics acls) which is stored in ZK could be easily stored in a good readableformat and can be used as source of truth
Hence I saw the need of Kafkawize ndash a self service topic management tool to keep ZERO interaction by other Teamswith Kafka team for topics acls or schemas creation by introducing roles and authorizations to users of teams It isdeveloped with a simple front end and backend with Spring boot It saves time and effort governs information avoidsrisk in kafka config loss avoids manual activities and mistakes maintains up to date SOT (source of truth)
3
Kafkawize Documentation Release 41
13 Key features
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
14 41 glimpse
15 How is it developed
Simple technologies have been used to develop Kafkawize Front end with Angular HTML CSS MaterialViewbootstrap theme and Javascript Backend with Java Kafka Admin client libraries Spring boot and Spring securityframeworks First version of Kafkawize has been released in Sep 2018 with basic features and gradually enhanced it
16 Architecture
Kafkawize contains two main Apis (User Interface API and Cluster management API) and a Front end
User Interface Api directly communicates between Frontend UI and Cluster API
Front end is built with AngularJs HTML and Java script
Cluster API acts as middle layer between Kafka brokers and UserInterface API
4 Chapter 1 Introduction
Kafkawize Documentation Release 41
Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client
Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth
Requests from users are directed to cluster api and also data is stored metastore
On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application
17 Git Repositories
UI Api httpsgithubcommuralibasanikafkawize
Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi
18 Developer
Muralidhar Basani
LinkedIn httpswwwlinkedincominmuralibasani
Web httpskafkawizecom
17 Git Repositories 5
Kafkawize Documentation Release 41
6 Chapter 1 Introduction
CHAPTER 2
Getting Started
21 Demo
Lets start with setting up Kafkawize on your WindowsLinuxUnix systems
22 Prerequisites
bull Any IDE like IntelliJNetbeans or Eclipse to build the applications
bull Java installed machines to run the applications
bull Rdbms (like OracleMySQL) or Apache Cassandra
23 1 Download Kafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi
OR
Download Kafkawize 41 bundle here Kafkawize_41
24 2 Download Metastore
If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload
If Rdbms is used as metastore download
Mysql httpsdevmysqlcomdownloads
7
Kafkawize Documentation Release 41
Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml
Or any other Rdbms is ok
25 3 License Key
License key is not required from 41 version
26 4 Build KW Cluster Api Application
This Api does the below
bull Receives and responds to calls from UI Api
bull Connects to Kafka Brokers with Kafka AdminClient Api
bull There is no connection to any metastore
bull Swagger enabled
Unzip kafkawizeclusterapizip and open in any IDE
Configure the application properties (srcmainresources) if port has to be changed else default port is 9343
and run maven command to create a runnable jar
mvn clean package
This should create a jar (kafkawizeclusterapi-41jar) in target dir
Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even
27 5 Configure Kafkawize UI Api Application
This Api does the below
bull Users interact with interface with this Api
bull All the end points in this application either connect to Metastore or Cluster Api or both
Unzip kafkawize-(version)zip and open in any IDE
Configure application properties
In kafkawizesrcmainresourcesapplicationproperties
default port is 9097 if port has to be changed else
serverport9097
If metastore is cassandra (from step 4)
setstore type as cassandra
customdbstoretype=cassandra
8 Chapter 2 Getting Started
Kafkawize Documentation Release 41
configure cassandra host port and keyspace
customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042
If metastore is rdbms (from step 4)
setstore type as rdbms
customdbstoretype=rdbms
bull Install and run Rdbms (like MysqlOracle) and create a db schema or database
configure db properties like below
Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver
271 Configure Cluster Api
configure cluster api host and port details
customclusterapiurlhttplocalhost9343
bull ignore userpwd of cluster api properties
28 6 Metastore setup
281 Metastore Cassandra
In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo
bull Create tables and run insert scripts in Cassandra
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql
28 6 Metastore setup 9
Kafkawize Documentation Release 41
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
282 Metastore Rdbms
In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run MysqlOracle and create a db schema or database
bull Create tables and run insert scripts in Database
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
Run maven command to create a runnable jar
mvn clean package
This should create a jar in target dir (kafkawizetargetkafkawize-41jar)
29 7 Run KW and KWClusterApi
Run
java -jar kafkawizeclusterapi-41jar
Run
java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar
If application is running you can access UI from http[host][port]kafkawize
10 Chapter 2 Getting Started
Kafkawize Documentation Release 41
210 8 Kafka Connectivity
Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed
bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)
Examples SSL Based Acl (Note of double quotes in the below command if copied properly)
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster
Examples IP Based Acl
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster
bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application
211 9 Final Check
bull Cluster Api is running
bull Metastore (Cassandra or Rdbms) is running and has tables and data
bull UI Api is running
bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)
bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize
Default users passwords and roles
uiuser1user USER
uiuser2user USER
uiuser3user USER
uiuser4user ADMIN
uiuser5user ADMIN
superuseruser SUPERUSER
210 8 Kafka Connectivity 11
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
412 Synchronize Metadata 25413 Admin 26414 Profile 26415 Logout 26
5 Functionalities 27
6 Customize 29
7 Trouble shooting 31
8 Release Notes 33
9 Roadmap 35
10 Contact Me 37
11 Submit a Review 39
12 Indices and tables 41
ii
Kafkawize Documentation Release 41
Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization Read more
Contents
Contents 1
Kafkawize Documentation Release 41
2 Contents
CHAPTER 1
Introduction
11 What is Kafkawize
Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization
12 Background
Let me give some background Apache Kafka is widely being adapted in organizations irrespective of the scale Itrsquosunique features like scalability retention and reliability unlike the traditional messaging platforms makes it stand out
While adapting Kafka you would notice there are few manual activities like creating topics acls updating configu-ration etc There have been few topic management tools in the market which provide automation to certain extenthowever its not complete Assume there is a team requesting for a kafka topic with in an organization They wouldeither request for it through email with a template or any user interface The actual kafka team in the organization ex-ecutes the commands to create topics With Development Test Acceptance and Production environments managingthese configurations by the Kafka team and handling all the requests from various teams would become a hassle asyou see the growth of teams using Kafka It is just not manual activity but Audit time and effort the team has to putin
Apart from that this meta data (Kafka topics acls) which is stored in ZK could be easily stored in a good readableformat and can be used as source of truth
Hence I saw the need of Kafkawize ndash a self service topic management tool to keep ZERO interaction by other Teamswith Kafka team for topics acls or schemas creation by introducing roles and authorizations to users of teams It isdeveloped with a simple front end and backend with Spring boot It saves time and effort governs information avoidsrisk in kafka config loss avoids manual activities and mistakes maintains up to date SOT (source of truth)
3
Kafkawize Documentation Release 41
13 Key features
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
14 41 glimpse
15 How is it developed
Simple technologies have been used to develop Kafkawize Front end with Angular HTML CSS MaterialViewbootstrap theme and Javascript Backend with Java Kafka Admin client libraries Spring boot and Spring securityframeworks First version of Kafkawize has been released in Sep 2018 with basic features and gradually enhanced it
16 Architecture
Kafkawize contains two main Apis (User Interface API and Cluster management API) and a Front end
User Interface Api directly communicates between Frontend UI and Cluster API
Front end is built with AngularJs HTML and Java script
Cluster API acts as middle layer between Kafka brokers and UserInterface API
4 Chapter 1 Introduction
Kafkawize Documentation Release 41
Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client
Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth
Requests from users are directed to cluster api and also data is stored metastore
On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application
17 Git Repositories
UI Api httpsgithubcommuralibasanikafkawize
Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi
18 Developer
Muralidhar Basani
LinkedIn httpswwwlinkedincominmuralibasani
Web httpskafkawizecom
17 Git Repositories 5
Kafkawize Documentation Release 41
6 Chapter 1 Introduction
CHAPTER 2
Getting Started
21 Demo
Lets start with setting up Kafkawize on your WindowsLinuxUnix systems
22 Prerequisites
bull Any IDE like IntelliJNetbeans or Eclipse to build the applications
bull Java installed machines to run the applications
bull Rdbms (like OracleMySQL) or Apache Cassandra
23 1 Download Kafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi
OR
Download Kafkawize 41 bundle here Kafkawize_41
24 2 Download Metastore
If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload
If Rdbms is used as metastore download
Mysql httpsdevmysqlcomdownloads
7
Kafkawize Documentation Release 41
Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml
Or any other Rdbms is ok
25 3 License Key
License key is not required from 41 version
26 4 Build KW Cluster Api Application
This Api does the below
bull Receives and responds to calls from UI Api
bull Connects to Kafka Brokers with Kafka AdminClient Api
bull There is no connection to any metastore
bull Swagger enabled
Unzip kafkawizeclusterapizip and open in any IDE
Configure the application properties (srcmainresources) if port has to be changed else default port is 9343
and run maven command to create a runnable jar
mvn clean package
This should create a jar (kafkawizeclusterapi-41jar) in target dir
Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even
27 5 Configure Kafkawize UI Api Application
This Api does the below
bull Users interact with interface with this Api
bull All the end points in this application either connect to Metastore or Cluster Api or both
Unzip kafkawize-(version)zip and open in any IDE
Configure application properties
In kafkawizesrcmainresourcesapplicationproperties
default port is 9097 if port has to be changed else
serverport9097
If metastore is cassandra (from step 4)
setstore type as cassandra
customdbstoretype=cassandra
8 Chapter 2 Getting Started
Kafkawize Documentation Release 41
configure cassandra host port and keyspace
customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042
If metastore is rdbms (from step 4)
setstore type as rdbms
customdbstoretype=rdbms
bull Install and run Rdbms (like MysqlOracle) and create a db schema or database
configure db properties like below
Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver
271 Configure Cluster Api
configure cluster api host and port details
customclusterapiurlhttplocalhost9343
bull ignore userpwd of cluster api properties
28 6 Metastore setup
281 Metastore Cassandra
In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo
bull Create tables and run insert scripts in Cassandra
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql
28 6 Metastore setup 9
Kafkawize Documentation Release 41
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
282 Metastore Rdbms
In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run MysqlOracle and create a db schema or database
bull Create tables and run insert scripts in Database
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
Run maven command to create a runnable jar
mvn clean package
This should create a jar in target dir (kafkawizetargetkafkawize-41jar)
29 7 Run KW and KWClusterApi
Run
java -jar kafkawizeclusterapi-41jar
Run
java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar
If application is running you can access UI from http[host][port]kafkawize
10 Chapter 2 Getting Started
Kafkawize Documentation Release 41
210 8 Kafka Connectivity
Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed
bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)
Examples SSL Based Acl (Note of double quotes in the below command if copied properly)
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster
Examples IP Based Acl
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster
bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application
211 9 Final Check
bull Cluster Api is running
bull Metastore (Cassandra or Rdbms) is running and has tables and data
bull UI Api is running
bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)
bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize
Default users passwords and roles
uiuser1user USER
uiuser2user USER
uiuser3user USER
uiuser4user ADMIN
uiuser5user ADMIN
superuseruser SUPERUSER
210 8 Kafka Connectivity 11
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization Read more
Contents
Contents 1
Kafkawize Documentation Release 41
2 Contents
CHAPTER 1
Introduction
11 What is Kafkawize
Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization
12 Background
Let me give some background Apache Kafka is widely being adapted in organizations irrespective of the scale Itrsquosunique features like scalability retention and reliability unlike the traditional messaging platforms makes it stand out
While adapting Kafka you would notice there are few manual activities like creating topics acls updating configu-ration etc There have been few topic management tools in the market which provide automation to certain extenthowever its not complete Assume there is a team requesting for a kafka topic with in an organization They wouldeither request for it through email with a template or any user interface The actual kafka team in the organization ex-ecutes the commands to create topics With Development Test Acceptance and Production environments managingthese configurations by the Kafka team and handling all the requests from various teams would become a hassle asyou see the growth of teams using Kafka It is just not manual activity but Audit time and effort the team has to putin
Apart from that this meta data (Kafka topics acls) which is stored in ZK could be easily stored in a good readableformat and can be used as source of truth
Hence I saw the need of Kafkawize ndash a self service topic management tool to keep ZERO interaction by other Teamswith Kafka team for topics acls or schemas creation by introducing roles and authorizations to users of teams It isdeveloped with a simple front end and backend with Spring boot It saves time and effort governs information avoidsrisk in kafka config loss avoids manual activities and mistakes maintains up to date SOT (source of truth)
3
Kafkawize Documentation Release 41
13 Key features
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
14 41 glimpse
15 How is it developed
Simple technologies have been used to develop Kafkawize Front end with Angular HTML CSS MaterialViewbootstrap theme and Javascript Backend with Java Kafka Admin client libraries Spring boot and Spring securityframeworks First version of Kafkawize has been released in Sep 2018 with basic features and gradually enhanced it
16 Architecture
Kafkawize contains two main Apis (User Interface API and Cluster management API) and a Front end
User Interface Api directly communicates between Frontend UI and Cluster API
Front end is built with AngularJs HTML and Java script
Cluster API acts as middle layer between Kafka brokers and UserInterface API
4 Chapter 1 Introduction
Kafkawize Documentation Release 41
Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client
Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth
Requests from users are directed to cluster api and also data is stored metastore
On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application
17 Git Repositories
UI Api httpsgithubcommuralibasanikafkawize
Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi
18 Developer
Muralidhar Basani
LinkedIn httpswwwlinkedincominmuralibasani
Web httpskafkawizecom
17 Git Repositories 5
Kafkawize Documentation Release 41
6 Chapter 1 Introduction
CHAPTER 2
Getting Started
21 Demo
Lets start with setting up Kafkawize on your WindowsLinuxUnix systems
22 Prerequisites
bull Any IDE like IntelliJNetbeans or Eclipse to build the applications
bull Java installed machines to run the applications
bull Rdbms (like OracleMySQL) or Apache Cassandra
23 1 Download Kafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi
OR
Download Kafkawize 41 bundle here Kafkawize_41
24 2 Download Metastore
If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload
If Rdbms is used as metastore download
Mysql httpsdevmysqlcomdownloads
7
Kafkawize Documentation Release 41
Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml
Or any other Rdbms is ok
25 3 License Key
License key is not required from 41 version
26 4 Build KW Cluster Api Application
This Api does the below
bull Receives and responds to calls from UI Api
bull Connects to Kafka Brokers with Kafka AdminClient Api
bull There is no connection to any metastore
bull Swagger enabled
Unzip kafkawizeclusterapizip and open in any IDE
Configure the application properties (srcmainresources) if port has to be changed else default port is 9343
and run maven command to create a runnable jar
mvn clean package
This should create a jar (kafkawizeclusterapi-41jar) in target dir
Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even
27 5 Configure Kafkawize UI Api Application
This Api does the below
bull Users interact with interface with this Api
bull All the end points in this application either connect to Metastore or Cluster Api or both
Unzip kafkawize-(version)zip and open in any IDE
Configure application properties
In kafkawizesrcmainresourcesapplicationproperties
default port is 9097 if port has to be changed else
serverport9097
If metastore is cassandra (from step 4)
setstore type as cassandra
customdbstoretype=cassandra
8 Chapter 2 Getting Started
Kafkawize Documentation Release 41
configure cassandra host port and keyspace
customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042
If metastore is rdbms (from step 4)
setstore type as rdbms
customdbstoretype=rdbms
bull Install and run Rdbms (like MysqlOracle) and create a db schema or database
configure db properties like below
Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver
271 Configure Cluster Api
configure cluster api host and port details
customclusterapiurlhttplocalhost9343
bull ignore userpwd of cluster api properties
28 6 Metastore setup
281 Metastore Cassandra
In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo
bull Create tables and run insert scripts in Cassandra
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql
28 6 Metastore setup 9
Kafkawize Documentation Release 41
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
282 Metastore Rdbms
In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run MysqlOracle and create a db schema or database
bull Create tables and run insert scripts in Database
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
Run maven command to create a runnable jar
mvn clean package
This should create a jar in target dir (kafkawizetargetkafkawize-41jar)
29 7 Run KW and KWClusterApi
Run
java -jar kafkawizeclusterapi-41jar
Run
java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar
If application is running you can access UI from http[host][port]kafkawize
10 Chapter 2 Getting Started
Kafkawize Documentation Release 41
210 8 Kafka Connectivity
Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed
bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)
Examples SSL Based Acl (Note of double quotes in the below command if copied properly)
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster
Examples IP Based Acl
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster
bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application
211 9 Final Check
bull Cluster Api is running
bull Metastore (Cassandra or Rdbms) is running and has tables and data
bull UI Api is running
bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)
bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize
Default users passwords and roles
uiuser1user USER
uiuser2user USER
uiuser3user USER
uiuser4user ADMIN
uiuser5user ADMIN
superuseruser SUPERUSER
210 8 Kafka Connectivity 11
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
2 Contents
CHAPTER 1
Introduction
11 What is Kafkawize
Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization
12 Background
Let me give some background Apache Kafka is widely being adapted in organizations irrespective of the scale Itrsquosunique features like scalability retention and reliability unlike the traditional messaging platforms makes it stand out
While adapting Kafka you would notice there are few manual activities like creating topics acls updating configu-ration etc There have been few topic management tools in the market which provide automation to certain extenthowever its not complete Assume there is a team requesting for a kafka topic with in an organization They wouldeither request for it through email with a template or any user interface The actual kafka team in the organization ex-ecutes the commands to create topics With Development Test Acceptance and Production environments managingthese configurations by the Kafka team and handling all the requests from various teams would become a hassle asyou see the growth of teams using Kafka It is just not manual activity but Audit time and effort the team has to putin
Apart from that this meta data (Kafka topics acls) which is stored in ZK could be easily stored in a good readableformat and can be used as source of truth
Hence I saw the need of Kafkawize ndash a self service topic management tool to keep ZERO interaction by other Teamswith Kafka team for topics acls or schemas creation by introducing roles and authorizations to users of teams It isdeveloped with a simple front end and backend with Spring boot It saves time and effort governs information avoidsrisk in kafka config loss avoids manual activities and mistakes maintains up to date SOT (source of truth)
3
Kafkawize Documentation Release 41
13 Key features
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
14 41 glimpse
15 How is it developed
Simple technologies have been used to develop Kafkawize Front end with Angular HTML CSS MaterialViewbootstrap theme and Javascript Backend with Java Kafka Admin client libraries Spring boot and Spring securityframeworks First version of Kafkawize has been released in Sep 2018 with basic features and gradually enhanced it
16 Architecture
Kafkawize contains two main Apis (User Interface API and Cluster management API) and a Front end
User Interface Api directly communicates between Frontend UI and Cluster API
Front end is built with AngularJs HTML and Java script
Cluster API acts as middle layer between Kafka brokers and UserInterface API
4 Chapter 1 Introduction
Kafkawize Documentation Release 41
Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client
Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth
Requests from users are directed to cluster api and also data is stored metastore
On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application
17 Git Repositories
UI Api httpsgithubcommuralibasanikafkawize
Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi
18 Developer
Muralidhar Basani
LinkedIn httpswwwlinkedincominmuralibasani
Web httpskafkawizecom
17 Git Repositories 5
Kafkawize Documentation Release 41
6 Chapter 1 Introduction
CHAPTER 2
Getting Started
21 Demo
Lets start with setting up Kafkawize on your WindowsLinuxUnix systems
22 Prerequisites
bull Any IDE like IntelliJNetbeans or Eclipse to build the applications
bull Java installed machines to run the applications
bull Rdbms (like OracleMySQL) or Apache Cassandra
23 1 Download Kafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi
OR
Download Kafkawize 41 bundle here Kafkawize_41
24 2 Download Metastore
If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload
If Rdbms is used as metastore download
Mysql httpsdevmysqlcomdownloads
7
Kafkawize Documentation Release 41
Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml
Or any other Rdbms is ok
25 3 License Key
License key is not required from 41 version
26 4 Build KW Cluster Api Application
This Api does the below
bull Receives and responds to calls from UI Api
bull Connects to Kafka Brokers with Kafka AdminClient Api
bull There is no connection to any metastore
bull Swagger enabled
Unzip kafkawizeclusterapizip and open in any IDE
Configure the application properties (srcmainresources) if port has to be changed else default port is 9343
and run maven command to create a runnable jar
mvn clean package
This should create a jar (kafkawizeclusterapi-41jar) in target dir
Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even
27 5 Configure Kafkawize UI Api Application
This Api does the below
bull Users interact with interface with this Api
bull All the end points in this application either connect to Metastore or Cluster Api or both
Unzip kafkawize-(version)zip and open in any IDE
Configure application properties
In kafkawizesrcmainresourcesapplicationproperties
default port is 9097 if port has to be changed else
serverport9097
If metastore is cassandra (from step 4)
setstore type as cassandra
customdbstoretype=cassandra
8 Chapter 2 Getting Started
Kafkawize Documentation Release 41
configure cassandra host port and keyspace
customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042
If metastore is rdbms (from step 4)
setstore type as rdbms
customdbstoretype=rdbms
bull Install and run Rdbms (like MysqlOracle) and create a db schema or database
configure db properties like below
Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver
271 Configure Cluster Api
configure cluster api host and port details
customclusterapiurlhttplocalhost9343
bull ignore userpwd of cluster api properties
28 6 Metastore setup
281 Metastore Cassandra
In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo
bull Create tables and run insert scripts in Cassandra
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql
28 6 Metastore setup 9
Kafkawize Documentation Release 41
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
282 Metastore Rdbms
In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run MysqlOracle and create a db schema or database
bull Create tables and run insert scripts in Database
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
Run maven command to create a runnable jar
mvn clean package
This should create a jar in target dir (kafkawizetargetkafkawize-41jar)
29 7 Run KW and KWClusterApi
Run
java -jar kafkawizeclusterapi-41jar
Run
java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar
If application is running you can access UI from http[host][port]kafkawize
10 Chapter 2 Getting Started
Kafkawize Documentation Release 41
210 8 Kafka Connectivity
Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed
bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)
Examples SSL Based Acl (Note of double quotes in the below command if copied properly)
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster
Examples IP Based Acl
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster
bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application
211 9 Final Check
bull Cluster Api is running
bull Metastore (Cassandra or Rdbms) is running and has tables and data
bull UI Api is running
bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)
bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize
Default users passwords and roles
uiuser1user USER
uiuser2user USER
uiuser3user USER
uiuser4user ADMIN
uiuser5user ADMIN
superuseruser SUPERUSER
210 8 Kafka Connectivity 11
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
CHAPTER 1
Introduction
11 What is Kafkawize
Kafkawize is a Self service Apache Kafka Topic Management toolportal It is a web application which automates theprocess of creating and browsing Kafka topics acls schemas by introducing rolesauthorizations to users of variousteams of an organization
12 Background
Let me give some background Apache Kafka is widely being adapted in organizations irrespective of the scale Itrsquosunique features like scalability retention and reliability unlike the traditional messaging platforms makes it stand out
While adapting Kafka you would notice there are few manual activities like creating topics acls updating configu-ration etc There have been few topic management tools in the market which provide automation to certain extenthowever its not complete Assume there is a team requesting for a kafka topic with in an organization They wouldeither request for it through email with a template or any user interface The actual kafka team in the organization ex-ecutes the commands to create topics With Development Test Acceptance and Production environments managingthese configurations by the Kafka team and handling all the requests from various teams would become a hassle asyou see the growth of teams using Kafka It is just not manual activity but Audit time and effort the team has to putin
Apart from that this meta data (Kafka topics acls) which is stored in ZK could be easily stored in a good readableformat and can be used as source of truth
Hence I saw the need of Kafkawize ndash a self service topic management tool to keep ZERO interaction by other Teamswith Kafka team for topics acls or schemas creation by introducing roles and authorizations to users of teams It isdeveloped with a simple front end and backend with Spring boot It saves time and effort governs information avoidsrisk in kafka config loss avoids manual activities and mistakes maintains up to date SOT (source of truth)
3
Kafkawize Documentation Release 41
13 Key features
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
14 41 glimpse
15 How is it developed
Simple technologies have been used to develop Kafkawize Front end with Angular HTML CSS MaterialViewbootstrap theme and Javascript Backend with Java Kafka Admin client libraries Spring boot and Spring securityframeworks First version of Kafkawize has been released in Sep 2018 with basic features and gradually enhanced it
16 Architecture
Kafkawize contains two main Apis (User Interface API and Cluster management API) and a Front end
User Interface Api directly communicates between Frontend UI and Cluster API
Front end is built with AngularJs HTML and Java script
Cluster API acts as middle layer between Kafka brokers and UserInterface API
4 Chapter 1 Introduction
Kafkawize Documentation Release 41
Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client
Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth
Requests from users are directed to cluster api and also data is stored metastore
On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application
17 Git Repositories
UI Api httpsgithubcommuralibasanikafkawize
Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi
18 Developer
Muralidhar Basani
LinkedIn httpswwwlinkedincominmuralibasani
Web httpskafkawizecom
17 Git Repositories 5
Kafkawize Documentation Release 41
6 Chapter 1 Introduction
CHAPTER 2
Getting Started
21 Demo
Lets start with setting up Kafkawize on your WindowsLinuxUnix systems
22 Prerequisites
bull Any IDE like IntelliJNetbeans or Eclipse to build the applications
bull Java installed machines to run the applications
bull Rdbms (like OracleMySQL) or Apache Cassandra
23 1 Download Kafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi
OR
Download Kafkawize 41 bundle here Kafkawize_41
24 2 Download Metastore
If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload
If Rdbms is used as metastore download
Mysql httpsdevmysqlcomdownloads
7
Kafkawize Documentation Release 41
Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml
Or any other Rdbms is ok
25 3 License Key
License key is not required from 41 version
26 4 Build KW Cluster Api Application
This Api does the below
bull Receives and responds to calls from UI Api
bull Connects to Kafka Brokers with Kafka AdminClient Api
bull There is no connection to any metastore
bull Swagger enabled
Unzip kafkawizeclusterapizip and open in any IDE
Configure the application properties (srcmainresources) if port has to be changed else default port is 9343
and run maven command to create a runnable jar
mvn clean package
This should create a jar (kafkawizeclusterapi-41jar) in target dir
Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even
27 5 Configure Kafkawize UI Api Application
This Api does the below
bull Users interact with interface with this Api
bull All the end points in this application either connect to Metastore or Cluster Api or both
Unzip kafkawize-(version)zip and open in any IDE
Configure application properties
In kafkawizesrcmainresourcesapplicationproperties
default port is 9097 if port has to be changed else
serverport9097
If metastore is cassandra (from step 4)
setstore type as cassandra
customdbstoretype=cassandra
8 Chapter 2 Getting Started
Kafkawize Documentation Release 41
configure cassandra host port and keyspace
customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042
If metastore is rdbms (from step 4)
setstore type as rdbms
customdbstoretype=rdbms
bull Install and run Rdbms (like MysqlOracle) and create a db schema or database
configure db properties like below
Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver
271 Configure Cluster Api
configure cluster api host and port details
customclusterapiurlhttplocalhost9343
bull ignore userpwd of cluster api properties
28 6 Metastore setup
281 Metastore Cassandra
In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo
bull Create tables and run insert scripts in Cassandra
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql
28 6 Metastore setup 9
Kafkawize Documentation Release 41
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
282 Metastore Rdbms
In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run MysqlOracle and create a db schema or database
bull Create tables and run insert scripts in Database
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
Run maven command to create a runnable jar
mvn clean package
This should create a jar in target dir (kafkawizetargetkafkawize-41jar)
29 7 Run KW and KWClusterApi
Run
java -jar kafkawizeclusterapi-41jar
Run
java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar
If application is running you can access UI from http[host][port]kafkawize
10 Chapter 2 Getting Started
Kafkawize Documentation Release 41
210 8 Kafka Connectivity
Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed
bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)
Examples SSL Based Acl (Note of double quotes in the below command if copied properly)
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster
Examples IP Based Acl
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster
bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application
211 9 Final Check
bull Cluster Api is running
bull Metastore (Cassandra or Rdbms) is running and has tables and data
bull UI Api is running
bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)
bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize
Default users passwords and roles
uiuser1user USER
uiuser2user USER
uiuser3user USER
uiuser4user ADMIN
uiuser5user ADMIN
superuseruser SUPERUSER
210 8 Kafka Connectivity 11
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
13 Key features
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
14 41 glimpse
15 How is it developed
Simple technologies have been used to develop Kafkawize Front end with Angular HTML CSS MaterialViewbootstrap theme and Javascript Backend with Java Kafka Admin client libraries Spring boot and Spring securityframeworks First version of Kafkawize has been released in Sep 2018 with basic features and gradually enhanced it
16 Architecture
Kafkawize contains two main Apis (User Interface API and Cluster management API) and a Front end
User Interface Api directly communicates between Frontend UI and Cluster API
Front end is built with AngularJs HTML and Java script
Cluster API acts as middle layer between Kafka brokers and UserInterface API
4 Chapter 1 Introduction
Kafkawize Documentation Release 41
Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client
Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth
Requests from users are directed to cluster api and also data is stored metastore
On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application
17 Git Repositories
UI Api httpsgithubcommuralibasanikafkawize
Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi
18 Developer
Muralidhar Basani
LinkedIn httpswwwlinkedincominmuralibasani
Web httpskafkawizecom
17 Git Repositories 5
Kafkawize Documentation Release 41
6 Chapter 1 Introduction
CHAPTER 2
Getting Started
21 Demo
Lets start with setting up Kafkawize on your WindowsLinuxUnix systems
22 Prerequisites
bull Any IDE like IntelliJNetbeans or Eclipse to build the applications
bull Java installed machines to run the applications
bull Rdbms (like OracleMySQL) or Apache Cassandra
23 1 Download Kafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi
OR
Download Kafkawize 41 bundle here Kafkawize_41
24 2 Download Metastore
If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload
If Rdbms is used as metastore download
Mysql httpsdevmysqlcomdownloads
7
Kafkawize Documentation Release 41
Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml
Or any other Rdbms is ok
25 3 License Key
License key is not required from 41 version
26 4 Build KW Cluster Api Application
This Api does the below
bull Receives and responds to calls from UI Api
bull Connects to Kafka Brokers with Kafka AdminClient Api
bull There is no connection to any metastore
bull Swagger enabled
Unzip kafkawizeclusterapizip and open in any IDE
Configure the application properties (srcmainresources) if port has to be changed else default port is 9343
and run maven command to create a runnable jar
mvn clean package
This should create a jar (kafkawizeclusterapi-41jar) in target dir
Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even
27 5 Configure Kafkawize UI Api Application
This Api does the below
bull Users interact with interface with this Api
bull All the end points in this application either connect to Metastore or Cluster Api or both
Unzip kafkawize-(version)zip and open in any IDE
Configure application properties
In kafkawizesrcmainresourcesapplicationproperties
default port is 9097 if port has to be changed else
serverport9097
If metastore is cassandra (from step 4)
setstore type as cassandra
customdbstoretype=cassandra
8 Chapter 2 Getting Started
Kafkawize Documentation Release 41
configure cassandra host port and keyspace
customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042
If metastore is rdbms (from step 4)
setstore type as rdbms
customdbstoretype=rdbms
bull Install and run Rdbms (like MysqlOracle) and create a db schema or database
configure db properties like below
Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver
271 Configure Cluster Api
configure cluster api host and port details
customclusterapiurlhttplocalhost9343
bull ignore userpwd of cluster api properties
28 6 Metastore setup
281 Metastore Cassandra
In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo
bull Create tables and run insert scripts in Cassandra
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql
28 6 Metastore setup 9
Kafkawize Documentation Release 41
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
282 Metastore Rdbms
In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run MysqlOracle and create a db schema or database
bull Create tables and run insert scripts in Database
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
Run maven command to create a runnable jar
mvn clean package
This should create a jar in target dir (kafkawizetargetkafkawize-41jar)
29 7 Run KW and KWClusterApi
Run
java -jar kafkawizeclusterapi-41jar
Run
java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar
If application is running you can access UI from http[host][port]kafkawize
10 Chapter 2 Getting Started
Kafkawize Documentation Release 41
210 8 Kafka Connectivity
Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed
bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)
Examples SSL Based Acl (Note of double quotes in the below command if copied properly)
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster
Examples IP Based Acl
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster
bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application
211 9 Final Check
bull Cluster Api is running
bull Metastore (Cassandra or Rdbms) is running and has tables and data
bull UI Api is running
bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)
bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize
Default users passwords and roles
uiuser1user USER
uiuser2user USER
uiuser3user USER
uiuser4user ADMIN
uiuser5user ADMIN
superuseruser SUPERUSER
210 8 Kafka Connectivity 11
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
Cluster API creates Kafka Admin Client and executes the requests for Topic Acls or Schema registry Kafka Adminclient libraries are used to create the client
Apache Cassandra Or RDBMS(MySql for ex) datastore stores all the meta information like users teams topicRe-quests request and execution data from all the users and to maintain source of truth
Requests from users are directed to cluster api and also data is stored metastore
On the backend side Spring Security Spring Boot frameworks Hibernate are used to develop this application
17 Git Repositories
UI Api httpsgithubcommuralibasanikafkawize
Cluster Api httpsgithubcommuralibasanikafkawizeclusterapi
18 Developer
Muralidhar Basani
LinkedIn httpswwwlinkedincominmuralibasani
Web httpskafkawizecom
17 Git Repositories 5
Kafkawize Documentation Release 41
6 Chapter 1 Introduction
CHAPTER 2
Getting Started
21 Demo
Lets start with setting up Kafkawize on your WindowsLinuxUnix systems
22 Prerequisites
bull Any IDE like IntelliJNetbeans or Eclipse to build the applications
bull Java installed machines to run the applications
bull Rdbms (like OracleMySQL) or Apache Cassandra
23 1 Download Kafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi
OR
Download Kafkawize 41 bundle here Kafkawize_41
24 2 Download Metastore
If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload
If Rdbms is used as metastore download
Mysql httpsdevmysqlcomdownloads
7
Kafkawize Documentation Release 41
Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml
Or any other Rdbms is ok
25 3 License Key
License key is not required from 41 version
26 4 Build KW Cluster Api Application
This Api does the below
bull Receives and responds to calls from UI Api
bull Connects to Kafka Brokers with Kafka AdminClient Api
bull There is no connection to any metastore
bull Swagger enabled
Unzip kafkawizeclusterapizip and open in any IDE
Configure the application properties (srcmainresources) if port has to be changed else default port is 9343
and run maven command to create a runnable jar
mvn clean package
This should create a jar (kafkawizeclusterapi-41jar) in target dir
Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even
27 5 Configure Kafkawize UI Api Application
This Api does the below
bull Users interact with interface with this Api
bull All the end points in this application either connect to Metastore or Cluster Api or both
Unzip kafkawize-(version)zip and open in any IDE
Configure application properties
In kafkawizesrcmainresourcesapplicationproperties
default port is 9097 if port has to be changed else
serverport9097
If metastore is cassandra (from step 4)
setstore type as cassandra
customdbstoretype=cassandra
8 Chapter 2 Getting Started
Kafkawize Documentation Release 41
configure cassandra host port and keyspace
customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042
If metastore is rdbms (from step 4)
setstore type as rdbms
customdbstoretype=rdbms
bull Install and run Rdbms (like MysqlOracle) and create a db schema or database
configure db properties like below
Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver
271 Configure Cluster Api
configure cluster api host and port details
customclusterapiurlhttplocalhost9343
bull ignore userpwd of cluster api properties
28 6 Metastore setup
281 Metastore Cassandra
In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo
bull Create tables and run insert scripts in Cassandra
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql
28 6 Metastore setup 9
Kafkawize Documentation Release 41
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
282 Metastore Rdbms
In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run MysqlOracle and create a db schema or database
bull Create tables and run insert scripts in Database
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
Run maven command to create a runnable jar
mvn clean package
This should create a jar in target dir (kafkawizetargetkafkawize-41jar)
29 7 Run KW and KWClusterApi
Run
java -jar kafkawizeclusterapi-41jar
Run
java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar
If application is running you can access UI from http[host][port]kafkawize
10 Chapter 2 Getting Started
Kafkawize Documentation Release 41
210 8 Kafka Connectivity
Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed
bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)
Examples SSL Based Acl (Note of double quotes in the below command if copied properly)
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster
Examples IP Based Acl
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster
bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application
211 9 Final Check
bull Cluster Api is running
bull Metastore (Cassandra or Rdbms) is running and has tables and data
bull UI Api is running
bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)
bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize
Default users passwords and roles
uiuser1user USER
uiuser2user USER
uiuser3user USER
uiuser4user ADMIN
uiuser5user ADMIN
superuseruser SUPERUSER
210 8 Kafka Connectivity 11
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
6 Chapter 1 Introduction
CHAPTER 2
Getting Started
21 Demo
Lets start with setting up Kafkawize on your WindowsLinuxUnix systems
22 Prerequisites
bull Any IDE like IntelliJNetbeans or Eclipse to build the applications
bull Java installed machines to run the applications
bull Rdbms (like OracleMySQL) or Apache Cassandra
23 1 Download Kafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi
OR
Download Kafkawize 41 bundle here Kafkawize_41
24 2 Download Metastore
If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload
If Rdbms is used as metastore download
Mysql httpsdevmysqlcomdownloads
7
Kafkawize Documentation Release 41
Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml
Or any other Rdbms is ok
25 3 License Key
License key is not required from 41 version
26 4 Build KW Cluster Api Application
This Api does the below
bull Receives and responds to calls from UI Api
bull Connects to Kafka Brokers with Kafka AdminClient Api
bull There is no connection to any metastore
bull Swagger enabled
Unzip kafkawizeclusterapizip and open in any IDE
Configure the application properties (srcmainresources) if port has to be changed else default port is 9343
and run maven command to create a runnable jar
mvn clean package
This should create a jar (kafkawizeclusterapi-41jar) in target dir
Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even
27 5 Configure Kafkawize UI Api Application
This Api does the below
bull Users interact with interface with this Api
bull All the end points in this application either connect to Metastore or Cluster Api or both
Unzip kafkawize-(version)zip and open in any IDE
Configure application properties
In kafkawizesrcmainresourcesapplicationproperties
default port is 9097 if port has to be changed else
serverport9097
If metastore is cassandra (from step 4)
setstore type as cassandra
customdbstoretype=cassandra
8 Chapter 2 Getting Started
Kafkawize Documentation Release 41
configure cassandra host port and keyspace
customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042
If metastore is rdbms (from step 4)
setstore type as rdbms
customdbstoretype=rdbms
bull Install and run Rdbms (like MysqlOracle) and create a db schema or database
configure db properties like below
Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver
271 Configure Cluster Api
configure cluster api host and port details
customclusterapiurlhttplocalhost9343
bull ignore userpwd of cluster api properties
28 6 Metastore setup
281 Metastore Cassandra
In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo
bull Create tables and run insert scripts in Cassandra
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql
28 6 Metastore setup 9
Kafkawize Documentation Release 41
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
282 Metastore Rdbms
In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run MysqlOracle and create a db schema or database
bull Create tables and run insert scripts in Database
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
Run maven command to create a runnable jar
mvn clean package
This should create a jar in target dir (kafkawizetargetkafkawize-41jar)
29 7 Run KW and KWClusterApi
Run
java -jar kafkawizeclusterapi-41jar
Run
java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar
If application is running you can access UI from http[host][port]kafkawize
10 Chapter 2 Getting Started
Kafkawize Documentation Release 41
210 8 Kafka Connectivity
Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed
bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)
Examples SSL Based Acl (Note of double quotes in the below command if copied properly)
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster
Examples IP Based Acl
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster
bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application
211 9 Final Check
bull Cluster Api is running
bull Metastore (Cassandra or Rdbms) is running and has tables and data
bull UI Api is running
bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)
bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize
Default users passwords and roles
uiuser1user USER
uiuser2user USER
uiuser3user USER
uiuser4user ADMIN
uiuser5user ADMIN
superuseruser SUPERUSER
210 8 Kafka Connectivity 11
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
CHAPTER 2
Getting Started
21 Demo
Lets start with setting up Kafkawize on your WindowsLinuxUnix systems
22 Prerequisites
bull Any IDE like IntelliJNetbeans or Eclipse to build the applications
bull Java installed machines to run the applications
bull Rdbms (like OracleMySQL) or Apache Cassandra
23 1 Download Kafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawize
Download the latest version (41) or clone from the git repo httpsgithubcommuralibasanikafkawizeclusterapi
OR
Download Kafkawize 41 bundle here Kafkawize_41
24 2 Download Metastore
If Cassandra is used as metastore you can download from httpcassandraapacheorgdownload
If Rdbms is used as metastore download
Mysql httpsdevmysqlcomdownloads
7
Kafkawize Documentation Release 41
Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml
Or any other Rdbms is ok
25 3 License Key
License key is not required from 41 version
26 4 Build KW Cluster Api Application
This Api does the below
bull Receives and responds to calls from UI Api
bull Connects to Kafka Brokers with Kafka AdminClient Api
bull There is no connection to any metastore
bull Swagger enabled
Unzip kafkawizeclusterapizip and open in any IDE
Configure the application properties (srcmainresources) if port has to be changed else default port is 9343
and run maven command to create a runnable jar
mvn clean package
This should create a jar (kafkawizeclusterapi-41jar) in target dir
Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even
27 5 Configure Kafkawize UI Api Application
This Api does the below
bull Users interact with interface with this Api
bull All the end points in this application either connect to Metastore or Cluster Api or both
Unzip kafkawize-(version)zip and open in any IDE
Configure application properties
In kafkawizesrcmainresourcesapplicationproperties
default port is 9097 if port has to be changed else
serverport9097
If metastore is cassandra (from step 4)
setstore type as cassandra
customdbstoretype=cassandra
8 Chapter 2 Getting Started
Kafkawize Documentation Release 41
configure cassandra host port and keyspace
customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042
If metastore is rdbms (from step 4)
setstore type as rdbms
customdbstoretype=rdbms
bull Install and run Rdbms (like MysqlOracle) and create a db schema or database
configure db properties like below
Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver
271 Configure Cluster Api
configure cluster api host and port details
customclusterapiurlhttplocalhost9343
bull ignore userpwd of cluster api properties
28 6 Metastore setup
281 Metastore Cassandra
In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo
bull Create tables and run insert scripts in Cassandra
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql
28 6 Metastore setup 9
Kafkawize Documentation Release 41
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
282 Metastore Rdbms
In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run MysqlOracle and create a db schema or database
bull Create tables and run insert scripts in Database
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
Run maven command to create a runnable jar
mvn clean package
This should create a jar in target dir (kafkawizetargetkafkawize-41jar)
29 7 Run KW and KWClusterApi
Run
java -jar kafkawizeclusterapi-41jar
Run
java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar
If application is running you can access UI from http[host][port]kafkawize
10 Chapter 2 Getting Started
Kafkawize Documentation Release 41
210 8 Kafka Connectivity
Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed
bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)
Examples SSL Based Acl (Note of double quotes in the below command if copied properly)
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster
Examples IP Based Acl
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster
bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application
211 9 Final Check
bull Cluster Api is running
bull Metastore (Cassandra or Rdbms) is running and has tables and data
bull UI Api is running
bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)
bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize
Default users passwords and roles
uiuser1user USER
uiuser2user USER
uiuser3user USER
uiuser4user ADMIN
uiuser5user ADMIN
superuseruser SUPERUSER
210 8 Kafka Connectivity 11
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
Oracle httpswwworaclecomdatabasetechnologiesoracle-database-software-downloadshtml
Or any other Rdbms is ok
25 3 License Key
License key is not required from 41 version
26 4 Build KW Cluster Api Application
This Api does the below
bull Receives and responds to calls from UI Api
bull Connects to Kafka Brokers with Kafka AdminClient Api
bull There is no connection to any metastore
bull Swagger enabled
Unzip kafkawizeclusterapizip and open in any IDE
Configure the application properties (srcmainresources) if port has to be changed else default port is 9343
and run maven command to create a runnable jar
mvn clean package
This should create a jar (kafkawizeclusterapi-41jar) in target dir
Kafka connectivity between ClusterApi application and Kafka cluster if SSL connection needs to be configured con-figure ldquoenvironmentrdquoconnect_with_ssl_kafkacluster in application properties to true and configure the other keystoreproperties even
27 5 Configure Kafkawize UI Api Application
This Api does the below
bull Users interact with interface with this Api
bull All the end points in this application either connect to Metastore or Cluster Api or both
Unzip kafkawize-(version)zip and open in any IDE
Configure application properties
In kafkawizesrcmainresourcesapplicationproperties
default port is 9097 if port has to be changed else
serverport9097
If metastore is cassandra (from step 4)
setstore type as cassandra
customdbstoretype=cassandra
8 Chapter 2 Getting Started
Kafkawize Documentation Release 41
configure cassandra host port and keyspace
customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042
If metastore is rdbms (from step 4)
setstore type as rdbms
customdbstoretype=rdbms
bull Install and run Rdbms (like MysqlOracle) and create a db schema or database
configure db properties like below
Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver
271 Configure Cluster Api
configure cluster api host and port details
customclusterapiurlhttplocalhost9343
bull ignore userpwd of cluster api properties
28 6 Metastore setup
281 Metastore Cassandra
In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo
bull Create tables and run insert scripts in Cassandra
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql
28 6 Metastore setup 9
Kafkawize Documentation Release 41
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
282 Metastore Rdbms
In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run MysqlOracle and create a db schema or database
bull Create tables and run insert scripts in Database
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
Run maven command to create a runnable jar
mvn clean package
This should create a jar in target dir (kafkawizetargetkafkawize-41jar)
29 7 Run KW and KWClusterApi
Run
java -jar kafkawizeclusterapi-41jar
Run
java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar
If application is running you can access UI from http[host][port]kafkawize
10 Chapter 2 Getting Started
Kafkawize Documentation Release 41
210 8 Kafka Connectivity
Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed
bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)
Examples SSL Based Acl (Note of double quotes in the below command if copied properly)
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster
Examples IP Based Acl
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster
bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application
211 9 Final Check
bull Cluster Api is running
bull Metastore (Cassandra or Rdbms) is running and has tables and data
bull UI Api is running
bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)
bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize
Default users passwords and roles
uiuser1user USER
uiuser2user USER
uiuser3user USER
uiuser4user ADMIN
uiuser5user ADMIN
superuseruser SUPERUSER
210 8 Kafka Connectivity 11
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
configure cassandra host port and keyspace
customcassandradbkeyspacekafkamanagementapicustomcassandradburllocalhostcustomcassandradbport9042
If metastore is rdbms (from step 4)
setstore type as rdbms
customdbstoretype=rdbms
bull Install and run Rdbms (like MysqlOracle) and create a db schema or database
configure db properties like below
Spring JPA propertiesspringdatasourceurl=jdbcmysqllocalhost3306kafkametadbautoReconnect=trueamprarr˓useUnicode=trueampuseJDBCCompliantTimezoneShift=trueampuseLegacyDatetimeCode=falseamprarr˓serverTimezone=UTCspringdatasourceusername=kafkauserspringdatasourcepassword=kafkauser123springdatasourcedriverclass=commysqlcjjdbcDriver
271 Configure Cluster Api
configure cluster api host and port details
customclusterapiurlhttplocalhost9343
bull ignore userpwd of cluster api properties
28 6 Metastore setup
281 Metastore Cassandra
In applicationproperties (kafkawizesrcmainresourcesapplicationproperties) configure cus-tomdbscriptsexecution=auto or manual to get the required database scripts executed
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run Cassandra and create a keyspace lsquokafkamanagementapirsquo
bull Create tables and run insert scripts in Cassandra
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandracreatecassandrasql
28 6 Metastore setup 9
Kafkawize Documentation Release 41
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
282 Metastore Rdbms
In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run MysqlOracle and create a db schema or database
bull Create tables and run insert scripts in Database
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
Run maven command to create a runnable jar
mvn clean package
This should create a jar in target dir (kafkawizetargetkafkawize-41jar)
29 7 Run KW and KWClusterApi
Run
java -jar kafkawizeclusterapi-41jar
Run
java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar
If application is running you can access UI from http[host][port]kafkawize
10 Chapter 2 Getting Started
Kafkawize Documentation Release 41
210 8 Kafka Connectivity
Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed
bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)
Examples SSL Based Acl (Note of double quotes in the below command if copied properly)
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster
Examples IP Based Acl
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster
bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application
211 9 Final Check
bull Cluster Api is running
bull Metastore (Cassandra or Rdbms) is running and has tables and data
bull UI Api is running
bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)
bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize
Default users passwords and roles
uiuser1user USER
uiuser2user USER
uiuser3user USER
uiuser4user ADMIN
uiuser5user ADMIN
superuseruser SUPERUSER
210 8 Kafka Connectivity 11
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
kafkawizekafkawize-websrcmainresourcesscriptsbasecassandrainsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
282 Metastore Rdbms
In applicationproperties configure customdbscriptsexecution=auto or manual to get the required database scriptsexecuted
Configure customdbscriptsdropall_recreate=false or true to drop all the metadata scripts and recreate This propertyis only valid if the above property is set to auto
Configure in application properties
customdbscriptsexecution = auto or manualcustomdbscriptsdropall_recreate = false or true
If the customdbscriptsexecution property is set to auto all the scripts will be executed automatically
If the customdbscriptsexecution property is set to manual all the scripts should be executed manually
bull Install and run MysqlOracle and create a db schema or database
bull Create tables and run insert scripts in Database
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsddl-jdbcsql
kafkawizekafkawize-websrcmainresourcesscriptsbaserdbmsinsertdatasql
(Scripts available in kafkawize-41zip)
bull Above scripts will create tables and insert initial set of Environments Teams and Users which you can deleteanytime from UI
Run maven command to create a runnable jar
mvn clean package
This should create a jar in target dir (kafkawizetargetkafkawize-41jar)
29 7 Run KW and KWClusterApi
Run
java -jar kafkawizeclusterapi-41jar
Run
java -jar springconfiglocation=classpathapplicationproperties kafkawize-41jar
If application is running you can access UI from http[host][port]kafkawize
10 Chapter 2 Getting Started
Kafkawize Documentation Release 41
210 8 Kafka Connectivity
Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed
bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)
Examples SSL Based Acl (Note of double quotes in the below command if copied properly)
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster
Examples IP Based Acl
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster
bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application
211 9 Final Check
bull Cluster Api is running
bull Metastore (Cassandra or Rdbms) is running and has tables and data
bull UI Api is running
bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)
bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize
Default users passwords and roles
uiuser1user USER
uiuser2user USER
uiuser3user USER
uiuser4user ADMIN
uiuser5user ADMIN
superuseruser SUPERUSER
210 8 Kafka Connectivity 11
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
210 8 Kafka Connectivity
Cluster Api Application connects to Kafka brokers with Kafka AdminClient Api and needs Describe access on alltopics through the cluster Hence the below wildcard acl has to be executed
bull If Acls are enabled on Kafka brokers make sure ldquoCluster Apirdquo application host is authorized to read topics (Aread Acl is enough on the topic)
Examples SSL Based Acl (Note of double quotes in the below command if copied properly)
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principalUserCN=MOOU=MOO=WAL=WAST=WAC=HOrdquo ndashoperation All ndashtopic ldquordquo ndashcluster
Examples IP Based Acl
binkafka-acls ndashauthorizer-properties zookeeperconnect=localhost2181 ndashadd ndashallow-principal Userrdquordquondashallow-host 127001 ndashoperation All ndashtopic ldquordquo ndashcluster
bull If SASLSSL is configured make sure they right properties are configured in AdminClient properties in ClusterApi application
211 9 Final Check
bull Cluster Api is running
bull Metastore (Cassandra or Rdbms) is running and has tables and data
bull UI Api is running
bull Cluster Api is authorized to read topics and acls on topics information(Acls should be configured)
bull Access UI from http[host][port]kafkawize where host and port are UI Api application Example httplocalhost9097kafkawize
Default users passwords and roles
uiuser1user USER
uiuser2user USER
uiuser3user USER
uiuser4user ADMIN
uiuser5user ADMIN
superuseruser SUPERUSER
210 8 Kafka Connectivity 11
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
12 Chapter 2 Getting Started
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
CHAPTER 3
Migration Kafkawize from 35 to 40
1 Follow the instructions in getting started page and download both Kw and KwClusterApi 2 Set the below properties
customdbscriptsexecution = manualcustomdbscriptsdropall_recreate = false
4 Make sure to update the other required config for application properties
customkafkawizeversion=lt41gt copy from the new downloaded zip
13
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
14 Chapter 3 Migration Kafkawize from 35 to 40
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
CHAPTER 4
Features
Kafkawize is built with the key features as below
- 4 eyed principle - Requesting and Approving topicsaclsschemas- Spring based security- Fully automated- Browse Acls Producers and Consumers- Synchronize Source of truth with Meta store- Support for Cassandra or an Rdbms
41 Security
There are two main applications of Kafkawize UiApi and ClusterApi There is no security applied on Cluster ApiEnd points are exposed to anyone who can reach the host and swagger is enabled too Users can directly postgetrequests to this application
On the UI Api Spring security is enabled As the application boots it loads all the users from databasecassandra intoits memory and would be referred to it
Security on the Kafka Broker is out of scope for Kafkawize and it should be handled by you however for Kafkawizeto connect to Kafka broker AclsAdminClient properties need to be provided to avoid authorization issues
42 Teams and Role Management
Kafkawize comes with 3 mains roles
- USER- ADMIN- SUPERUSER
lsquoUserrsquo has the rights to request and browse data
15
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
lsquoAdminrsquo has the all the rights as USER and approve all requests
lsquoSuperuserrsquo has all the rights as ADMIN and synchronize cluster data with metadata For ex assign a team to a topicor acl
A team can have multiple users with the above roles Depending on the role of a user he is authorized to perform therelevant actions on the portal User receives an authorization error if heshe performs an action if not authorized to
43 Login
Portal can be accessed only if the user logs in New users are created by Super users
After a user is logged in a session is created for the user and depending on the role heshe is authorized to performcertain actions
Default login url httplocalhost9097kafkawize
44 Dashboard
After a user is logged in heshe will be redirected to homepage
16 Chapter 4 Features
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
45 Browse
All the logged in users with any roles can browse the below components
451 Topics
A user is allowed to a select an environment and the topics are loaded If you do not see the topics for some reasoncheck for any errors in Cluster Api application and UI Api application
A filter option is also provided and users can search for specific topics
Note Make sure kafkawize has access to read topics (Acl is required if acls are enabled on the cluster)
452 Acls
Acl - Access Controls
From the Browse topics screen user can select and clicking on a topic displays all the available acls of that topic
Acls are either Producers or Consumers Producers have write access on the topics and Consumers have read accesson the topics
45 Browse 17
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
Acls are either IP Address based or SSL based If IP based you should see the IP addresses of client who has accessIf SSL based the DN name should be seen
Every Acl should be owned by a team If no team is assigned to an Acl it can be assigned by Synchronize Acl optionbut by a Super user
There can be multiple producers and multiple consumers for a topic owning by different teams
Unknown team is displayed in case of that acl is not owned by any team
46 Requests
All the users can request for topics acls or avro schemas
Topic Requests can be approved by Admins or Super users from the same team
Acl Requests can be approved by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
bull Notifications After every request is raised there is a notification (bell icon) at the top for the relevant user whocan approve these requests
461 Topics
To request for a topic all the mandatory parameters need to be provided
Topic name partitions owning team application name and if any remarks On submission you can view your requestsin AuditMy Requests and it can be deleted if required
Default partitions size maximum partitions size and default replication factor can be configured per environment inClusters page
462 Acls
To request for a acl all the mandatory parameters need to be provided
18 Chapter 4 Features
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
After a user selects an environment topics are loaded and topic has to be selected
Acl type Producer or Consumer
IP Addresses or SSL DN name should be provided for authorizations
Only one IP Address or one SSL should be filled
On submission you can view your requests in AuditMy Requests and it can be deleted if required
463 Avro Schemas
An Avro schema can also be requested for a particular subject (topic)
47 Clusters
All the clusters are of type lsquokafkarsquo or lsquoschemaregistryrsquo
47 Clusters 19
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
New clusters can be added by Superusers only There is a other parameters field added into which default partitionsmax partitions size and replication factor can be added
A cluster can be deleted by any Admin user or super user
48 Audit
All the requests from users are audited and can be viewed
481 My Topic Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
20 Chapter 4 Features
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
482 My Acl Requests
Requests (TopicsAclsSchemas) from their own team can be viewed and deleted
width 500px align center
483 Activity Log
All the requests requested and approved can be viewed Users loginlogout are not part of this log yet
48 Audit 21
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
49 Approve Or Decline
Topic Requests can be approved or declined by Admins or Super users from the same team
Acl Requests can be approveddeclined by Admins or Super users who belong to Topic Ownerrsquos team
Schema Requests can be approved by Admins or Super users from the same team
Notifications are visible on the top right side of the portal
491 Topics
If a topic is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approveddeclined by lsquouser2rsquo from lsquoTeam1rsquo only providedlsquouser2rsquo has AdminSuper role
492 Acls
If a acl is requested by lsquouser1rsquo on topic which is owned by lsquoTeam2rsquo it can be approveddeclined by lsquouser2rsquo fromlsquoTeam2rsquo only provided lsquouser2rsquo has AdminSuper role
22 Chapter 4 Features
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
493 Avro Schemas
If a schema is requested by lsquouser1rsquo from lsquoTeam1rsquo it can be approved by lsquouser2rsquo from lsquoTeam1rsquo only provided lsquouser2rsquohas AdminSuper role
410 Users
All the users are visible to any logged in user A new user can be addeddeleted by only Super users
4101 View Users
From this page all the users can be seen and also be deleted
4102 Add User
With this form a new user can be added
410 Users 23
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
411 Teams
All the teams are visible to any logged in user A new team can be added or a team can be deleted by only Superusers
4111 View Teams
From this page all the users can be seen
4112 Add Team
With this form a new team can be added
24 Chapter 4 Features
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
412 Synchronize Metadata
A situation where Kafka cluster already exists and would like to adapt Kafkawize in your organization all the topicsand acls need to have their owners
This feature is possible with Synchronization of Topics or Acls
4121 Topics
After a environment is selected topics are displayed and a team can be assigned to it And this action team becomesthe Owner team
4122 Acls
After a environment is selected Consumer Acls are displayed and a team can be assigned to it And this action teambecomes the Owner of the acl
412 Synchronize Metadata 25
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
413 Admin
4131 Server Config
All the Server configuration including custom properties (application properties can be viewed)
414 Profile
All Users can view their profile with the button available in the top right corner of the portal
4141 Change Password
All the logged in users can change the password however it will be effective only after the UI Api Application isrestarted Enhancement will be provided on this soon
415 Logout
Users can logout after clicking on the logout button on the top right corner Session will be killed after this action
26 Chapter 4 Features
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
CHAPTER 5
Functionalities
bull Create Topics on Kafka Clusters Automated (Request and Approve)
bull Create Acls Automated (Request and Approve)
bull Register Avro schemas Automated (Request and Approve)
bull Search Topics with a User Interface
bull Search Acls with a User Interfaces
bull Search Producer and Consumer Teams
bull Synchronize Kafka topics with Teams
bull Synchronize Kafka acls with Teams
bull Users Teams and Roles based authorizations on features
bull View server configuration (application properties)
bull Audit of activities performed by users
bull Login and Logout
27
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
28 Chapter 5 Functionalities
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
CHAPTER 6
Customize
Kafkawize can be customized according to your requirement very easily
As the code is available you can make the changes locally in Front end (AngularJS HTML CSS ) or backend(Java)
29
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
30 Chapter 6 Customize
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
CHAPTER 7
Trouble shooting
bull Check for logs in Cluster Api UI Api
bull Check if user has the right access
bull Check if tables and data in the metastore can be accessible(grantspermissions) by Kafkawize
31
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
32 Chapter 7 Trouble shooting
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
CHAPTER 8
Release Notes
Kafkawize 41 release notes
Changes include 1 Critical bug fix - concurrent user access 2 Removed license check
33
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
34 Chapter 8 Release Notes
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
CHAPTER 9
Roadmap
bull Topic promotion
bull Email notifications
bull Delete Topic
bull Delete Acl
35
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
36 Chapter 9 Roadmap
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
CHAPTER 10
Contact Me
If you have any issues with kafkawize please raise it in
httpsgithubcommuralibasanikafkawizeissues
If you want to post a topic or question either you can post it in Stackoverflow or
httpskafkawizecomcommunity
You can also send me an email at infokafkawizecom or kafkawizegmailcom
You can also use the form httpskafkawizecomcontact-me
Muralidhar Basani
Linkedin httpswwwlinkedincominmuralibasani
37
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
38 Chapter 10 Contact Me
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
CHAPTER 11
Submit a Review
I will be pleased if you can write a testimonial about kafkawize at
httpskafkawizecomkafkawize_reviews
Please do not forget to star my project
httpsgithubcommuralibasanikafkawizestargazers
Thank you
39
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
Kafkawize Documentation Release 41
40 Chapter 11 Submit a Review
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-
CHAPTER 12
Indices and tables
bull genindex
bull modindex
bull search
41
- Introduction
-
- What is Kafkawize
- Background
- Key features
- 41 glimpse
- How is it developed
- Architecture
- Git Repositories
- Developer
-
- Getting Started
-
- Demo
- Prerequisites
- 1 Download Kafkawize
- 2 Download Metastore
- 3 License Key
- 4 Build KW Cluster Api Application
- 5 Configure Kafkawize UI Api Application
- 6 Metastore setup
- 7 Run KW and KWClusterApi
- 8 Kafka Connectivity
- 9 Final Check
-
- Migration Kafkawize from 35 to 40
- Features
-
- Security
- Teams and Role Management
- Login
- Dashboard
- Browse
- Requests
- Clusters
- Audit
- Approve Or Decline
- Users
- Teams
- Synchronize Metadata
- Admin
- Profile
- Logout
-
- Functionalities
- Customize
- Trouble shooting
- Release Notes
- Roadmap
- Contact Me
- Submit a Review
- Indices and tables
-