Overview

Learn how to install Apache Kafka.

Zookeeper Installation

Zookeeper is required to install Kafka. Recently, it has become possible to use KRaft instead of Zookeeper.

For those unfamiliar with [Zookeeper](https://zookeeper.apache.org/index.html), it can be briefly described as a configuration store for distributed applications.

Due to the nature of Hadoop, applications run in distributed environments, and there are inevitably times when you need to store global information or need locks for synchronization.

In such cases, you can use Zookeeper. The official Zookeeper website describes it as follows:

> ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services.

Download and Extract Zookeeper Tar

At the current time (20221214), the latest release version of Zookeeper is 3.8.0, and the tar.gz can be downloaded from the site below.

https://www.apache.org/dyn/closer.lua/zookeeper/zookeeper-3.8.0/apache-zookeeper-3.8.0-bin.tar.gz

wget https://dlcdn.apache.org/zookeeper/zookeeper-3.8.0/apache-zookeeper-3.8.0-bin.tar.gz

tar -zxvf apache-zookeeper-3.8.0-bin.tar.gz

Modify zoo.cfg

Since we will configure the Zookeeper server with 3 nodes, enter the FQDN of each Zookeeper server along with the port range 2888:3888 for `server.1`, `server.2`, and `server.3` respectively. (You may use IP addresses instead of FQDNs.)

Modify `dataDir`, which is the location where the myid file will be stored. I set it to `/var/zookeeper`.

The number of milliseconds of each tick

tickTime=2000

The number of ticks that the initial

synchronization phase can take

initLimit=10

The number of ticks that can pass between

sending a request and getting an acknowledgement

syncLimit=5

the directory where the snapshot is stored.

do not use /tmp for storage, /tmp here is just

example sakes.

dataDir=/var/zookeeper

the port at which the clients will connect

clientPort=2181

server.1=server1:2888:3888

server.2=server2:2888:3888

server.3=server3:2888:3888

Create myid File

Enter a unique myid for each server in the `dataDir` specified above.

For example, on server1 enter the following:

1

On server2, enter the following:

2

Only numbers between 1 and 255 are allowed.

Start Zookeeper Server

Run the command on the server where Zookeeper is installed to start Zookeeper.

apache-zookeeper-3.8.0-bin/bin/zkServer.sh start

Kafka Installation

Download and Extract Kafka Binary

wget https://dlcdn.apache.org/kafka/3.3.1/kafka_2.13-3.3.1.tgz

tar -xzf kafka_2.13-3.3.1.tgz

Modify server.properties

Just as you changed the myid file for each server during Zookeeper installation, set a unique `broker.id` for each Kafka server.

Enter the Zookeeper cluster information in `zookeeper.connect`.

broker.id=1

zookeeper.connect=server1:2181,server2:2181,server3:2181

Start Kafka Broker

kafka_2.13-3.3.1/bin/kafka-server-start.sh config/server.properties

Create a Topic

On one of the nodes where Kafka was started, create a topic called test using the command below. If the message `Created topic test.` is displayed, the installation was successful.

bin/kafka-topics.sh --create --bootstrap-server localhost:9092 --replication-factor 3 --partitions 20 --topic test

Quiz

Q1: What is the main topic covered in "Apache Kafka Installation Guide (with Zookeeper)"?

Learn how to install Apache Kafka (with Zookeeper).

At the current time (20221214), the latest release version of Zookeeper is 3.8.0, and the tar.gz

can be downloaded from the site below.

https://www.apache.org/dyn/closer.lua/zookeeper/zookeeper-3.8.0/apache-zookeeper-3.8.0-bin.tar.gz

Since we will configure the Zookeeper server with 3 nodes, enter the FQDN of each Zookeeper server

along with the port range 2888:3888 for server.1, server.2, and server.3 respectively.

Enter a unique myid for each server in the dataDir specified above. For example, on server1 enter

the following: On server2, enter the following: Only numbers between 1 and 255 are allowed.

Run the command on the server where Zookeeper is installed to start Zookeeper. Kafka Installation