linux contos6.8 to deploy an kafka cluster

There are three servers, ip 192.168.174.10, 192.168.174.11 and 192.168.174.12 respectively.

1. Download it from the official website and unzip and install it on each machine

#  create kafka Installation directory of 
mkdir -p /usr/local/software/kafka
#  Unpack the 
tar -xvf kafka_2.12-1.1.0.tgz -C /usr/local/software/kafka/

2. Modify the /etc/profile file of each server, set the kafka environment variable, and add the following contents

export KAFKA_HOME=/usr/local/software/kafka/kafka_2.12-1.1.0
export PATH=$PATH:$KAFKA_HOME/bin

After adding the configuration restart, the command is as follows:

source /etc/profile

3. Modify the server. properties file under $KAFKA_HOME/config for each server, modify the agent id and log storage directory

# Designated agent id . borker.id It can be specified arbitrarily, as long as each machine in the cluster is guaranteed broker.id only 1 In the first 2 Machine set to 2... And so on 
broker.id=1
# Set the log directory 
log.dirs=/opt/data/kafka/kafka-logs
# Set up the zookeeper The cluster address 
zookeeper.connect=192.168.174.7:2181,192.168.174.8:2181,192.168.174.9:2181
# Set the local address to the local server ip Address. If not set, this will happen when the topic is created and the message is sent NOT LEADER FOR PARTITION The exception. 
host.name=192.168.174.10

4. After the configuration is completed, start kafka of 3 servers respectively, and ensure that the zookeeper cluster has been successfully started before starting kafka.

kafka-server-start.sh -daemon server.properties

5. Verify whether the startup is successful

jps

If you see the kafka process, it started successfully.

You can also view the zookeeper directory structure through the zookeeper client, as follows:

# The login zookeeper
zkCli . sh -server 192.168.174.7:2181
# View directory structure 
ls /
# View the started ones kafka The agent 
ls /brokers/ids