How to Setup Apache Zookeeper on Hadoop Cluster

ZooKeeper is an open-source high-performance distributed application coordination service. Coordination and management of service in a distributed environment is a difficult task. ZooKeeper solves this problem with its straightforward architecture and API. It exposes common services in a simple interface, such as naming, configuration management, synchronization, and group services, so you don't have to write them from scratch. It is ready-to-use for implementing consensus, group management, leader election, and presence protocols. And you can expand on it to meet your own unique requirements.

The ZooKeeper framework was created by Yahoo to make it easier and more robust to access their applications. Later, Apache ZooKeeper became a distributed framework standard for organized service, used by Hadoop, HBase, and other frameworks managed by Apache Software Foundation. This tutorial covers how to install and deploy a ZooKeeper cluster in a distributed environment.

Step 1: Verifying Java Installation

$ java -version

Step 2: Download ZooKeeper

Visit the following link https://zookeeper.apache.org/releases.html and download the latest version of ZooKeeper.

We are going to use the latest 3.7.0 version for now.

Step 3: Extract the tar file

$ tar -zxf apache-zookeeper-3.7.0-bin.tar.gz
$ cd apache-zookeeper-3.7.0-bin
$ mkdir data

Step 4: Create the configuration file

$ nano conf/zoo.cfg
# Enter these configs
tickTime = 2000
dataDir = /home/hdoop/apache-zookeeper-3.7.0-bin/data
clientPort = 2181
initLimit = 5
syncLimit = 2
admin.serverPort=8020

We have to add this admin.serverPort config because when I installed spark, it also ran in 8080 port so there was a clash of the port IDs so we had configured zookeeper port now to 8020. The error I faced:

Caused by: java.io.IOException: Failed to bind to /0.0.0.0:8080
    at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:346)
    at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:308)
    at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
    at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.eclipse.jetty.server.Server.doStart(Server.java:396)
    at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
    at org.apache.zookeeper.server.admin.JettyAdminServer.start(JettyAdminServer.java:103)
    ... 5 more
Caused by: java.net.BindException: Address already in use: bind
        at sun.nio.ch.Net.bind0(Native Method)
        at sun.nio.ch.Net.bind(Net.java:433)
        at sun.nio.ch.Net.bind(Net.java:425)
        at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223)
        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74)
        at org.eclipse.jetty.server.ServerConnector.openAcceptChannel(ServerConnector.java:342)
        ... 12 more
Unable to start AdminServer, exiting abnormally

Return to the terminal once the configuration file has been successfully saved. The zookeeper server can now be started.

Step 5: Start the ZooKeeper server

$ bin/zkServer.sh start

You will get the following response:

JMX enabled by default
Using config: /Users/../zookeeper-3.7.0/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED

Step 6: Start CLI

To start CLI, you can write

$ bin/zkCli.sh

You will get the following response:

Connecting to localhost:2181
................
................
................
Welcome to ZooKeeper!
................
................
WATCHER::
WatchedEvent state:SyncConnected type: None path:null
[zk: localhost:2181(CONNECTED) 0]

Stop ZooKeeper Server

After connecting to the server and completing all operations, use the following code to terminate the zookeeper server.

$ bin/zkServer.sh stop