Download Kafka Mac

by



If you are running Mac or Linux or Windows 10 Pro version, you are already sorted and can ignore the above gibberish! Techstack Laptop with Windows 10 HOME version Oracle VM Virtual Box CentOS as the OS in the VM Kafka Docker image from landloop JAVA 1.8. If you do not have brew installed on your mac, Install homebrew on your mac by running the following command on your Terminal. $ /usr/bin/ruby -e “ $ (curl -fsSL https: / /raw.githubusercontent.com/Homebrew /install/master/install) ' Install kafka in MacOS Catalina First install openjdk 8 using brew cask and then install kakfa. MAC/Unix./zookeeper-server-start.sh./config/zookeeper.properties How to start a Kafka Broker? Windows: kafka-server-start.bat. Config server.properties MAC/Unix./kafka-server-start.sh./config/server.properties How to check the configuration of all the topics in a broker? Kafka-topics.bat -describe -zookeeper localhost. Installing Apache Kafka on macOS. Prerequisite: java with version 8 should be installed on the system. To download the latest version of Apache Kafka(or as per the need), the following steps will be used. The program lies within Development Tools, more precisely Debugging Tools. The most popular version of the Kafka Tool is 1.0. The Kafka Tool installer is commonly called kafkatool.exe. The latest installer that can be downloaded is 50.3 MB in size. This download was checked by our antivirus and was rated as clean.

Download
Contents
  • Install Apache Kafka on Mac

    • To install Apache Kafka on Mac, Java is the only prerequisite. First we shall look into the installation steps of Java and then we shall setup Apache Kafka and run it on the Mac.

    Install Java

    1. Open a browser and hit the url [http://www.oracle.com/technetwork/java/javase/downloads/index.html].
    2. Click on JDK, check “Accept License Agreement” and download .dmg file for installation on Mac.
    3. Double click on the downloaded file and proceed with the installation steps.
    4. You have successfully installed Java. You may delete the .dmg file, as memory is costly on Mac.

    You may verify the installation of Java on Mac, by running the following command on a Terminal.

    java -version

    Install Apache Kafka on Mac

    1. Download the latest Apache Kafka from [https://kafka.apache.org/downloads] under Binary downloads.
    2. Click on any of the binary downloads, or choose a specific scala version if you have any dependency with scala in your development.
    3. Go with the recommended mirror site.
    4. Extract the contents. Navigate to root of Apache Kafka folder and open a Terminal. Or Open a Terminal and navigate to the root directory of Apache Kafka.
      apples-MacBook-Pro:kafka_2.12-1.0.0Prasanth$ ls

    Start Zookeeper

    Apache Kafka depends on Zookeeper for cluster management. Hence, prior to starting Kafka, Zookeeper has to be started. There is no need to explicitly install Zookeeper, as it comes included with Apache Kafka.

    From the root of Apache Kafka, run the following command to start Zookeeper :

    ~$sh bin/zookeeper-server-start.sh config/zookeeper.properties

    The zookeeper should be started with a similar following trace in the output.

    Download Kafka Manager

    apples-MacBook-Pro:kafka_2.12-1.0.0Prasanth$ sh bin/zookeeper-server-start.sh config/zookeeper.properties
    [2017-12-3115:18:36,556] INFO Reading configuration from:config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
    [2017-12-3115:18:36,560] INFO autopurge.snapRetainCount set to3 (org.apache.zookeeper.server.DatadirCleanupManager)
    [2017-12-3115:18:36,560] INFO autopurge.purgeInterval set to0 (org.apache.zookeeper.server.DatadirCleanupManager)
    [2017-12-3115:18:36,560] INFO Purge task isnotscheduled. (org.apache.zookeeper.server.DatadirCleanupManager)
    [2017-12-3115:18:36,560] WARN Either no config orno quorum defined inconfig, running instandalone mode (org.apache.zookeeper.server.quorum.QuorumPeerMain)
    [2017-12-3115:18:36,582] INFO Reading configuration from:config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
    [2017-12-3115:18:36,582] INFO Starting server (org.apache.zookeeper.server.ZooKeeperServerMain)
    [2017-12-3115:18:36,650] INFO Server environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on03/23/201710:13GMT (org.apache.zookeeper.server.ZooKeeperServer)
    [2017-12-3115:18:36,650] INFO Server environment:host.name=192.168.0.104 (org.apache.zookeeper.server.ZooKeeperServer)

    Start Apache Kafka Server

    Open another Terminal and run the following command from the root of Apache Kafka to start Apache Kafka.

    Download

    Kafka Tool

    ~$sh bin/kafka-server-start.sh config/server.properties

    Following should be the end of trace stating that Kafka server is started.

    [2017-12-3115:52:20,291] INFO Initiating client connection, connectString=localhost:2181sessionTimeout=6000watcher=org.I0Itec.zkclient.ZkClient@4690b489 (org.apache.zookeeper.ZooKeeper)
    [2017-12-3115:52:20,307] INFO Waiting forkeeper state SyncConnected (org.I0Itec.zkclient.ZkClient)
    [2017-12-3115:52:20,310] INFO Opening socket connection toserver localhost/0:0:0:0:0:0:0:1:2181.Will notattempt toauthenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
    [2017-12-3115:52:20,333] INFO Socket connection established tolocalhost/0:0:0:0:0:0:0:1:2181, initiating session (org.apache.zookeeper.ClientCnxn)
    [2017-12-3115:52:20,423] INFO Session establishment complete on server localhost/0:0:0:0:0:0:0:1:2181, sessionid=0x160abf917fa0000, negotiated timeout=6000 (org.apache.zookeeper.ClientCnxn)
    [2017-12-3115:52:20,425] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
    [2017-12-3115:52:20,798] INFO Cluster ID=diRCJM2mQ4qI9KzjiiLRVA (kafka.server.KafkaServer)
    [2017-12-3115:52:20,820] WARN No meta.properties file under dir/tmp/kafka-logs/meta.properties (kafka.server.BrokerMetadataCheckpoint)
    [2017-12-3115:52:20,866] INFO [ThrottledRequestReaper-Fetch]:Starting (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
    [2017-12-3115:52:20,869] INFO [ThrottledRequestReaper-Produce]:Starting (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
    [2017-12-3115:52:20,870] INFO [ThrottledRequestReaper-Request]:Starting (kafka.server.ClientQuotaManager$ThrottledRequestReaper)
    [2017-12-3115:52:20,923] INFO Log directory'/tmp/kafka-logs'notfound, creating it. (kafka.log.LogManager)
    [2017-12-3115:52:20,943] INFO Loading logs. (kafka.log.LogManager)
    [2017-12-3115:52:20,973] INFO Logs loading complete in29ms. (kafka.log.LogManager)
    [2017-12-3115:52:21,155] INFO Starting log cleanup withaperiod of300000ms. (kafka.log.LogManager)
    [2017-12-3115:52:21,159] INFO Starting log flusher withadefaultperiod of9223372036854775807ms. (kafka.log.LogManager)
    [2017-12-3115:52:21,535] INFO Awaiting socket connections on0.0.0.0:9092. (kafka.network.Acceptor)
    [2017-12-3115:52:21,538] INFO [SocketServer brokerId=0] Started1acceptor threads (kafka.network.SocketServer)
    [2017-12-3115:52:21,571] INFO [ExpirationReaper-0-Produce]:Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
    [2017-12-3115:52:21,573] INFO [ExpirationReaper-0-Fetch]:Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
    [2017-12-3115:52:21,575] INFO [ExpirationReaper-0-DeleteRecords]:Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
    [2017-12-3115:52:21,588] INFO [LogDirFailureHandler]:Starting (kafka.server.ReplicaManager$LogDirFailureHandler)
    [2017-12-3115:52:21,653] INFO [ExpirationReaper-0-topic]:Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
    [2017-12-3115:52:21,658] INFO [ExpirationReaper-0-Heartbeat]:Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
    [2017-12-3115:52:21,658] INFO Creating/controller (isit secure? false) (kafka.utils.ZKCheckedEphemeral)
    [2017-12-3115:52:21,659] INFO [ExpirationReaper-0-Rebalance]:Starting (kafka.server.DelayedOperationPurgatory$ExpiredOperationReaper)
    [2017-12-3115:52:21,666] INFO Result of znode creation is:OK (kafka.utils.ZKCheckedEphemeral)
    [2017-12-3115:52:21,683] INFO [GroupCoordinator0]:Starting up. (kafka.coordinator.group.GroupCoordinator)
    [2017-12-3115:52:21,684] INFO [GroupCoordinator0]:Startup complete. (kafka.coordinator.group.GroupCoordinator)
    [2017-12-3115:52:21,688] INFO [GroupMetadataManager brokerId=0] Removed0expired offsets in4milliseconds. (kafka.coordinator.group.GroupMetadataManager)
    [2017-12-3115:52:21,727] INFO [ProducerId Manager0]:Acquired newproducerId block (brokerId:0,blockStartProducerId:0,blockEndProducerId:999) by writing toZk with path version1 (kafka.coordinator.transaction.ProducerIdManager)
    [2017-12-3115:52:21,782] INFO [TransactionCoordinator id=0] Starting up. (kafka.coordinator.transaction.TransactionCoordinator)
    [2017-12-3115:52:21,784] INFO [Transaction Marker Channel Manager0]:Starting (kafka.coordinator.transaction.TransactionMarkerChannelManager)
    [2017-12-3115:52:21,797] INFO [TransactionCoordinator id=0] Startup complete. (kafka.coordinator.transaction.TransactionCoordinator)
    [2017-12-3115:52:21,889] INFO Creating/brokers/ids/0 (isit secure? false) (kafka.utils.ZKCheckedEphemeral)
    [2017-12-3115:52:21,895] INFO Result of znode creation is:OK (kafka.utils.ZKCheckedEphemeral)
    [2017-12-3115:52:21,897] INFO Registered broker0at path/brokers/ids/0with addresses:EndPoint(192.168.0.104,9092,ListenerName(PLAINTEXT),PLAINTEXT) (kafka.utils.ZkUtils)
    [2017-12-3115:52:21,900] WARN No meta.properties file under dir/tmp/kafka-logs/meta.properties (kafka.server.BrokerMetadataCheckpoint)
    [2017-12-3115:52:21,920] INFO Kafka version:1.0.0 (org.apache.kafka.common.utils.AppInfoParser)
    [2017-12-3115:52:21,920] INFO Kafka commitId:aaa7af6d4a11b29d (org.apache.kafka.common.utils.AppInfoParser)
    [2017-12-3115:52:21,922] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)

    Conclusion :

    In this Apache Kafka Tutorial, we have successfully learnt to install Apache Kafka on Mac and start Kafka along with Zookeeper.