Apache Kafka is a popular distributed message broker designed to handle large volumes of real-time data efficiently. A Kafka cluster is not only highly scalable and fault-tolerant, but it also has a much higher throughput compared to other message brokers such as ActiveMQ and RabbitMQ. Though it is generally used as a pub/sub messaging system, a lot of organizations also use it for log aggregation because it offers persistent storage for published messages.
In this tutorial, you will learn how to install and use Apache Kafka 0.8.2.1 on Ubuntu 14.04.
To follow along, you will need:
As Kafka can handle requests over a network, you should create a dedicated user for it. This minimizes damage to your Ubuntu machine should the Kafka server be comprised.
Note: After setting up Apache Kafka, it is recommended that you create a different non-root user to perform other tasks on this server.
As root, create a user called kafka using the
Set its password using
Add it to the
sudo group so that it has the privileges required to install Kafka's dependencies. This can be done using the
Your Kafka user is now ready. Log into it using