In this article, we will be discussing some of its features and different use cases.
So to start with we can say that Kafka is a very fast, highly reliable, extremely fast, distributed pub-sub messaging system.
Today Kafka is being used by many big tech giants like Netflix, Twitter, LinkedIn, Uber, etc. all across the globe.
Kafka is so popular because of its exceptional speed and performance. It is really easy to set up and use.
Kafka is distributed which means that we can any time scale it up when needed and as it supports horizontal scaling so we can add new nodes to the Kafka cluster.
Kafka has very low latency and it can handle several millions of messages per second which makes it easy to use for the real-time processing of data.
It is written in Scala and Java and has a very high throughput.
Kafka Vs Traditional messaging
Now if we compare Kafka versus Traditional messaging systems we can see the following differences,
Kafka has a record retention feature which means that Kafka retains the record even after it is consumed, for a period of 7 days which is the default.
For example, RabbitMQ pushes messages to its consumers, on the other hand, Kafka supports pulling messages by its consumers.
It is designed to be scaled horizontally as opposed to the traditional messaging queues which are designed to scale vertically.
Kafka also supports multiple subscribers means the same message can be consumed by multiple consumers if they do not belong to the same consumer group.
Whereas in RabbitMQ, a message can only be consumed once, and once it is consumed the message will be deleted.
Kafka can process several millions of messages per second that too with a very less no of nodes, whereas RabbitMQ can also process millions of messages per second but it requires above 30 nodes.
Kafka provides the feature of message ordering and messages are sent to topics with the Key, whereas in RabbitMQ this feature is not supported.
Use Cases
Now so far we have discussed some of the important features of Kafka, So what are the best use cases under which we should be using Kafka.
Some of them are mentioned below,
- It can be used in Application activity tracking, and we can actually track the activity of a user in almost real-time mode. We can publish each event to a separate topic like User clicks, likes, searches, etc.
- It can be used for logging like we can publish logs to the Kafka topics, and then these logs can be aggregated or processed, and then these logs can be transformed and then can be stored in log storage systems.
- It is extremely useful in real-time data processing, for example, If there is any fraudulent activity or event then we can publish a message on a particular topic and can consume the message to handle or block the fraudulent activity in real-time.
- If we talk about Netflix then it is using Kafka for the real-time monitoring systems.
Kafka supports large-scale message processing capabilities as it has built-in partitioning and very high fault tolerance due to multiple brokers. It also has features like replication which makes it easier to replace the traditional message brokers.
SUMMARY
In this article, we have discussed Kafka features and real-life use cases. In the next article, we will be actually setting up Kafka and see how it works in action.
I hope you find this article helpful. Stay tuned for more … Cheers!!