Kafka For Mere Mortals: Better compatibility with WSL2

We have already discussed the KAFKA installing process for Windows OS but as time goes on you realize that it was not a good choice to install Kafka on plain Windows. You need to understand that Kafka wasn’t intended to be installed on Windows initially.

Here are the issues you may face when Installing Kafka for plain Windows.

1. Not possible to delete the Kafka topic (KAFKA-8811)
2. Error deleting old log files (KAFKA-1194)

These errors are fatal errors which means it is still not possible (so far) to bypass them.

Fortunately, we have Windows subsystems for Linux ( in short WSL), and using it, it is possible to “simulate” Linux and bypass all the above issues. WSL is a Linux subsystem that allows to use of Linux distributions without a virtual machine.

PS: to follow the installation process you need to have Windows 10 or above with build 19041 or higher.

Here are the steps to install and run Kafka for Windows using WSL2.

Install WSL

Open Powershell using administrator access and run.

wsl –install

Provide username and password when asked by installation above.

For WSL2, it has a networking issue that doesn’t allow outside programs to connect to Kafka

To fix these problems run the following command after installation is completed

sudo sysctl -w net.ipv6.conf.all.disable_ipv6=1
sudo sysctl -w net.ipv6.conf.default.disable_ipv6=1

Install latest Java JDK (Amazon Corretto 17 Installation Instructions for Debian-Based, RPM-Based and Alpine Linux Distributions - Amazon Corretto 17)

wget -O- https://apt.corretto.aws/corretto.key | sudo apt-key add - 
 sudo add-apt-repository 'deb https://apt.corretto.aws stable main'
sudo apt-get update; sudo apt-get install -y java-17-amazon-corretto-jdk

PS: You have to face installing issues if you’re using some type of proxy tool like Zscaler. Please disable them, otherwise, you’ll get the following problem.

Here is the success run for the above code.

Run the java -version to see the below response.

And now it is time to install Kafka.

Go to Apache Kafka and copy the link from the binary file (https://downloads.apache.org/kafka/3.6.0/kafka_2.13-3.6.0.tgz)

Use the following command to install Kafka.

wget https://downloads.apache.org/kafka/3.6.0/kafka_2.13-3.6.0.tgz

Extract Kafka using tar command.

tar xzf kafka_2.13-3.6.0.tgz

1. Run Zookeeper using the following command.

~/kafka_2.13-3.6.0/bin/zookeeper-server-start.sh ~/kafka_2.13-3.6.0/config/zookeeper.properties

 

Run Kafka using following command.

~/kafka_2.13-3.6.0/bin/kafka-server-start.sh ~/kafka_2.13-3.6.0/config/server.properties

Don’t forget that the best way to install Kafka with Zookeeper depends on your requirements but using Docker is the most secure and the easiest way of installing them. As an adaptive solution, it will help you to run the combination on any major OS (Windows, Linux, Mac).