This week I started to write some code to be processed by Kafka, but as I have heard before, set up this environment is usually trouble. So I just figure it out that Spotify has made a single docker image with Kafka and Zookeper. You just need to download and run. Usually, most of the images are only Kafka, so you need to bring both Kafka and AZookeper with docker-compose. Just a little bit more config, but this one was so confortable for my case.
So you just need this:
This will setup a Kafka on port 9092, adding only one parameter that NUM_PARTITIONS=10, meaning that all topic created will have 10 partitions, so I can register 10 consumers. To try this, I was using the following python library kafka-python, which is very convenient for small apps:
This is a producer, which its automatically serialize the data to work with json.