Useful Commands and tools for Kafka
From Luis Gallego Hurtado - Not Another IT guy
Cheatsheets
https://gist.github.com/ursuad/e5b8542024a15e4db601f34906b30bb5
Utilities for Kafka
Kafkacat
https://github.com/edenhill/kafkacat
You can install Kafkacat with aptitude:
apt-get install kafkacat
Kafka useful commands
- List all topics in cluster
kafka-topics.sh --list --zookeeper <COMMA_SEPARATED_LIST_OF_PAIRS_ZOOKEEPER_HOST:PORT>
- Manually delete a topic
zookeeper-shell.sh <ZOOKEEPER_HOST:PORT>
rmr /brokers/topics/<TOPIC_NAME> rmr /admin/delete_topics/<TOPIC_NAME>
- Get all topic offsets
kafka-run-class.sh kafka.tools.ExportZkOffsets --output-file <PATH_TO_DESTINATION_FILE> --zkconnect <ZOOKEEPER_HOST:PORT>
kafka-run-class kafka.tools.GetOffsetShell --broker-list <COMMA_SEPARATED_LIST_OF_PAIRS_KAFKA_BROKER_HOST:PORT> --topic <TOPIC_NAME>
- List all consumer groups
kafka-consumer-groups.sh --list --bootstrap-server <COMMA_SEPARATED_LIST_OF_PAIRS_KAFKA_BROKER_HOST:PORT>
- Describe a consumer group
kafka-consumer-groups.sh --describe --bootstrap-server <COMMA_SEPARATED_LIST_OF_PAIRS_KAFKA_BROKER_HOST:PORT> --group <GROUP_NAME>
- Get the lag of different topics
kafka-replica-verification.sh --broker-list <COMMA_SEPARATED_LIST_OF_PAIRS_KAFKA_BROKER_HOST:PORT>