Useful Commands and tools for Kafka

From Luis Gallego Hurtado - Not Another IT guy
Jump to: navigation, search


Cheatsheets

https://gist.github.com/ursuad/e5b8542024a15e4db601f34906b30bb5

Utilities for Kafka

Kafkacat

https://github.com/edenhill/kafkacat

You can install Kafkacat with aptitude:

apt-get install kafkacat

Kafka useful commands

  • List all topics in cluster
kafka-topics.sh --list --zookeeper <COMMA_SEPARATED_LIST_OF_PAIRS_ZOOKEEPER_HOST:PORT>
  • Manually delete a topic
zookeeper-shell.sh <ZOOKEEPER_HOST:PORT>
rmr /brokers/topics/<TOPIC_NAME>
rmr /admin/delete_topics/<TOPIC_NAME>
  • Get all topic offsets
kafka-run-class.sh kafka.tools.ExportZkOffsets --output-file <PATH_TO_DESTINATION_FILE> --zkconnect <ZOOKEEPER_HOST:PORT>
kafka-run-class kafka.tools.GetOffsetShell --broker-list  <COMMA_SEPARATED_LIST_OF_PAIRS_KAFKA_BROKER_HOST:PORT> --topic <TOPIC_NAME>
  • List all consumer groups
kafka-consumer-groups.sh --list --bootstrap-server <COMMA_SEPARATED_LIST_OF_PAIRS_KAFKA_BROKER_HOST:PORT>
  • Describe a consumer group
kafka-consumer-groups.sh --describe --bootstrap-server <COMMA_SEPARATED_LIST_OF_PAIRS_KAFKA_BROKER_HOST:PORT> --group <GROUP_NAME>
  • Get the lag of different topics
kafka-replica-verification.sh --broker-list  <COMMA_SEPARATED_LIST_OF_PAIRS_KAFKA_BROKER_HOST:PORT>