Kafka的使用和错误解决
单节点:233
目录:/mnt/kfk
在这里插入图片描述
启动zk

   /mnt/kfk/zookeeper-3.4.12/bin/zkServer.sh start

启动kafka

    后台启动
bin/kafka-server-start.sh -daemon config/server.properties

创建主题

 /mnt/kfk/kafka_2.11-2.1.0/bin/kafka-topics.sh --create --zookeeper localhost:2281/kafka --topic KafkaTestTopic --partitions 1 --replication-factor 1

其中 --zookeeper 指定了 Kafka 所连接的 ZooKeeper 服务地址,–topic 指定了所要创建主题的名称,–replication-factor 指定了副本因子,–partitions 指定了分区个数,–create 是创建主题的动作指令。

通过 --describe 展示主题的更多具体信息

/mnt/kfk/kafka_2.11-2.1.0/bin/kafka-topics.sh --zookeeper localhost:2181/kafka --describe --topic KafkaTestTopic

在这里插入图片描述
创建生产者

/mnt/kfk/kafka_2.11-2.1.0/bin/kafka-console-producer.sh --topic KafkaTestTopic --broker-list localhost:9092

查看server.properties中的#listeners=PLAINTEXT://:9092,获取kafka的端口

创建消费者

/mnt/kfk/kafka_2.11-2.1.0/bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic KafkaTestTopic

可能出现的错误:
(1)
[root@mail bin]# kafka-topics.sh --create --zookeeper localhost:2281 --topic KafkaTestTopic --partitions 1 --replication-factor 1
Error while executing topic command : Replication factor: 1 larger than available brokers: 0.
[2018-11-20 16:44:16,269] ERROR org.apache.kafka.common.errors.InvalidReplicationFactorException: Replication factor: 1 larger than available brokers: 0.
(kafka.admin.TopicCommand$)

解决:修改server.properties中的:zookeeper.connect=localhost:2281,让2281端口号和zookeeper.properties中的zookeeper端口号一致,然后重启kafka。**

(2)
kafka.common.KafkaException: fetching topic metadata for topics [Set(KafkaTestTopic)] from broker [ArrayBuffer(BrokerEndPoint(0,123.125.50.7,9092))] failed
at kafka.client.ClientUtils . f e t c h T o p i c M e t a d a t a ( C l i e n t U t i l s . s c a l a : 77 ) a t k a f k a . c l i e n t . C l i e n t U t i l s .fetchTopicMetadata(ClientUtils.scala:77) at kafka.client.ClientUtils .fetchTopicMetadata(ClientUtils.scala:77)atkafka.client.ClientUtils.fetchTopicMetadata(ClientUtils.scala:98)
at kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:67)
(3)
[2018-11-20 17:28:53,411] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 52 : {KafkaTestTopic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2018-11-20 17:28:53,513] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 53 : {KafkaTestTopic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2018-11-20 17:28:53,617] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 54 : {KafkaTestTopic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
[2018-11-20 17:28:53,721] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 55 : {KafkaTestTopic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)

解决(2)和(3)的错误:
修改server.properties中的
(1) listeners=PLAINTEXT://localhost:9092
(2) advertised.listeners=PLAINTEXT://localhost:9092

Logo

Kafka开源项目指南提供详尽教程,助开发者掌握其架构、配置和使用,实现高效数据流管理和实时处理。它高性能、可扩展,适合日志收集和实时数据处理,通过持久化保障数据安全,是企业大数据生态系统的核心。

更多推荐