问题:批处理写入kafka,结束关闭kafka时出现错误:

Caused by: java.lang.NoSuchMethodError: org.apache.kafka.clients.producer.KafkaProducer.close(Ljava/time/Duration;)V
	at org.apache.flink.streaming.connectors.kafka.internals.FlinkKafkaInternalProducer.close(FlinkKafkaInternalProducer.java:172) ~[flink-connector-kafka_2.12-1.0.0.jar:?]
	at org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducer.close(FlinkKafkaProducer.java:941) ~[flink-connector-kafka_2.12-1.0.0.jar:?]
	at org.apache.flink.api.common.functions.util.FunctionUtils.closeFunction(FunctionUtils.java:41) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
	at org.apache.flink.streaming.api.operators.AbstractUdfStreamOperator.close(AbstractUdfStreamOperator.java:109) ~[flink-dist_2.12-1.13.2.jar:1.13.2]
	at org.apache.flink.streaming.runtime.tasks.StreamOperatorWrapper.lambda$closeOperator$5(StreamOperatorWrapper.java:213) ~[flink-dist_2.12-1.13.2.jar:1.13.2]

分析:是版本冲突:

  1、    flink-avro-confluent-registry 里面包含了kafka-clients 版本是5.5

 2、kafka connector里面包含了kafka-clients 版本是2.4.1

Logo

Kafka开源项目指南提供详尽教程,助开发者掌握其架构、配置和使用,实现高效数据流管理和实时处理。它高性能、可扩展,适合日志收集和实时数据处理,通过持久化保障数据安全,是企业大数据生态系统的核心。

更多推荐