Kafka生产者发送消息的方式解析
kafka生产者为我们提供了三种发送消息的方式。
·
kafka生产者发送消息的方式可以分为:发送并忘记、同步发送、异步发送。
1、发送并忘记
生产者只要调用send方法就可以了,不需要处理send的返回值,大多数情况下消息都可以正常发送出去,但可能会丢失消息。
public class KafkaProducer {
public static void main(String[] args) {
KafkaProducer<String, String> producer = new KafkaProducer(producerConfigs());
ProducerRecord<String, String> record = new ProducerRecord("test_topic", "hello kafka");
//调用send方法即完成发送
producer.send(record);
}
public static Map<String, Object> producerConfigs() {
Map<String, Object> props = new HashMap<>();
props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, "192.168.93.132:9092");
props.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
props.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
return props;
}
}
但是当我们获取元数据时出现问题,send的方法依旧会被阻塞。
try {
//这边会阻塞
clusterAndWaitTime = waitOnMetadata(record.topic(), record.partition(), maxBlockTimeMs);
} catch (KafkaException e) {
if (metadata.isClosed())
throw new KafkaException("Producer closed while send in progress", e);
throw e;
}
比如指定一个不存在的分区发送消息,就会因为元数据获取问题,导致阻塞,默认是等待60s。
Exception in thread "main" java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Topic test_topic not present in metadata after 1000 ms.
at org.apache.kafka.clients.producer.KafkaProducer$FutureFailure.<init>(KafkaProducer.java:1269)
at org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:933)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:856)
at org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:743)
at cn.enjoyedu.sendtype.KafkaFutureProducer.main(KafkaFutureProducer.java:26)
Caused by: org.apache.kafka.common.errors.TimeoutException: Topic test_topic not present in metadata after 1000 ms.
2、同步发送
接收send方法的返回值Future对象,并调用get方法。
public static void main(String[] args) throws ExecutionException, InterruptedException {
KafkaProducer<String, String> producer = new KafkaProducer(producerConfigs());
ProducerRecord<String, String> record = new ProducerRecord("test_topic", "hello kafka");
Future<RecordMetadata> send = producer.send(record);
//get方法会阻塞
RecordMetadata recordMetadata = send.get();
System.out.println("topic: " + recordMetadata.topic() + ", offset: " + recordMetadata.offset() + ", partition: " + recordMetadata.partition());
}
可以获取发送的一些信息。
topic: test_topic, offset: 232, partition: 3
3、异步发送
使用带Callback的send方法,提供请求完成的异步处理,当发送到服务器的记录已被确认时,将调用此方法。
public Future<RecordMetadata> send(ProducerRecord<K, V> record, Callback callback) {
// intercept the record, which can be potentially modified; this method does not throw exceptions
ProducerRecord<K, V> interceptedRecord = this.interceptors.onSend(record);
return doSend(interceptedRecord, callback);
}
public static void main(String[] args) {
KafkaProducer<String, String> producer = new KafkaProducer(producerConfigs());
ProducerRecord<String, String> record = new ProducerRecord("test_topic", "hello_key", "hello kafka");
//callback1
producer.send(record,
(metadata, e) -> {
if (e != null) {
e.printStackTrace();
} else {
System.out.println("The offset of the record we just sent is: " + metadata.offset());
}
});
//callback2
producer.send(record,
(metadata, e) -> {
if (e != null) {
e.printStackTrace();
} else {
System.out.println("The offset of the record we just sent is: " + metadata.offset());
}
});
producer.close();
}
The offset of the record we just sent is: 98
The offset of the record we just sent is: 99
对于发送到同一分区的记录,保证按顺序执行回调,也就是说,在上面的示例中,保证在callback2之前执行callback1。
更多推荐
已为社区贡献9条内容
所有评论(0)