前提:需要实现kafka生产者,如果没有实现,请查看我的这篇kafka生产者的使用
https://blog.csdn.net/LB3701/article/details/106933936

发送JSON消息请查看:发送JSON消息,接受JSON消息
**

kafka(卡夫卡消费者)的使用。

**
一,创建消费者MyConsumer.java

package com.kuxingseng.lbw.mq;

import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.time.Duration;
import java.util.Arrays;
import java.util.Properties;

public class MyConsumer implements Runnable {
    private final Logger logger = LoggerFactory.getLogger(this.getClass().getName());

    private Properties props = getProperties();
    private final String TOPIC = "LBW_EPM"; //消费者主题 和消费者主题一样("LBW_EPM")

    @Override
    public void run() {
        //创建消息者实例
        KafkaConsumer<String, KafkaDataConsumer> consumer = new KafkaConsumer<>(props);
        //订阅topic1的消息
        consumer.subscribe(Arrays.asList(TOPIC));
        try {
            while (true) {
                Thread.sleep(3000);
                //到服务器中读取记录
                ConsumerRecords<String, KafkaDataConsumer> records = consumer.poll(Duration.ofMillis(1000));
                for (ConsumerRecord<String, KafkaDataConsumer> record : records) {
                    System.out.println("key:" + record.key());
                    System.out.println("value:" + record.value());
                }
            }
        } catch (InterruptedException e) {
            logger.error("MyConsumer error:", e);
        } finally {
            consumer.close();
        }

    }

    private static Properties getProperties() {
        //配置信息
        Properties props = new Properties();
        //kafka服务器地址
        props.put("bootstrap.servers", "192.168.199.128:9092");
        //必须指定消费者组 "DEMO" 自定义任何值/可以选择已有的消费者组
        props.put("group.id", "DEMO");
        //设置数据key和value的序列化处理类
        props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
//        props.put("value.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        props.put("value.deserializer", "com.kuxingseng.lbw.mq.KafkaEntityDeserializer");
        return props;
    }
}


二,创建一个和生产者一样的实体类KafkaData.java,序列化的时候使用(一般生产者 和消费者是两个服务间的通信,我这里是在同一服务中演示的直接复用)。

package com.kuxingseng.lbw.mq;

import java.util.List;
import java.util.Map;

public class KafkaData {
    private List<Map> listMap;
    private int code = 0;

    public KafkaData() {
    }

    public KafkaData(int code) {
        this.code = code;
    }

    public KafkaData(List<Map> listMap, int code) {
        this.listMap = listMap;
        this.code = code;
    }

    public List<Map> getListMap() {
        return listMap;
    }

    public void setListMap(List<Map> listMap) {
        this.listMap = listMap;
    }

    public int getCode() {
        return code;
    }

    public void setCode(int code) {
        this.code = code;
    }

    @Override
    public String toString() {
        return "KafkaData{" +
                "listMap=" + listMap +
                ", code=" + code +
                '}';
    }

}


三,创建 自定义KafkaEntityDeserializer.java 反序列化器。这里getProperties()用的到【props.put("value.deserializer", "com.kuxingseng.lbw.mq.KafkaEntityDeserializer");

package com.kuxingseng.lbw.mq;

import com.fasterxml.jackson.databind.ObjectMapper;
import org.apache.kafka.common.serialization.Deserializer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import java.io.IOException;
import java.util.Map;

public class KafkaEntityDeserializer implements Deserializer {
    private final Logger logger = LoggerFactory.getLogger(this.getClass().getName());

    private ObjectMapper mapper;

    @Override
    public void configure(Map map, boolean b) {
        mapper = new ObjectMapper();
    }

    @Override
    public Object deserialize(String s, byte[] bytes) {
        try {
            return mapper.readValue(bytes, KafkaData.class);
        } catch (IOException e) {
            logger.error("My Deserializer error:", e);
        }
        return null;
    }

    @Override
    public void close() {

    }
}


验证前提:需要实现kafka生产者/使用kakfa脚本也可以发送消息。kafka生产者发送定义数据

我这里使用SpringBoot 调用接口发送kafka消息,内部再实现kafka消费者。
在这里插入图片描述

Logo

Kafka开源项目指南提供详尽教程,助开发者掌握其架构、配置和使用,实现高效数据流管理和实时处理。它高性能、可扩展,适合日志收集和实时数据处理,通过持久化保障数据安全,是企业大数据生态系统的核心。

更多推荐