学习kafka:log4j写入kafka
最近花了一天时间把kafka官方文档看了一遍,自己测试了一下,特记录与此,原理概念看完文档就能理解,主要是问题。1 版本POM配置,注意jar包可能会冲突: <dependency><groupId>org.apache.kafka</groupId><artifactId>kafka_2.10</artifactId>
·
最近花了一天时间把kafka官方文档看了一遍,自己测试了一下,特记录与此,原理概念看完文档就能理解,主要是问题。
1 版本POM配置 ,注意jar包可能会冲突:
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka_2.10</artifactId>
<version>0.8.2.2</version>
<!--<exclusions>-->
<!--<exclusion>-->
<!--<groupId>org.slf4j</groupId>-->
<!--<artifactId>slf4j-log4j12</artifactId>-->
<!--</exclusion>-->
<!--</exclusions>-->
</dependency>
<!-- https://mvnrepository.com/artifact/org.apache.kafka/kafka-log4j-appender -->
<dependency>
<groupId>org.apache.kafka</groupId>
<artifactId>kafka-log4j-appender</artifactId>
<version>0.10.2.1</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-log4j12</artifactId>
<version>1.7.5</version>
</dependency>
2 log4j配置:
log4j.rootLogger=INFO,console,KAFKA
## appender KAFKA
log4j.appender.KAFKA=kafka.producer.KafkaLog4jAppender
log4j.appender.KAFKA.topic=test
log4j.appender.KAFKA.brokerList=127.0.0.1:9092
log4j.appender.KAFKA.pa=127.0.0.1:9092
log4j.appender.KAFKA.compressionType=none
log4j.appender.KAFKA.syncSend=true
log4j.appender.KAFKA.layout=org.apache.log4j.PatternLayout
log4j.appender.KAFKA.ThresholdFilter.level=INFO
log4j.appender.KAFKA.ThresholdFilter.onMatch=ACCEPT
log4j.appender.KAFKA.ThresholdFilter.onMismatch=DENY
log4j.appender.KAFKA.layout.ConversionPattern=%d{yyyy-MM-dd HH:mm:ss} %-5p %c{1}:%L %% - %m%n
## appender console
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.out
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d (%t) [%p - %l] %m%n
3 测试代码:
package com.need.kafka.services.demo;
import com.need.kafka.services.utils.KafkaPropertiesManager;
import kafka.javaapi.producer.Producer;
import kafka.producer.KeyedMessage;
import kafka.producer.ProducerConfig;
import org.slf4j.impl.StaticLoggerBinder;
import org.slf4j.*;
import java.util.ArrayList;
import java.util.List;
/**
* 这个是demo 主要用来测试生产者的单条信息发送和批量信息发送
* <p/>
*/
public class ProducerDemo {
private static final Logger log = LoggerFactory.getLogger(ProducerDemo.class);
public static void main(String args[]) throws InterruptedException {
// ProducerConfig producerConfig = new ProducerConfig(KafkaPropertiesManager.getProducerProps());
// Producer<String, String> producer = new Producer<String, String>(producerConfig);
// // 单个发送
// for (int i = 0; i <= 10; i++) {
// KeyedMessage<String, String> message =
// new KeyedMessage<String, String>(KafkaProperties.TEST_TOPIC, i + "", "Message" + i);
// producer.send(message);
// }
//
// 批量发送
// List<KeyedMessage<String, String>> messages = new ArrayList<KeyedMessage<String, String>>(100);
// for (int i = 0; i <= 100; i++) {
// KeyedMessage<String, String> message =
// new KeyedMessage<String, String>(KafkaProperties.TEST_TOPIC, i + "", "Message" + i);
// messages.add(message);
// if (i % 100 == 0) {
// producer.send(messages);
// messages.clear();
// }
// }
// producer.send(messages);
for(int i = 0;i <= 10; i++) {
log.info("trace:This is Message [" + i + "] from log4j producer .. ");
log.info("This is Message [" + i + "] from log4j producer .. ");
}
}
}
注意:类的来源,有的类不同jar都会有
4结果:
2017-05-21 10:41:09 INFO ProducerDemo:44 % - trace:This is Message [5] from log4j producer ..
2017-05-21 10:41:09 INFO ProducerDemo:45 % - This is Message [5] from log4j producer ..
2017-05-21 10:41:09 INFO ProducerDemo:44 % - trace:This is Message [6] from log4j producer ..
2017-05-21 10:41:09 INFO ProducerDemo:45 % - This is Message [6] from log4j producer ..
2017-05-21 10:41:09 INFO ProducerDemo:44 % - trace:This is Message [7] from log4j producer ..
2017-05-21 10:41:09 INFO ProducerDemo:45 % - This is Message [7] from log4j producer ..
2017-05-21 10:41:09 INFO ProducerDemo:44 % - trace:This is Message [8] from log4j producer ..
2017-05-21 10:41:09 INFO ProducerDemo:45 % - This is Message [8] from log4j producer ..
2017-05-21 10:41:09 INFO ProducerDemo:44 % - trace:This is Message [9] from log4j producer ..
2017-05-21 10:41:09 INFO ProducerDemo:45 % - This is Message [9] from log4j producer ..
2017-05-21 10:41:09 INFO ProducerDemo:44 % - trace:This is Message [10] from log4j producer ..
2017-05-21 10:41:09 INFO ProducerDemo:45 % - This is Message [10] from log4j producer ..
注释:最麻烦的可能就是原项目已经使用了log4j,引入了一堆jar,可能需要重新修改jar的引入。
更多推荐
已为社区贡献1条内容
所有评论(0)