본문 바로가기

빅데이터/Kafka

Compacted topic에 null key 레코드를 전송하면?

Compacted topic에 null key를 전송(produce)하면 어떻게 될까요?

 

1) Compacted 토픽 생성

$ ./kafka-topics.sh --bootstrap-server localhost:9092 --topic compact-test --config "cleanup.policy=compact" --create
Created topic compact-test.

$ ./kafka-topics.sh --bootstrap-server localhost:9092 --topic compact-test --describe
Topic: compact-test	PartitionCount: 1	ReplicationFactor: 1	Configs: cleanup.policy=compact,segment.bytes=1073741824
	Topic: compact-test	Partition: 0	Leader: 0	Replicas: 0	Isr: 0

 

2) null key 레코드 전송

public static void main(String[] args) throws Exception{
    Properties configs = new Properties();
    configs.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, BOOTSTRAP_SERVERS);
    configs.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
    configs.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());

    KafkaProducer<String, String> producer = new KafkaProducer<>(configs);
    ProducerRecord<String, String> record = new ProducerRecord<>(TOPIC_NAME, "test");
    RecordMetadata meta = producer.send(record).get();
    logger.info(meta.toString());
    producer.flush();
    producer.close();
}

 

3) 결과 로그

Exception in thread "main" java.util.concurrent.ExecutionException: org.apache.kafka.common.InvalidRecordException: This record has failed the validation on broker and hence be rejected.
	at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:98)
	at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:67)
	at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:30)
	at com.example.SimpleProducer.main(SimpleProducer.java:27)
Caused by: org.apache.kafka.common.InvalidRecordException: This record has failed the validation on broker and hence be rejected.

FAILURE: Build failed with an exception.

InvalidRecordException와 함께 레코드가 전송되지 않는다!

 

 

반응형