The main code for initialization is as follows:
config := sarama.NewConfig()
config.Producer.RequiredAcks = sarama.WaitForAll // Wait for all in-sync replicas to ack the message
config.Producer.Retry.Max = 10 // Retry up to 10 times to produce the message
config.Producer.Return.Successes = true
config.Producer.Compression = sarama.CompressionSnappy // Compress messages
config.Producer.Partitioner = sarama.NewRandomPartitioner
config.Producer.Return.Errors = true
producer, err := sarama.NewSyncProducer(brokerList, config)
if err != nil {
log.Printf("sarama.NewSyncProducer fail:%+v\n", err)
return err
}
The main code of the production message is as follows:
msg := sarama.ProducerMessage{
Topic: monitorInstance.topic,
Value: sarama.StringEncoder(resportString),
//Key: sarama.StringEncoder("monitor_data"),
}
partition, offset, err := (*producer).SendMessage(&msg)
if err != nil {
log.Printf("kafka SendMessage fail:%+v\n", err)
return
}
log.Printf("Your data is stored with unique identifier important/%d/%d\n", partition, offset)
Tried the following methods will not work:
1. Change the producer from async to sync
2. Increase the key in msg
3. Try to replace the topic
Finally found this problem:
config.Producer.Compression = sarama.CompressionSnappy
commented out this line is normal.
Can't it be compressed? Asked the engineer of Tencent Cloud and got a solution:
config.Version = sarama.V2_1_0_0
config.Producer.Compression = sarama.CompressionSnappy