flink：java.lang.NoClassDefFoundError: org/apache/kafka/common/errors/InvalidTxnStateException报错

Report an error

It is normal when flink consumes Kafka, but an error is reported when producing messages to Kafka:

java.lang.NoClassDefFoundError: org/apache/kafka/common/errors/InvalidTxnStateException

Troubleshoot

At first I thought it was caused by the jar package conflict, because both kafka-clients and flink-connector-kafka_2.11 were used in the project, and the kafka-clients under the two dependencies were excluded.

		<dependency>
			<groupId>org.apache.flink</groupId>
			<artifactId>flink-connector-kafka_2.11</artifactId>
			<version>${flink.version}</version>
			<exclusions>
				<exclusion>
					<groupId>org.apache.kafka</groupId>
					<artifactId>kafka-clients</artifactId>
				</exclusion>
			</exclusions>
		</dependency>

The problem is not resolved. I think that my Kafka connector uses flink-connector-kafka_2.11, and the Kafka cluster is installed with version 0.11.0. The official website says that flink-connector-kafka_2.11 will adapt to the latest version of Kafka , but for Kafka For 0.11.x and 0.10.x versions, it is recommended to use dedicated flink-connector-kafka-0.11_2.11 and flink-connector-kafka-0.10_2.11 respectively.

So modify the flink-kafka connector:

		<dependency>
			<groupId>org.apache.flink</groupId>
			<artifactId>flink-connector-kafka-0.11_2.11</artifactId>
			<version>${flink.version}</version>
		</dependency>

problem solved.

PS: My ${flink.version} is 1.7.0. Modify the version number according to your actual situation.