在这里插入图片描述

1.背景

新下载了一个kafka ,然后执行【Kafka】Kafka如何开启SSL 控制台消费与生产代码消费与生产的创建脚本，脚本如下

#! /bin/bash
echo "Step1: Config env"
BASE_DIR=/Users/lcc/soft/kafka/kafka_2.11-1.1.0_author
CERT_OUTPUT_PATH="$BASE_DIR/certificates"
PASSWORD=ke123456
KEY_STORE="$CERT_OUTPUT_PATH/kafka.keystore"
TRUST_STORE="$CERT_OUTPUT_PATH/kafka.truststore"
KEY_PASSWORD=$PASSWORD
STORE_PASSWORD=$PASSWORD
TRUST_KEY_PASSWORD=$PASSWORD
TRUST_STORE_PASSWORD=$PASSWORD
CLUSTER_NAME=test-cluster-01
CERT_AUTH_FILE="$CERT_OUTPUT_PATH/ca-cert"
CLUSTER_CERT_FILE="$CERT_OUTPUT_PATH/${CLUSTER_NAME}-cert"
DAYS_VALID=365
D_NAME="CN=localhost, OU=YourDept, O=YourCompany, L=China, ST=China, C=localhost"

mkdir -p $CERT_OUTPUT_PATH

echo "Step2: Create certificate to keystore"
keytool -keystore $KEY_STORE -alias $CLUSTER_NAME -validity $DAYS_VALID -genkey -keyalg RSA -storepass $STORE_PASSWORD -keypass $KEY_PASSWORD -dname "$D_NAME"

echo "Step3: Create CA"
openssl req -new -x509 -keyout $CERT_OUTPUT_PATH/ca-key -out "$CERT_AUTH_FILE" -days "$DAYS_VALID" -passin pass:"$PASSWORD" -passout pass:"$PASSWORD" -subj "/C=CN/ST=XX/L=XX/O=XX/CN=XX"


echo "Step4: Import CA into truststore"
keytool -keystore "$TRUST_STORE" -alias CARoot -import -file "$CERT_AUTH_FILE" -storepass "$TRUST_STORE_PASSWORD" -keypass "$TRUST_KEY_PASS" -noprompt

echo "Step5: Export certificate from keystore"
keytool -keystore "$KEY_STORE" -alias "$CLUSTER_NAME" -certreq -file "$CLUSTER_CERT_FILE" -storepass "$STORE_PASSWORD" -keypass "$KEY_PASSWORD" -noprompt

echo "Step6: Signing the certificate"
openssl x509 -req -CA "$CERT_AUTH_FILE" -CAkey $CERT_OUTPUT_PATH/ca-key -in "$CLUSTER_CERT_FILE" -out "${CLUSTER_CERT_FILE}-signed" -days "$DAYS_VALID" -CAcreateserial -passin pass:"$PASSWORD"

echo "Setp7: Import CA into keystore"
keytool -keystore "$KEY_STORE" -alias CARoot -import -file "$CERT_AUTH_FILE" -storepass "$STORE_PASSWORD" -keypass "$KEY_PASSWORD" -noprompt


echo "Setp8: Import signed certificate into keystore"
keytool -keystore "$KEY_STORE" -alias "${CLUSTER_NAME}" -import -file "${CLUSTER_CERT_FILE}-signed" -storepass "$STORE_PASSWORD" -keypass "$KEY_PASSWORD" -noprompt

1.1 配置Kafka



delete.topic.enable=true
auto.create.topics.enable=true

listeners=SASL_SSL://localhost:9093,PLAINTEXT://localhost:9092
advertised.listeners=SASL_SSL://localhost:9093,PLAINTEXT://localhost:9092
# inter.broker.listener.name=SASL_SSL

sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
#security.inter.broker.protocol=SASL_PLAINTEXT
security.inter.broker.protocol=SASL_SSL
ssl.endpoint.identification.algorithm=HTTPS

ssl.keystore.location=/Users/lcc/soft/kafka/kafka_2.11-1.1.0_author_scram/certificates/kafka.keystore.jks
ssl.keystore.password=ke123456
ssl.key.password=ke123456
ssl.truststore.location=/Users/lcc/soft/kafka/kafka_2.11-1.1.0_author_scram/certificates/kafka.truststore.jks
ssl.truststore.password=ke123456

ssl.client.auth=required
ssl.enabled.protocols=TLSv1.2,TLSv1.1,TLSv1
ssl.keystore.type=JKS
ssl.truststore.type=JKS
ssl.secure.random.implementation=SHA1PRNG




# ACL
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
allow.everyone.if.no.acl.found=true
super.users=User:admin

1.2 创建 SCRAM 证书

bin/kafka-configs.sh --zookeeper localhost:2181 --alter --add-config 'SCRAM-SHA-256=[password=admin-secret],SCRAM-SHA-512=[password=admin-secret]' --entity-type users --entity-name admin

1.3.配置zk

这里暂时不要用自己安装的zk，用kafka自带的zk，否则会有问题
kafka的zk

cc@lcc ~/soft/kafka/kafka_2.11-1.1.0_author_scram]$ pwd
/Users/lcc/soft/kafka/kafka_2.11-1.1.0_author_scram
[lcc@lcc ~/soft/kafka/kafka_2.11-1.1.0_author_scram]$ vi config/zookeeper.properties

authProvider.1=org.apache.zookeeper.server.auth.SASLAuthenticationProvider
requireClientAuthScheme=sasl
jaasLoginRenew=3600000

1.4 .启动zk

Add the zookeeper_jaas.conf file to the environment variable KAFKA_OPTS before starting zookeeper.

$ export KAFKA_OPTS="-Djava.security.auth.login.config=/KAFKA_HOME/config/zookeeper_jaas.conf"
$ bin/zookeeper-server-start.sh -daemon config/zookeeper.properties

这里要启动kafka自带的zk

1.5.启动kakfa

Add the kafka_server_jaas.conf file to the environment variable KAFKA_OPTS before starting kafka server.

$ export KAFKA_OPTS="-Djava.security.auth.login.config=/KAFKA_HOME/config/kafka_server_jaas.conf"
bin/kafka-server-start.sh -daemon config/server.properties

然后就大量报错：

connection to node 0 failed authentication due to: authentication failed due to invalid credentials with SASL mech
connection to node 0 failed authentication due to: authentication failed due to invalid credentials with SASL mech
connection to node 0 failed authentication due to: authentication failed due to invalid credentials with SASL mech

相似的错误是【Kafka】kafka This may indicate that authentication failed due to invalid credentials

尝试重新生成脚本也不对，使用hostname生成credentials也不对。

【Kafka】kafka Authentication failed credentials with SASL mech