kafka学习笔记十kafka-SSL安全认证机制

其他 2018-09-14 10:18:48 阅读次数: 0

一、生成配置cert

1.1生成 server keystore

[root@iZbp19pf0xgaj0oo0041u6Z cert]# keytool -keystore server.keystore.jks -alias kafka-server -validity 365 -keyalg RSA -genkey
Enter keystore password:  
Re-enter new password: 
What is your first and last name?
  [Unknown]:  hanwan
What is the name of your organizational unit?
  [Unknown]:  kafka
What is the name of your organization?
  [Unknown]:  kafka
What is the name of your City or Locality?
  [Unknown]:  HZ
What is the name of your State or Province?
  [Unknown]:  ZJ
What is the two-letter country code for this unit?
  [Unknown]:  CN
Is CN=hanwan, OU=kafka, O=kafka, L=HZ, ST=ZJ, C=CN correct?
  [no]:  YES^H^H

Enter key password for <kafka-server>
        (RETURN if same as keystore password):  
Re-enter new password: 

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore server.keystore.jks -destkeystore server.keystore.jks -deststoretype pkcs12".
[root@iZbp19pf0xgaj0oo0041u6Z cert]# keytool -list server.keystore.jks -v
Illegal option:  server.keystore.jks
keytool -list [OPTION]...

Lists entries in a keystore

Options:

 -rfc                            output in RFC style
 -alias <alias>                  alias name of the entry to process
 -keystore <keystore>            keystore name
 -storepass <arg>                keystore password
 -storetype <storetype>          keystore type
 -providername <providername>    provider name
 -providerclass <providerclass>  provider class name
 -providerarg <arg>              provider argument
 -providerpath <pathlist>        provider classpath
 -v                              verbose output
 -protected                      password through protected mechanism

Use "keytool -help" for all available commands
[root@iZbp19pf0xgaj0oo0041u6Z cert]# keytool -list -keystore server.keystore.jks -v
Enter keystore password:  
Keystore type: jks
Keystore provider: SUN

Your keystore contains 1 entry

Alias name: kafka-server
Creation date: Aug 30, 2018
Entry type: PrivateKeyEntry
Certificate chain length: 1
Certificate[1]:
Owner: CN=hanwan, OU=kafka, O=kafka, L=HZ, ST=ZJ, C=CN
Issuer: CN=hanwan, OU=kafka, O=kafka, L=HZ, ST=ZJ, C=CN
Serial number: 1a0a5446
Valid from: Thu Aug 30 09:13:14 CST 2018 until: Fri Aug 30 09:13:14 CST 2019
Certificate fingerprints:
         MD5:  DF:79:13:05:23:4F:56:FB:7C:E4:F1:33:C0:9F:53:61
         SHA1: CD:AE:2F:5B:F9:A1:11:4B:D5:8C:E2:52:2E:3E:0A:59:A2:F2:8A:64
         SHA256: 30:B1:0B:39:C1:AB:FC:AD:DD:86:C8:F8:F6:04:0C:71:A6:10:61:7D:37:91:2F:72:53:4C:A0:FC:57:7E:A9:2A
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions: 

#1: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: AB 31 16 F2 62 95 41 F2   2E C0 A2 49 A1 85 61 C1  .1..b.A....I..a.
0010: 0C C6 A1 83                                        ....
]
]



*******************************************
*******************************************



Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore server.keystore.jks -destkeystore server.keystore.jks -deststoretype pkcs12".

1.2生成client keystore

[root@iZbp19pf0xgaj0oo0041u6Z cert]# keytool -keystore client.keystore.jks -alias kafka-client -validity 365 -keyalg RSA -genkey    
Enter keystore password:  
Re-enter new password: 
What is your first and last name?
  [Unknown]:  hanwan
What is the name of your organizational unit?
  [Unknown]:  kafka
What is the name of your organization?
  [Unknown]:  kafka
What is the name of your City or Locality?
  [Unknown]:  HZ
What is the name of your State or Province?
  [Unknown]:  ZJ
What is the two-letter country code for this unit?
  [Unknown]:  CN
Is CN=hanwan, OU=kafka, O=kafka, L=HZ, ST=ZJ, C=CN correct?
  [no]:  yes

Enter key password for <kafka-client>
        (RETURN if same as keystore password):  
Re-enter new password: 

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore client.keystore.jks -destkeystore client.keystore.jks -deststoretype pkcs12".
[root@iZbp19pf0xgaj0oo0041u6Z cert]# ls
client.keystore.jks  server.keystore.jks

1.3 生成CA证书

[root@iZbp19pf0xgaj0oo0041u6Z cert]# openssl req -new -x509 -keyout ca.key -out ca.crt -days 365
Generating a 2048 bit RSA private key
...............+++
...............................................................+++
writing new private key to 'ca.key'
Enter PEM pass phrase:
Verifying - Enter PEM pass phrase:
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:CN
State or Province Name (full name) []:ZJ
Locality Name (eg, city) [Default City]:HZ
Organization Name (eg, company) [Default Company Ltd]:kafka
Organizational Unit Name (eg, section) []:kafka
Common Name (eg, your name or your server's hostname) []:hanwan
Email Address []:[email protected]
[root@iZbp19pf0xgaj0oo0041u6Z cert]# ls
ca.crt  ca.key  client.keystore.jks  server.keystore.jks

1.4 import the ca crt to server and client trust keystore.

[root@iZbp19pf0xgaj0oo0041u6Z cert]# keytool -keystore server.trustkeystore.jks -alias CARoot -import -file ca.crt   
Enter keystore password:  
Re-enter new password: 
Owner: [email protected], CN=hanwan, OU=kafka, O=kafka, L=HZ, ST=ZJ, C=CN
Issuer: [email protected], CN=hanwan, OU=kafka, O=kafka, L=HZ, ST=ZJ, C=CN
Serial number: 9f37ba082b41d6b1
Valid from: Thu Aug 30 09:30:54 CST 2018 until: Fri Aug 30 09:30:54 CST 2019
Certificate fingerprints:
         MD5:  F7:B0:95:61:BE:30:E9:C1:22:BF:81:91:5D:AA:9E:CE
         SHA1: 26:9E:05:CE:AF:E5:53:19:1C:A9:A7:77:1F:21:88:C8:7F:F0:BC:1D
         SHA256: 9C:78:E5:86:48:91:EC:90:38:B3:38:C9:76:BE:C8:C0:B5:54:0B:7F:BD:32:99:10:9A:24:0E:3B:D8:87:C7:6F
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions: 

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: EB E7 38 B1 04 E3 CA 0C   30 E1 1E C1 55 95 42 0D  ..8.....0...U.B.
0010: CC 07 E1 D3                                        ....
]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:true
  PathLen:2147483647
]

#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: EB E7 38 B1 04 E3 CA 0C   30 E1 1E C1 55 95 42 0D  ..8.....0...U.B.
0010: CC 07 E1 D3                                        ....
]
]

Trust this certificate? [no]:  yes
Certificate was added to keystore
[root@iZbp19pf0xgaj0oo0041u6Z cert]# ls
ca.crt  ca.key  client.keystore.jks  server.keystore.jks  server.trustkeystore.jks

[root@iZbp19pf0xgaj0oo0041u6Z cert]# keytool -keystore client.trustkeystore.jks -alias CARoot -import -file ca.crt
Enter keystore password:  
Re-enter new password: 
Owner: [email protected], CN=hanwan, OU=kafka, O=kafka, L=HZ, ST=ZJ, C=CN
Issuer: [email protected], CN=hanwan, OU=kafka, O=kafka, L=HZ, ST=ZJ, C=CN
Serial number: 9f37ba082b41d6b1
Valid from: Thu Aug 30 09:30:54 CST 2018 until: Fri Aug 30 09:30:54 CST 2019
Certificate fingerprints:
         MD5:  F7:B0:95:61:BE:30:E9:C1:22:BF:81:91:5D:AA:9E:CE
         SHA1: 26:9E:05:CE:AF:E5:53:19:1C:A9:A7:77:1F:21:88:C8:7F:F0:BC:1D
         SHA256: 9C:78:E5:86:48:91:EC:90:38:B3:38:C9:76:BE:C8:C0:B5:54:0B:7F:BD:32:99:10:9A:24:0E:3B:D8:87:C7:6F
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions: 

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: EB E7 38 B1 04 E3 CA 0C   30 E1 1E C1 55 95 42 0D  ..8.....0...U.B.
0010: CC 07 E1 D3                                        ....
]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:true
  PathLen:2147483647
]

#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: EB E7 38 B1 04 E3 CA 0C   30 E1 1E C1 55 95 42 0D  ..8.....0...U.B.
0010: CC 07 E1 D3                                        ....
]
]

Trust this certificate? [no]:  yes
Certificate was added to keystore
[root@iZbp19pf0xgaj0oo0041u6Z cert]# ls
ca.crt  ca.key  client.keystore.jks  client.trustkeystore.jks  server.keystore.jks  server.trustkeystore.jks

1.5 export the cert

[root@iZbp19pf0xgaj0oo0041u6Z cert]# keytool -keystore server.keystore.jks -alias kafka-server -certreq -file kafka.server.crt   
Enter keystore password:  

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore server.keystore.jks -destkeystore server.keystore.jks -deststoretype pkcs12".
[root@iZbp19pf0xgaj0oo0041u6Z cert]# ls
ca.crt  ca.key  client.keystore.jks  client.trustkeystore.jks  kafka.server.crt  server.keystore.jks  server.trustkeystore.jks
[root@iZbp19pf0xgaj0oo0041u6Z cert]# keytool -keystore client.keystore.jks -alias kafka-client -certreq -file kafka.client.crt      
Enter keystore password:  

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore client.keystore.jks -destkeystore client.keystore.jks -deststoretype pkcs12".
[root@iZbp19pf0xgaj0oo0041u6Z cert]# ls
ca.crt  client.keystore.jks       kafka.client.crt  server.keystore.jks
ca.key  client.trustkeystore.jks  kafka.server.crt  server.trustkeystore.jks

1.6 use the ca cert do signed task

[root@iZbp19pf0xgaj0oo0041u6Z cert]# openssl x509 -req -CA ca.crt -CAkey ca.key -in kafka.server.crt -out kafka.server.signed.crt -days 365 -CAcreateserial 
Signature ok
subject=/C=CN/ST=ZJ/L=HZ/O=kafka/OU=kafka/CN=hanwan
Getting CA Private Key
Enter pass phrase for ca.key:
[root@iZbp19pf0xgaj0oo0041u6Z cert]# openssl x509 -req -CA ca.crt -CAkey ca.key -in kafka.client.crt -out kafka.client.signed.crt -days 365 -CAcreateserial             
Signature ok
subject=/C=CN/ST=ZJ/L=HZ/O=kafka/OU=kafka/CN=hanwan
Getting CA Private Key
Enter pass phrase for ca.key:
[root@iZbp19pf0xgaj0oo0041u6Z cert]# ls
ca.crt  ca.srl               client.trustkeystore.jks  kafka.client.signed.crt  kafka.server.signed.crt  server.trustkeystore.jks
ca.key  client.keystore.jks  kafka.client.crt          kafka.server.crt         server.keystore.jks
[root@iZbp19pf0xgaj0oo0041u6Z cert]#

1.7 import the ca cert into server and client keystore.

[root@iZbp19pf0xgaj0oo0041u6Z cert]# keytool -keystore server.keystore.jks -alias CARoot -import -file ca.crt 
Enter keystore password:  
Owner: [email protected], CN=hanwan, OU=kafka, O=kafka, L=HZ, ST=ZJ, C=CN
Issuer: [email protected], CN=hanwan, OU=kafka, O=kafka, L=HZ, ST=ZJ, C=CN
Serial number: 9f37ba082b41d6b1
Valid from: Thu Aug 30 09:30:54 CST 2018 until: Fri Aug 30 09:30:54 CST 2019
Certificate fingerprints:
         MD5:  F7:B0:95:61:BE:30:E9:C1:22:BF:81:91:5D:AA:9E:CE
         SHA1: 26:9E:05:CE:AF:E5:53:19:1C:A9:A7:77:1F:21:88:C8:7F:F0:BC:1D
         SHA256: 9C:78:E5:86:48:91:EC:90:38:B3:38:C9:76:BE:C8:C0:B5:54:0B:7F:BD:32:99:10:9A:24:0E:3B:D8:87:C7:6F
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions: 

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: EB E7 38 B1 04 E3 CA 0C   30 E1 1E C1 55 95 42 0D  ..8.....0...U.B.
0010: CC 07 E1 D3                                        ....
]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:true
  PathLen:2147483647
]

#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: EB E7 38 B1 04 E3 CA 0C   30 E1 1E C1 55 95 42 0D  ..8.....0...U.B.
0010: CC 07 E1 D3                                        ....
]
]

Trust this certificate? [no]:  yes
Certificate was added to keystore

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore server.keystore.jks -destkeystore server.keystore.jks -deststoretype pkcs12".


[root@iZbp19pf0xgaj0oo0041u6Z cert]# keytool -keystore client.keystore.jks -alias CARoot -import -file ca.crt
Enter keystore password:  
Owner: [email protected], CN=hanwan, OU=kafka, O=kafka, L=HZ, ST=ZJ, C=CN
Issuer: [email protected], CN=hanwan, OU=kafka, O=kafka, L=HZ, ST=ZJ, C=CN
Serial number: 9f37ba082b41d6b1
Valid from: Thu Aug 30 09:30:54 CST 2018 until: Fri Aug 30 09:30:54 CST 2019
Certificate fingerprints:
         MD5:  F7:B0:95:61:BE:30:E9:C1:22:BF:81:91:5D:AA:9E:CE
         SHA1: 26:9E:05:CE:AF:E5:53:19:1C:A9:A7:77:1F:21:88:C8:7F:F0:BC:1D
         SHA256: 9C:78:E5:86:48:91:EC:90:38:B3:38:C9:76:BE:C8:C0:B5:54:0B:7F:BD:32:99:10:9A:24:0E:3B:D8:87:C7:6F
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3

Extensions: 

#1: ObjectId: 2.5.29.35 Criticality=false
AuthorityKeyIdentifier [
KeyIdentifier [
0000: EB E7 38 B1 04 E3 CA 0C   30 E1 1E C1 55 95 42 0D  ..8.....0...U.B.
0010: CC 07 E1 D3                                        ....
]
]

#2: ObjectId: 2.5.29.19 Criticality=false
BasicConstraints:[
  CA:true
  PathLen:2147483647
]

#3: ObjectId: 2.5.29.14 Criticality=false
SubjectKeyIdentifier [
KeyIdentifier [
0000: EB E7 38 B1 04 E3 CA 0C   30 E1 1E C1 55 95 42 0D  ..8.....0...U.B.
0010: CC 07 E1 D3                                        ....
]
]

Trust this certificate? [no]:  yes
Certificate was added to keystore

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore client.keystore.jks -destkeystore client.keystore.jks -deststoretype pkcs12".

1.8 import the cert to server and client keystore after signed by CA cert.

[root@iZbp19pf0xgaj0oo0041u6Z cert]# keytool -keystore server.keystore.jks -alias kafka-server -import -file kafka.server.signed.crt 
Enter keystore password:  
Certificate reply was installed in keystore

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore server.keystore.jks -destkeystore server.keystore.jks -deststoretype pkcs12".
[root@iZbp19pf0xgaj0oo0041u6Z cert]# keytool -keystore client.keystore.jks -alias kafka-client -import -file kafka.client.signed.crt 
Enter keystore password:  
Certificate reply was installed in keystore

Warning:
The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore client.keystore.jks -destkeystore client.keystore.jks -deststoretype pkcs12".

1.9 拷贝jks文件到其他集群服务器上

[root@iZbp19pf0xgaj0oo0041u6Z cert]# scp /usr/local/*.jks [email protected]:/usr/local/cert/
[email protected]'s password: 
/usr/local/*.jks: No such file or directory
[root@iZbp19pf0xgaj0oo0041u6Z cert]# scp /usr/local/cert/*.jks [email protected]:/usr/local/cert/ 
[email protected]'s password: 
client.keystore.jks                                                                               100% 4189     6.5MB/s   00:00    
client.trustkeystore.jks                                                                          100% 1032     2.0MB/s   00:00    
server.keystore.jks                                                                               100% 4188     7.0MB/s   00:00    
server.trustkeystore.jks                                                                          100% 1032     2.1MB/s   00:00    
[root@iZbp19pf0xgaj0oo0041u6Z cert]# ls

二、分别修改kafka配置文件server.properties

[root@iZbp1c8mn5lner8nhvreblZ config]# cp server.properties server-ssl.properties 
[root@iZbp1c8mn5lner8nhvreblZ config]# vim server-ssl.properties 
ssl.keystore.location=/usr/local/cert/server.keystore.jks
ssl.keystore.password=123456
ssl.key.password=123456
ssl.truststore.location=/usr/local/cert/server.trustkeystore.jks
ssl.truststore.password=123456
ssl.client.auth=required
listeners=PLAINTEXT://121.40.200.37:9092,SSL://121.40.200.37:9093
security.inter.broker.protocol=SSL

并分别启动kafka程序

[root@iZbp1c8mn5lner8nhvreblZ kafka_2.11-0.11.0.1]# bin/kafka-server-start.sh -daemon config/server-ssl.properties

3.创建clientssl.properties

"clientssl.properties" [New File]                                                                                 0,0-1         All
security.protocol=SSL
ssl.truststore.location=/usr/local/cert/client.trustkeystore.jks
ssl.truststore.password=123456
ssl.keystore.passwrod=123456
ssl.key.password=123456
ssl.keystore.location=/usr/local/cert/client.keystore.jks

4.测试

4.1 创建测试用的topic   ssl-topic

[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# bin/kafka-topics.sh --create --topic ssl-topic --partitions 3 --replication-factor 3 --zookeeper 10.174.32.122:2181,10.117.15.224:2181,10.168.96.198:2181
Created topic "ssl-topic".
[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# bin/kafka-topics.sh --describe --topic ssl-topic --zookeeper 10.174.32.122:2181
Topic:ssl-topic PartitionCount:3        ReplicationFactor:3     Configs:
        Topic: ssl-topic        Partition: 0    Leader: 1       Replicas: 1,2,0 Isr: 1,2,0
        Topic: ssl-topic        Partition: 1    Leader: 2       Replicas: 2,0,1 Isr: 2,0,1
        Topic: ssl-topic        Partition: 2    Leader: 0       Replicas: 0,1,2 Isr: 0,1,2

4.2 debug the ssl configuration by openssl tooling

[root@iZbp19pf0xgaj0oo0041u6Z config]# openssl s_client -debug -connect 120.55.125.58:9093 -tls1

4.3 produce the message by console producer script

[root@iZbp14ouog5ocoeakj39prZ kafka_2.11-0.11.0.1]# bin/kafka-console-producer.sh --topic ssl-topic --broker-list 120.55.125.58:9093,120.26.198.248:9093,121.40.200.37:9093 --producer.config config/clientssl.properties 
>hello
>ssl pass
>success
>

4.4 consume the message by console consume script

[root@iZbp19pf0xgaj0oo0041u6Z kafka_2.11-0.11.0.1]# bin/kafka-console-consumer.sh --topic ssl-topic --bootstrap-server 120.55.125.58:9093,120.26.198.248:9093,121.40.200.37:9093 --from-beginning --consumer.config config/clientssl.properties
success
hello
ssl pass

猜你喜欢

转载自blog.csdn.net/hwhanwan/article/details/82216016
今日推荐
美国拟限制 AI 大模型出口中国和俄罗斯
苹果将与 OpenAI 达成协议，将 ChatGPT 应用于 iPhone
openKylin 社区生态委员会第六次会议圆满召开
阿里云正式发布通义千问 2.5
Python 3.13 发布首个 Beta：实验性自由线程模式和 JIT、改进交互式解释器
Stack Overflow 拿我的代码去训练 AI 大模型，还封了我的账号
Pop!_OS 的 COSMIC 桌面完成 App Store 上架工作
报告：Django 仍然是 74% 开发者的首选
《2024 年一季度互联网投融资运行情况》研究报告
15 年前上了“FFmpeg 耻辱柱”，今天他还得谢谢咱——腾讯QQPlayer一雪前耻？
TIOBE 5 月榜单：Fortran “复活”进入 Top 10
GCC 14.1 发布
周排行
NEFU 117 素数个数的位数
Closest Common Ancestors (Lca,tarjan)
ELK部署
【转载】Hive笔记整理（三）
SQL语句（一）基本表的定义
关于Java web开发中的MySQL的事务语句
MFC创建自定义窗体
如何用一句话激怒程序员？
《逆袭大学》文摘——9.4 基础和应用的平衡中找到大学的节奏
【spring源码分析】@Value注解原理
每日归档
更多
2024-05-11(38)
2024-05-10(38)
2024-05-09(35)
2024-05-08(42)
2024-05-07(14)
2024-05-06(40)
2024-05-05(0)
2024-05-04(7)
2024-05-03(19)
2024-05-02(0)

代码天地 © 2018 codetd.com

境外服务器   最新电视剧2022