mall商城之k8s部署-4

一、k8s部署应用服务

1）master拷贝yaml

#将源码文件
mkdir  -p /opt/mall
scp -r [email protected]:/opt/ mall-swarm-1.0.2/document/k8s/ /opt/mall/

2）批量修改镜像地址

#进入yaml部署目录
cd /opt/mall/k8s

#批量修改镜像地址

find . -name "*yaml*"  |xargs -i sed  -i "s#macrodocker/#192.168.4.120/mall/#g" {}

#批量查看
find . -name "*yaml*" |xargs -i  cat {}|grep image

3）批量修改nacos地址

• 批量修改nacos地址，改为对的服务器IP地址。
• 本次nacos地址为：192.168.4.119:8848
• 访问地址为：http://192.168.4.119:8848/nacos/
• • 账户：nacos
• • 密码：nacos

#修改
find . -name "*yaml*"  |xargs -i sed -i  "s#192.168.3.101:8848#192.168.4.119:8848#g" {}
 
#查看
find . -name "*yaml*"  |xargs -i cat  {}|grep 8848

3）创建命名空间

[root@k8s-master-01 k8s]# kubectl create ns ms
namespace/ms created

[root@k8s-master-01 k8s]# kubectl get ns
NAME              STATUS   AGE
default           Active   6d10h
kube-node-lease   Active   6d10h
kube-public       Active   6d10h
kube-system       Active   6d10h
ms                Active   23s

4）创建取sercet

配置k8s拉取sercet，创建命名空间

 kubectl create secret docker-registry  registry-pull-secret-ms \
 --docker-server=192.168.4.120 \
 --docker-username=admin \
 --docker-password=Harbor12345 \
 [email protected] -n ms

[root@k8s-master-01 k8s]# kubectl get secret
NAME                  TYPE                                  DATA   AGE
default-token-x6729   kubernetes.io/service-account-token   3      6d10h
[root@k8s-master-01 k8s]# kubectl get secret -n ms
NAME                      TYPE                                  DATA   AGE
registry-pull-secret-ms   kubernetes.io/dockerconfigjson        1      19s

5）配置yaml

需要拉取镜像的k8s节点皆需要配置化harbor认证

#让Docker支持http上传镜像
echo '{ "insecure-registries":["192.168.4.119"] }'  >/etc/docker/daemon.json


#修改配置后需要使用如下命令使配置生效
systemctl daemon-reload

#重新启动Docker服务
 systemctl restart docker   

#开启防火墙的Docker构建端口
firewall-cmd --zone=public --add-port=2375/tcp --permanent
firewall-cmd --reload

#登录harbor
docker login -uadmin  -pHarbor12345  192.168.4.120

将secret认证加入部署的yaml中

#配置应用服务的yaml
    spec:
#加入    
#######################################
      imagePullSecrets:
      - name: registry-pull-secret-ms
#######################################
      containers:

#修改镜像名称
      containers:
      - name: 服务名
        image: 192.168.4.120/mall/镜像名:版本号

6）对象存储oss

【使用说明】

• http://www.macrozheng.com/mall/architect/mall_arch_10.html#oss%E7%9A%84%E7%9B%B8%E5%85%B3%E8%AE%BE%E7%BD%AE

【定义策略】

• http://www.macrozheng.com/mall/architect/mall_arch_10.html#%E6%B7%BB%E5%8A%A0osscontroller%E5%AE%9A%E4%B9%89%E6%8E%A5%E5%8F%A3

【可参考】

• https://blog.csdn.net/zhenghongcs/article/details/99311882
• http://www.macrozheng.com/mall/architect/mall_arch_10.html#oss

oss简介

阿里云对象存储服务（Object Storage Service，简称 OSS），是阿里云提供的海量、安全、低成本、高可靠的云存储服务。OSS可用于图片、音视频、日志等海量文件的存储。各种终端设备、Web网站程序、移动应用可以直接向OSS写入或读取数据

OSS中的相关概念：

• Endpoint：访问域名，通过该域名可以访问OSS服务的API，进行文件上传、下载等操作。
• Bucket：存储空间，是存储对象的容器，所有存储对象都必须隶属于某个存储空间。
• Object：对象，对象是 OSS 存储数据的基本单元，也被称为 OSS 的文件。
• AccessKey：访问密钥，指的是访问身份验证中用到的 AccessKeyId 和 AccessKeySecret

创建存储空间

• 查看AccessKey秘钥

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-o1Jw8Zz8-1681214409760)(http://m.qpic.cn/psc?/V53GU2po3e4Vc929i8d01BP45G32VjvB/ruAMsa53pVQWN7FLK88i5r5EOkKUjMyIcDRdM3J8F.4Kb4YGmMj285OVYpXqDyZpBlwZkYC7LrQ6whGmbsg8YL3QShyaal3T0odEz8zF8!/b&bo=eAGqAXgBqgEDFzI!&rf=viewer_4&t=5)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-yxEU8YC8-1681214409761)(http://m.qpic.cn/psc?/V53GU2po3e4Vc929i8d01BP45G32VjvB/ruAMsa53pVQWN7FLK88i5r5EOkKUjMyIcDRdM3J8F.5wxA5HXC3fvb5m1NA6ljHEowYuZxV2sktkeocM*mwptcu0LFk7VKhYqpDCstDoHY8!/b&bo=PgIJAT4CCQEDFzI!&rf=viewer_4&t=5)]

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-p6p74FEA-1681214409762)(http://m.qpic.cn/psc?/V53GU2po3e4Vc929i8d01BP45G32VjvB/ruAMsa53pVQWN7FLK88i5nUwWAaiphFYf8NCLVrIYicf*kgNZW.qSSuTRYH4A7ES39AKGNV2lTFok5r7Wb.nyUyaaRpWeVMfrPuPl0JpY!/b&bo=PgL5AD4C.QADFzI!&rf=viewer_4&t=5)]

查看 Secret 

#AccessKey ID
LTAI5tQRDpEA2aQANwDPXiTr
#AccessKey Secret
j3gn2K7JLv8vOscKEvEPrvsvPBn0oY

• 进入控制台新建Bucket

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-npCGLICK-1681214409762)(http://m.qpic.cn/psc?/V53GU2po3e4Vc929i8d01BP45G32VjvB/ruAMsa53pVQWN7FLK88i5tCvLvqZsrmQum0aM2ySgfRZebOhe6KE4knv0hgYUTfg.lzvs840li2beI424WwBafSwbCvMrhfqUpbzXCBOw!/b&bo=PgLpAT4C6QEDFzI!&rf=viewer_4&t=5)]

• 跨域资源共享（CORS）的设置

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Im7a3YTY-1681214409762)(http://m.qpic.cn/psc?/V53GU2po3e4Vc929i8d01BP45G32VjvB/ruAMsa53pVQWN7FLK88i5tCvLvqZsrmQum0aM2ySgfRZebOhe6KE4knv0hgYUTfg.lzvs840li2beI424WwBafSwbCvMrhfqUpbzXCBOw!/b&bo=PgLpAT4C6QEDFzI!&rf=viewer_4&t=5)]

• 进行跨域规则设置

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-Z7uIHC1a-1681214409763)(http://m.qpic.cn/psc?/V53GU2po3e4Vc929i8d01BP45G32VjvB/ruAMsa53pVQWN7FLK88i5tCvLvqZsrmQum0aM2ySgfTvcpYlEOtYl*cBFq9tjQm205YVb7QbwxMV6CNCkSe3Z7aG9t8bKHmr6SUr7iHOHl0!/b&bo=PgICAT4CAgEDFzI!&rf=viewer_4&t=5)]

7）查看nacos

-【修改nacos】http://www.macrozheng.com/mall/deploy/mall_swarm_deploy_k8s.html#%E4%BF%AE%E6%94%B9nacos%E9%85%8D%E7%BD%AE

• 将项目config目录下的所有配置都添加到Nacos中，由于应用服务都部署到了K8S中，所以需要修改相关配置。
• 将配置信息添加到Nacos中后显示信息如下，Nacos访问地址：http://192.168.4.119:8848/nacos/index.html

1、导入配置文件

• 将源码包下载到电脑上——解压——找到conf目录。
• 在admin目录下创建DEFAULT_GROUP目录，并且将yaml移动到目录中，压缩成zip文件。
• 其余monitor/portal/search/gateway/demo文件下也创建DEFAULT_GROUP，并且压缩。
• nacos中public命名空间中点击上传配置文件——上传压缩文件zip。

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-zpJPnQjV-1681214409763)(http://m.qpic.cn/psc?/V53GU2po3e4Vc929i8d01BP45G32VjvB/ruAMsa53pVQWN7FLK88i5ldNEVTyUw0oZZC44RWvrH.IQIn**cFLvxErKhOdo66iJAAzAk5BvJtp8uf3iS6A6xSi0FuiP1bo8A99wctVk!/b&bo=PgLUAT4C1AEDFzI!&rf=viewer_4&t=5)]

2、修改配置文件

mall-admin-dev.yaml

aliyun:
  oss:
    endpoint: oss-cn-beijing.aliyuncs.com # oss对外服务的访问域名
    accessKeyId: LTAI5tQRDpEA2aQANwDPXiTr # 访问身份验证中用到用户标识
    accessKeySecret: j3gn2K7JLv8vOscKEvEPrvsvPBn0oY # 用户用于加密签名字符串和oss用来验证签名字符串的密钥
    bucketName: macro-oss-jia # oss的存储空间
    policy:
      expire: 300 # 签名有效期(S)
    maxSize: 10 # 上传文件大小(M)
    callback: http://localhost:8080/aliyun/oss/callback   #文件上传成功后的回调地址
    dir:
      prefix: mall/images/ # 上传文件夹路径前缀
spring:
  datasource:
    url: jdbc:mysql://192.168.4.119:3306/mall?useUnicode=true&characterEncoding=utf-8&serverTimezone=Asia/Shanghai
    username: root
    password: root
  redis:
    host: 192.168.4.119 # Redis服务器地址
    database: 0 # Redis数据库索引（默认为0）
    port: 6379  # Redis服务器连接端口
    password: # Redis服务器连接密码（默认为空）
    timeout: 3000ms # 连接超时时间（毫秒）
logging:
  level:
    root: info
    com.macro.mall: debug
logstash:
  host: 192.168.4.119    #改为Elk地址，将pod的日志传送到ELK服务器上。

mall-admin-prod.yaml

aliyun:
  oss:
    endpoint: oss-cn-beijing.aliyuncs.com # oss对外服务的访问域名
    accessKeyId: LTAI5tQRDpEA2aQANwDPXiTr # 访问身份验证中用到用户标识
    accessKeySecret: j3gn2K7JLv8vOscKEvEPrvsvPBn0oY # 用户用于加密签名字符串和oss用来验证签名字符串的密钥
    bucketName: macro-oss-jia # oss的存储空间
    policy:
      expire: 300 # 签名有效期(S)
    maxSize: 10 # 上传文件大小(M)
    callback: http://localhost:8080/aliyun/oss/callback   #文件上传成功后的回调地址
    dir:
      prefix: mall/images/ # 上传文件夹路径前缀
spring:
  datasource:
    url: jdbc:mysql://192.168.4.119:3306/mall?useUnicode=true&characterEncoding=utf-8&serverTimezone=Asia/Shanghai
    username: root
    password: root
  redis:
    host: 192.168.4.119 # Redis服务器地址
    database: 0 # Redis数据库索引（默认为0）
    port: 6379  # Redis服务器连接端口
    password: # Redis服务器连接密码（默认为空）
    timeout: 3000ms # 连接超时时间（毫秒）
logging:
  level:
    root: info
    com.macro.mall: debug
logstash:
  host: 192.168.4.119

mall-demo-dev.yaml

spring:
  datasource:
    url: jdbc:mysql://192.168.4.119:3306/mall?useUnicode=true&characterEncoding=utf-8&serverTimezone=Asia/Shanghai
    username: root
    password: root
logging:
  level:
    root: info
    com.macro.mall: debug
logstash:
  host: 192.168.4.119

mall-demo-prod.yaml

spring:
  datasource:
    url: jdbc:mysql://192.168.4.119:3306/mall?useUnicode=true&characterEncoding=utf-8&serverTimezone=Asia/Shanghai
    username: reader
    password: 123456
logging:
  file:
    path: /var/logs
  level:
    root: info
    com.macro.mall: info
logstash:
  host: 192.168.4.119

mall-gateway-dev.yaml

spring:
  redis:
    host: 192.168.4.119   # Redis服务器地址
    database: 0   # Redis数据库索引（默认为0）
    port: 6379    # Redis服务器连接端口
    password:     # Redis服务器连接密码（默认为空）
    timeout: 3000ms # 连接超时时间（毫秒）
  security:
    oauth2:
      resourceserver:
        jwt:
          jwk-set-uri: 'http://mall-gateway-service.ms:8201/mall-auth/rsa/publicKey'
logging:
  level:
    root: info
    com.macro.mall: debug
logstash:
  host: 192.168.4.119

mall-gateway-prod.yaml

spring:
  redis:
    host: 192.168.4.119 # Redis服务器地址
    database: 0 # Redis数据库索引（默认为0）
    port: 6379 # Redis服务器连接端口
    password: #不设置密码
  security:
    oauth2:
      resourceserver:
        jwt:
          jwk-set-uri: 'http://mall-gateway-service.ms:8201/mall-auth/rsa/publicKey'
logging:
  file:
    path: /var/logs
  level:
    root: info
    com.macro.mall: info
logstash:
  host: 192.168.4.119

mall-portal-dev.yaml

spring:
  datasource:
    url: jdbc:mysql://192.168.4.119:3306/mall?useUnicode=true&characterEncoding=utf-8&serverTimezone=Asia/Shanghai
    username: root
    password: root
  data:
    mongodb:
      host: 192.168.4.119
      port: 27017
      database: mall-port
  redis:
    host: 192.168.4.119    # Redis服务器地址
    database: 0    # Redis数据库索引（默认为0）
    port: 6379     # Redis服务器连接端口
    password:      # Redis服务器连接密码（默认为空）
  rabbitmq:
    host:  192.168.4.119
    port: 5672
    virtual-host: /mall
    username: mall
    password: mall
# 日志配置
logging:
  level:
    root: info
    com.macro.mall: debug
logstash:
  host: 192.168.4.119

mall-portal-prod.yaml

spring:
  datasource:
    url: jdbc:mysql://192.168.4.119:3306/mall?useUnicode=true&characterEncoding=utf-8&serverTimezone=Asia/Shanghai
    username: root
    password: root
  data:
    mongodb:
      host: 192.168.4.119
      port: 27017
      database: mall-port
  redis:
    host: 192.168.4.119  # Redis服务器地址
    database: 0 # Redis数据库索引（默认为0）
    port: 6379 # Redis服务器连接端口
    password: #不设置密码
  rabbitmq:
    host: 192.168.4.119
    port: 5672
    virtual-host: /mall
    username: mall
    password: mall
    publisher-confirms: true #如果对异步消息需要回调必须设置为true
logging:
  file:
    path: /var/logs
  level:
    root: info
    com.macro.mall: info
logstash:
  host: 192.168.4.119

mall-search-dev.yaml

spring:
  datasource:
    url: jdbc:mysql://192.168.4.119:3306/mall?useUnicode=true&characterEncoding=utf-8&serverTimezone=Asia/Shanghai
    username: root
    password: root
  elasticsearch:
    rest:
      uris: http://192.168.4.119:9200
logging:
  level:
    root: info
    com.macro.mall: debug
logstash:
  host: 192.168.4.119

mall-search-prod.yaml

spring:
  datasource:
    url: jdbc:mysql://192.168.4.119:3306/mall?useUnicode=true&characterEncoding=utf-8&serverTimezone=Asia/Shanghai
    username: reader
    password: 123456
  elasticsearch:
    rest:
      uris: http://192.168.4.119:9200
management:
  health:
    elasticsearch:
      response-timeout: 1000ms #加大健康检查超时时间
logging:
  file:
    path: /var/logs
  level:
    root: info
    com.macro.mall: info
logstash:
  host: 192.168.4.119

注意：如果第一有很多报错，可以将所有服务全部干掉重新部署。

8）部署到ms命名空间

#批量更改命名空间
修改
find . -name "*yaml*"  |xargs -i sed  -i  "s/default/ms/g" {}
 
#查看
find . -name "*yaml*"  |xargs -i cat  {}|grep namespace

#部署
[root@k8s-master-01 k8s]# kubectl apply  -f  . -n ms
deployment.apps/mall-admin-deployment created
service/mall-admin-service created
deployment.apps/mall-auth-deployment created
service/mall-auth-service created
deployment.apps/mall-gateway-deployment created
service/mall-gateway-service created
deployment.apps/mall-monitor-deployment created
service/mall-monitor-service created
deployment.apps/mall-portal-deployment created
service/mall-portal-service created
deployment.apps/mall-search-deployment created
service/mall-search-service created

#查看
[root@k8s-master-01 k8s]#  kubectl get pods,svc -n ms 
NAME                                           READY   STATUS    RESTARTS   AGE
pod/mall-admin-deployment-58b9d86fb4-x2h4p     1/1     Running   0          14s
pod/mall-auth-deployment-bd5d8f8ff-jlppx       1/1     Running   0          14s
pod/mall-gateway-deployment-697475465c-h25xv   1/1     Running   0          14s
pod/mall-monitor-deployment-5fc45d5fbb-59wjb   1/1     Running   0          13s
pod/mall-portal-deployment-594857bf67-9b8qr    1/1     Running   0          13s
pod/mall-search-deployment-78cf844bb7-g4td2    1/1     Running   0          13s

NAME                           TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
service/mall-admin-service     ClusterIP   10.105.236.153   <none>        8080/TCP         14s
service/mall-auth-service      ClusterIP   10.106.183.1     <none>        8401/TCP         14s
service/mall-gateway-service   NodePort    10.111.67.220    <none>        8201:30201/TCP   14s
service/mall-monitor-service   ClusterIP   10.107.86.88     <none>        8101/TCP         13s
service/mall-portal-service    ClusterIP   10.100.205.139   <none>        8085/TCP         13s
service/mall-search-service    ClusterIP   10.96.10.87      <none>        8081/TCP         13s