K8s自动化集群环境搭建

一、环境规划

1.1 集群类型

Kubernetes集群大体上分为两类：一主多从和多主多从

• **一主多从：**一台master节点和多台node节点，搭建简单，但是有单机故障风险，适用于测试环境
• **多主多从：**多台master节点和多台node节点，搭建麻烦，安全性高，适用于生产环境

1.2 安装方式

Kubernetes有多种部署方式，目前主流的方式有 kubeadm 、 minikube 、 二进制包

• **Minikube：**一个用于快速搭建单节点kubernetes的工具
• **Kubeadm：**一个用于快速搭建kubernetes集群的工具，https: /kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/
• **二进制包：**从官网下载每个组件的二进制包，依次去安装，此方式对于理解kubernetes组件更加有 效，https:/github.com/kubernetes/kubernetes

说明：现在需要安装kubernetes的集群环境，但是又不想过于麻烦，所有选择使用kubeadm方式

二、环境搭建

1、主机安装

说明：
本次环境搭建需要安装三台Linux系统（一主二从），内置centos7.5系统，然后在每台linux中分别安
装docker。kubeadm（1.25），kubelet（1.25.4），kubelet（1.25.4）.

安装虚拟机过程中注意下面选项的设置：
操作系统环境：cpu2个 内存2G 硬盘50G centos7+
语言：中文简体/英文
软件选择：基础设施服务器
分区选择：自动分区/手动分区
网络配置：按照下面配置网络地址信息
网络地址：192.168.47.（137、10、20）
子网掩码：255.255.255.0
默认网关：192.168.47.254
DNS：8.8.8.8
主机名设置：
Master节点：master
Node节点：node1
Node节点：node2

角色	ip地址	组件
master	192.168.47.137	docker，kubectl，kubeadm，kubelet
node1	192.168.47.20	docker，kubectl，kubeadm，kubelet
node2	192.168.47.30	docker，kubectl，kubeadm，kubelet

以下环境准备需要三台同步做

2、主机名分配

（三个节点都需要做）

[root@master ~]# cat /etc/hosts 
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
192.168.47.137 master
192.168.47.20  node1
192.168.47.30  node2
[root@master ~]#scp /etc/hosts [email protected]:/etc/hosts

3、时钟同步

# kubernetes要求集群中的节点时间必须精确一致，这里使用chronyd服务从网络同步时间，企业
中建议配置内部的时间同步服务器
-master节点
[root@master ~]#vim /etc/chrony.conf
local stratum 10
[root@master ~]#systemctl restart chronyd.service
[root@master ~]#systemctl enable chronyd.service
[root@master ~]#hwclock -w
-node1节点
[root@node1 ~]#vim /etc/chrony.conf
server master.example.com iburst
.
[root@node1 ~]#systemctl restart chronyd.service
[root@node1 ~]#systemctl enable chronyd.service
[root@node1 ~]#hwclock -w
-node2节点
[root@node2 ~]#vim /etc/chrony.conf
server master.example.com iburst
.
[root@node2 ~]#systemctl restart chronyd.service
[root@node2 ~]#systemctl enable chronyd.service
[root@node2 ~]#hwclock -w

4. 禁用firewalld、selinux、postfix

（三个节点都做）

-master节点
[root@master ~]#systemctl disable -now firewalld
[root@master ~]#sed -i 's/enforcing/disabled/' /etc/selinux/config
[root@master ~]#setenforce 0
[root@master ~]#systemctl stop postfix
[root@master ~]#systemctl disable postfix

5. 禁用swap分区

-master节点
[root@master ~]#vim /etc/fstab # 注释掉swap分区那一行
[root@master ~]#swapoff -a
-node1节点
[root@node1 ~]#vim /etc/fstab # 注释掉swap分区那一行
[root@node1 ~]#swapoff -a
-node1节点
[root@node2 ~]#vim /etc/fstab # 注释掉swap分区那一行
[root@node2 ~]#swapoff -a

6. 开启IP转发，和修改内核信息

-master节点
[root@master ~]#vim /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
[root@master ~]#modprobe br_netfilter
[root@master ~]#sysctl -p /etc/sysctl.d/k8s.conf
net.ipv4.ip_forward = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1

7. 配置IPVS功能

-master节点
[root@master ~]#vim /etc/sysconfig/modules/ipvs.modules
!/bin/bash
modprobe - ip_vs
modprobe - ip_vs_rr
modprobe - ip_vs_wrr
modprobe - ip_vs_sh
[root@master ~]#chmod +x /etc/sysconfig/modules/ipvs.modules
[root@master ~]#bash /etc/sysconfig/modules/ipvs.modules
[root@master ~]#lsmod | grep -e ip_vs
ip_vs_sh 16384 0
ip_vs_wrr 16384 0
ip_vs_rr 16384 0
ip_vs 172032 6 ip_vs_rr,ip_vs_sh,ip_vs_wrr
nf_conntrack 172032 1 ip_vs
nf_defrag_ipv6 20480 2 nf_conntrack,ip_vs
libcrc32c 16384 3 nf_conntrack,xfs,ip_vs
[root@master ~]#reboot

8. ssh免密认证

[root@master ~]#ssh-keygen
Generating public/private rsa key pair.
Enter file in which to save the key (/root/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /root/.ssh/id_rsa.
Your public key has been saved in /root/.ssh/id_rsa.pub.
The key fingerprint is:
SHA256:VcZ6m+gceBJxwysFWwM08526KiBoSt9qdbDQoMSx3kU root@master
The key's randomart image is:
+ -[RSA 3072]----+
| . E .*+o.o |
| o . .* = . |
| . o. .+o+o |
|.....o o.o . |
| o . o S+.o o |
|.o. .o .o +.o |
|+ .o . = . |
|. o . .o |
| . . |
+----[SHA256]-----+
[root@master ~]#ssh-copy-id -i ~/.ssh/id_rsa.pub root@node1
/usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed:
"/root/.ssh/id_rsa.pub"
The authenticity of host 'node1 (192.168.47.137)' can't be established.

三、安装docker

1、更换阿里云镜像仓库

-master节点
[root@master /etc/yum.repos.d]#curl -o /etc/yum.repos.d/CentOS-Base.repo
https: /mirrors.aliyun.com/repo/Centos-vault-8.5.2111.repo
[root@master /etc/yum.repos.d]# dnf -y install epel-release
[root@master /etc/yum.repos.d]#wget https: /mirrors.aliyun.com/dockerce/linux/centos/docker-ce.repo

2、安装docker-ce

-master节点
[root@master ~]# dnf -y install docker-ce -allowerasing
[root@master ~]# systemctl restart docker
[root@master ~]# systemctl enable docker

3. 添加一个配置文件，配置docker仓库加速器

-master节点
[root@master ~]#cat > /etc/docker/daemon.json < EOF
{
"registry-mirrors": ["https: /6vrrj6n2.mirror.aliyuncs.com"],  //这里配置自己的阿里云加速器
"exec-opts": ["native.cgroupdriver=systemd"],
"log-driver": "json-file",
"log-opts": {
"max-size": "100m"
},
"storage-driver": "overlay2"
}
EOF
[root@master ~]#systemctl daemon-reload
[root@master ~]#systemctl restart docker

四、安装kubernetes组件

1、切换成国内的镜像源

由于kubernetes的镜像在国外，速度比较慢，

-master节点
[root@master ~]#cat > /etc/yum.repos.d/kubernetes.repo < EOF
[kubernetes]
name=Kubernetes
baseurl=https: /mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-
x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https: /mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg
https: /mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

2. 安kubeadmkubeletkubectl工具

-master节点
[root@master ~]#dnf -y install kubeadm kubelet kubectl
[root@master ~]#systemctl restart kubelet
[root@master ~]#systemctl enable kubelet

3. 配置containerd

# 为确保后面集群初始化及加入集群能够成功执行，需要配置containerd的配置文
件/etc/containerd/config.toml，此操作需要在所有节点执行
-master节点
[root@master ~]#containerd config default > /etc/containerd/config.toml
# 将/etc/containerd/config.toml文件中的k8s镜像仓库改为
registry.aliyuncs.com/google_containers
[root@master ~]#vim /etc/containerd/config.toml
sandbox_image = "registry.aliyuncs.com/google_containers/pause:3.6"
# 然后重启并设置containerd服务
[root@master ~]#systemctl restart containerd
[root@master ~]#systemctl enable containerd

4. 部署k8s的master节点

-master节点
[root@master ~]#kubeadm init \
-apiserver-advertise-address=192.168.47.137 \
-image-repository registry.aliyuncs.com/google_containers \
-kubernetes-version v1.25.4 \
-service-cidr=10.96.0.0/12 \
-pod-network-cidr=10.244.0.0/16


# 建议将初始化内容保存在某个文件中
[root@master ~]#vim k8s
To start using your cluster, you need to run the following as a regular
user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Alternatively, if you are the root user, you can run:
export KUBECONFIG=/etc/kubernetes/admin.conf
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed
at:
https: /kubernetes.io/docs/concepts/cluster-administration/addons/
Then you can join any number of worker nodes by running the following on
each as root:
kubeadm join 192.168.47.137:6443 -token eav8jn.zj2muv0thd7e8dad \
-discovery-token-ca-cert-hash
sha256:b38f8a6a6302e25c0bcba2a67c13b234fd0b9fdd8b0c0645154c79edf6555e09
[root@master ~]#vim /etc/profile.d/k8s.sh
export KUBECONFIG=/etc/kubernetes/admin.conf
[root@master ~]#source /etc/profile.d/k8s.sh

如果yml文件下载不了，则直接复制里面的配置文件在生成 ——（链接）
[root@master ~]# cat kube-flannel.yml | wc -l
204

5. 安装pod网络插件

-master节点
[root@master ~]#wget https: /raw.githubusercontent.com/flannelio/flannel/master/Documentation/kube-flannel.yml
[root@master ~]#kubectl apply -f kube-flannel.yml
namespace/kube-flannel created
clusterrole.rbac.authorization.k8s.io/flannel created
clusterrolebinding.rbac.authorization.k8s.io/flannel created
serviceaccount/flannel created
configmap/kube-flannel-cfg created
daemonset.apps/kube-flannel-ds created
[root@master ~]#kubectl get nodes
NAME STATUS ROLES AGE VERSION
master NotReady control-plane 6m41s v1.25.4

6. 将node节点加入到k8s集群中

-node1节点
[root@node1 ~]#kubeadm join 192.168.47.137:6443 -token
eav8jn.zj2muv0thd7e8dad \
> -discovery-token-ca-cert-hash
sha256:b38f8a6a6302e25c0bcba2a67c13b234fd0b9fdd8b0c0645154c79edf6555e09

-node2节点
[root@node2 ~]#kubeadm join 192.168.47.137:6443 -token
eav8jn.zj2muv0thd7e8dad \
> -discovery-token-ca-cert-hash
sha256:b38f8a6a6302e25c0bcba2a67c13b234fd0b9fdd8b0c0645154c79edf6555e09


-master节点
[root@master ~]#kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready control-plane 9m37s v1.25.4
node1 NotReady <none> 51s v1.25.4
node2 NotReady <none> 31s v1.25.4

7. 使用k8s集群创建一个pod，运行nginx容器，然后进行测试

[root@master ~]#kubectl create deployment nginx -image nginx
deployment.apps/nginx created
[root@master ~]#kubectl get pods
NAME READY STATUS RESTARTS AGE
nginx-76d6c9b8c-z7p4l 1/1 Running 0 35s
[root@master ~]#kubectl expose deployment nginx -port 80 -type
NodePort
service/nginx exposed
[root@master ~]#kubectl get pods -o wide
NAME READY STATUS RESTARTS AGE IP
NODE NOMINATED NODE READINESS GATES
nginx-76d6c9b8c-z7p4l 1/1 Running 0 119s 10.244.1.2
node1 <none> <none>
[root@master ~]#kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes 	ClusterIP 	10.96.0.1 		<none> 	443/TCP 15m
nginx 		NodePort 	10.109.37.202 	<none> 	80:31125/TCP 17s

9、测试访问

[root@master ~]# kubectl exec -it pod/nginx-76d6c9b8c-z7p4l - /bin/bash
root@nginx-76d6c9b8c-z7p4l:/# cd /usr/share/nginx/html/
echo "liutianyang" > index.html