【aliyun ECS】CentOS 7/8使用kubeadm部署kubernets(k8s)集群【附阿里云实验室】内有福利

好消息好消息，阿里云全线降价，大量服务提供免费甚至永久的试用，赶紧来薅大厂羊毛吧，坐电梯即可直达！
送福利！阿里云热门产品免费领（含ECS），点击进入：https://click.aliyun.com/m/1000370359/
送福利！阿里云热门产品免费领（含ECS），点击进入：https://click.aliyun.com/m/1000370359/
送福利！阿里云热门产品免费领（含ECS），点击进入：https://click.aliyun.com/m/1000370359/

配套实验环境

点击直达阿里云实验室
点击直达阿里云实验室
点击直达阿里云实验室

环境概览

软件	版本	数量
CentOS	7.9/8.x	3
Docker	23.0.3
kubeadm	v1.27.1
kubelet	v1.27.1
kubectl	v1.27.1

基础环境配置

关闭selinux和swap(云服务器可跳过此步骤)(每个节点操作)

setenforce 0
sed -i 's/^SELINUX=.*/SELINUX=permissive/g' /etc/selinux/config
sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab
swapoff -a

修改主机名(每个节点修改为正确的名字)

hostnamectl set-hostname k8s-master01
hostnamectl set-hostname k8s-node01
hostnamectl set-hostname k8s-node02

设置主机映射(每个节点操作)

cat >> /etc/hosts <<EOF
k8s-master01_IP k8s-master01
k8s-node01_IP k8s-node01
k8s-node02_IP k8s-node02

配置免密登录(每个节点都需要操作)—可选操作

# 一直回车就行
ssh-keygen

# 需要输入目标主机的密码
ssh-copy-id root@k8s-master
ssh-copy-id root@k8s-node01
ssh-copy-id root@k8s-node02

修改时区(时区正确可以跳过)(每个节点操作)

timedatectl set-timezone Asia/Shanghai
# 时间同步
yum -y install ntpdate
ntpdate ntp.aliyun.com
systemctl restart rsyslog crond

配置内核参数(每个节点操作)

modprobe overlay
modprobe ip_conntrack
# 加载网桥过滤模块
modprobe br_netfilter

cat > /etc/sysctl.d/kubernetes.conf<<EOF
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
net.ipv4.tcp_tw_recycle=0
vm.swappiness=0
vm.overcommit_memory=1
vm.panic_on_oom=0
fs.inotify.max_user_instances=8192
fs.inotify.max_user_watches=1048576
fs.file-max=52706963
fs.nr_open=52706963
net.ipv6.conf.all.disable_ipv6=1
net.netfilter.nf_conntrack_max=2310720
EOF

# 加载内核文件
sysctl -p /etc/sysctl.d/kubernetes.conf

# 配置ipvs功能
cat >/etc/sysconfig/modules/ipvs.modules <<EOF
#!/bin/bash
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
modprobe -- nf_conntrack_ipv4
EOF
chmod 755 /etc/sysconfig/modules/ipvs.modules
bash /etc/sysconfig/modules/ipvs.modules && lsmod |grep -e ip_vs -e nf_conntrack_ipv4
cat << EOF >> /etc/sysconfig/kubelet
KUBELET_CGROUP_ARGS="--cgroup-driver=systemd"
KUBE_PROXY_MODE="ipvs"
EOF

Docker、containerd安装及配置

清除docker软件包(每个节点操作)

yum remove docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-engine

设置docker源并安装docker-ce、containerd(每个节点操作)

wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
yum install -y yum-utils device-mapper-persistent-data lvm2
yum-config-manager --add-repo  https://download.docker.com/linux/centos/docker-ce.repo
yum install docker-ce-23.0.3 docker-ce-cli-23.0.3 containerd.io -y
systemctl enable docker --now

k8s配置及安装

配置k8s国内源(每个节点操作)

cat > /etc/yum.repos.d/kubernetes.repo << EOF
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64
enabled=1
gpgcheck=0
repo_gpgcheck=0
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
EOF

安装k8s相关软件包(每个节点操作)

yum -y install vim kubelet kubeadm kubectl --disableexcludes=kubernetes

修改containerd配置

pause_version=`kubeadm config images list|grep pause|awk -F '/' '{print $NF}'`
containerd config default > /etc/containerd/config.toml
sed -i 's/registry.k8s.io\/pause:3.6/registry.aliyuncs.com\/google_containers\/'"${pause_version}"'/' /etc/containerd/config.toml
sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
systemctl restart containerd

启动Kubelet

echo "export KUBECONFIG=/etc/kubernetes/admin.conf" >> /etc/profile
source /etc/profile
systemctl enable kubelet --now

master节点拉取k8s镜像(master节点操作)

cat > images.sh <<EOF 
#!/bin/bash
url=registry.aliyuncs.com/google_containers
version=(\`kubelet --version|awk -F " " '{print $2}'\`)
images=(\`kubeadm config images list --kubernetes-version=$version|awk -F '/' '{print \$NF}'\`)
org_hub=(\`kubeadm config images list --kubernetes-version=$version|awk -F '/' 'NR==1{print \$1}'\`)
for img in \${images[@]};do
    docker pull \${url}/\$img
    docker tag \$url/\$img \${org_hub}/\$img
    docker rmi -f \$url/\$img
done
EOF
chmod +x images.sh
./images.sh

生成初始化文件(master节点操作)

kubeadm config print init-defaults > kubeadm-config.yaml

修改kubeadm-config.yaml文件(master节点操作)

version=`kubelet --version|awk -F " " '{print $2}'`
adm_name=`hostname`
adm_ip=`hostname -I|awk -F " " '{print $1}'`
sed -i 's/name: node/name: '"${adm_name}"'/' kubeadm-config.yaml 
sed -i 's/1.2.3.4/'"${adm_ip}"'/' kubeadm-config.yaml
sed -i '/serviceSubnet/a\  podSubnet: 10.244.0.0/16' kubeadm-config.yaml
sed -i 's/registry.k8s.io/registry.aliyuncs.com\/google_containers/' kubeadm-config.yaml
sed -i 's/1.27.0/'"${version}"'/' kubeadm-config.yaml 

# 初始化集群
kubeadm init --config=kubeadm-config.yaml

设置k8s环境(master节点操作)

mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

# 将admin.conf文件拷贝给其他节点
scp /etc/kubernetes/admin.conf root@k8s-node01:/etc/kubernetes/
scp /etc/kubernetes/admin.conf root@k8s-node02:/etc/kubernetes/

node1/2配置环境

mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

部署网络 - flannel(master节点操作)

kubectl create -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
# 查看pod状态
kubectl get pod -n kube-system

node节点加入集群

用mater节点初始化输出的kubeadm join命令来操作，以下为示例

kubeadm join 172.16.22.24:6443 --token 8unt75.7rey15tpqsu1nxvs \ --discovery-token-ca-cert-hash sha256:3c7c07f74c965f26e0b51392103509ce4c5cdbc2ba0be49c9d98cf3679d21cdc

部署nginx测试

# 部署
kubectl create deployment nginx --image=nginx:1.20-alpine
kubectl expose deployment nginx --port=80 --type=NodePort

# 查看状态，查看对应开放的外部端口
kubectl get pods,service

# 可以用浏览器访问服务器的外部IP和端口

# 删除部署和服务
kubectl delete deployment nginx
kubectl delete service nginx