上来装centos7集群敲着命令呢，突然网断了，我以为我哪里命令搞错了，弄崩了系统，重装，之前因为已经zhua装好一个centos7并且多装了个vmtools,后面查明
cd /etc/sysconfig/network-scripts/ vim ifcfg-ens33 （ifconfig 获取）（最基本安装连这个命令都没有，我后面都带界面安装了）
ONBOOT=no
改成
ONBOOT=yes
service network restart
先讲用kubeadm安装kubernetes集群
systemctl disable firewalld
systemctl stop firewalld
setenforce 0或者改/etc/sysconfig/selinux 将SELINUX=enforcing 改成SELINUX=disabled
以上操作我暂时没动
首先是安装docker
yum install -y docker这个在国内貌似没问题
[lixl@localhost ~]$ docker --version
Docker version 1.13.1, build 07f3374/1.13.1
装kubeadm各路都有，主要是源的问题，我找的的方法是
https://blog.csdn.net/zhuchuangang/article/details/76572157#2
https://blog.csdn.net/zhuchuangang/article/details/76572157
#kubernetes yum源
vim /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
enabled=1
gpgcheck=0
然后就是
yum install -y kubelet
yum install -y kubeadm
安装好这两个，yum install -y kubectl kubernetes-cni时说都已经安装完成，估计其实是kubeadm都集成了，第一个命令都不一定需要。
systemctl start docker
systemctl enable docker
systemctl enable kubelet
systemctl start kubelet
[root@localhost ~]# kubeadm version
kubeadm version: &version.Info{Major:"1", Minor:"13", GitVersion:"v1.13.1", GitCommit:"eec55b9ba98609a46fee712359c7b5b365bdd920", GitTreeState:"clean", BuildDate:"2018-12-13T10:36:44Z", GoVersion:"go1.11.2", Compiler:"gc", Platform:"linux/amd64"}
接下来下载kubernetes的docker镜像
由于kubeadmjian将自动下载kubernetes的相关镜像，默认gcr.io下载（没看出来）
修改docker配置/vim /etc/sysconfig/docker，增加registry mirror参数=false那个是我加的
OPTIONS='--selinux-enabled=false --log-driver=journald --signature-verification=false'
OPTIONS='--registry-mirror=http://68e02ab9.m.daocloud.io'
kubeadm init --kubernetes-version=1.6.0
this version of kubeadm only supports deploying clusters with the control plane version >= 1.12.0. Current version: v1.6.0
[root@localhost ~]# kubeadm init --kubernetes-version=1.12.0
[init] Using Kubernetes version: v1.12.0
[preflight] Running pre-flight checks
[WARNING Firewalld]: firewalld is active, please ensure ports [6443 10250] are open or your cluster may not function correctly
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR FileContent--proc-sys-net-bridge-bridge-nf-call-iptables]: /proc/sys/net/bridge/bridge-nf-call-iptables contents are not set to 1
[ERROR Swap]: running with swap on is not supported. Please disable swap
[ERROR KubeletVersion]: the kubelet version is higher than the control plane version. This is not a supported version skew and may lead to a malfunctional cluster. Kubelet version: "1.13.1" Control plane version: "1.12.0"
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
systemctl disable firewalld
systemctl stop firewalld
vim /etc/sysctl.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
net.bridge.bridge-nf-call-arptables = 1重启
或者
创建/etc/sysctl.d/k8s.conf文件
sudo vi /etc/sysctl.d/k8s.conf
添加如下内容：
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
sudo sysctl -p /etc/sysctl.d/k8s.conf
https://blog.csdn.net/zzq900503/article/details/81710319
echo 1 > /proc/sys/net/bridge/bridge-nf-call-ip6tables
echo 1 > /proc/sys/net/bridge/bridge-nf-call-iptables
ansible all -m shell -a "iptables -P FORWARD ACCEPT"
永久修改：/usr/lib/sysctl.d/00-system.conf
SELINUX=enforcing
swapoff -a
https://blog.csdn.net/CSDN_duomaomao/article/details/75142769?locationNum=8&fps=1
[root@localhost sysctl.d]# kubeadm init --kubernetes-version=1.13.0
[init] Using Kubernetes version: v1.13.0
[preflight] Running pre-flight checks
[preflight] Pulling images required for setting up a Kubernetes cluster
[preflight] This might take a minute or two, depending on the speed of your internet connection
[preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
error execution phase preflight: [preflight] Some fatal errors occurred:
[ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-apiserver:v1.13.0: output: Trying to pull repository k8s.gcr.io/kube-apiserver ...
Get https://k8s.gcr.io/v1/_ping: dial tcp 108.177.97.82:443: getsockopt: connection refused
, error: exit status 1
[ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-controller-manager:v1.13.0: output: Trying to pull repository k8s.gcr.io/kube-controller-manager ...
Get https://k8s.gcr.io/v1/_ping: dial tcp 64.233.189.82:443: getsockopt: connection refused
, error: exit status 1
[ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-scheduler:v1.13.0: output: Trying to pull repository k8s.gcr.io/kube-scheduler ...
Get https://k8s.gcr.io/v1/_ping: dial tcp 64.233.187.82:443: getsockopt: connection refused
, error: exit status 1
[ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-proxy:v1.13.0: output: Trying to pull repository k8s.gcr.io/kube-proxy ...
Get https://k8s.gcr.io/v1/_ping: dial tcp 64.233.189.82:443: getsockopt: connection refused
, error: exit status 1
[ERROR ImagePull]: failed to pull image k8s.gcr.io/pause:3.1: output: Trying to pull repository k8s.gcr.io/pause ...
Get https://k8s.gcr.io/v1/_ping: dial tcp 108.177.125.82:443: getsockopt: connection refused
, error: exit status 1
[ERROR ImagePull]: failed to pull image k8s.gcr.io/etcd:3.2.24: output: Trying to pull repository k8s.gcr.io/etcd ...
Get https://k8s.gcr.io/v1/_ping: dial tcp 74.125.204.82:443: getsockopt: connection refused
, error: exit status 1
[ERROR ImagePull]: failed to pull image k8s.gcr.io/coredns:1.2.6: output: Trying to pull repository k8s.gcr.io/coredns ...
Get https://k8s.gcr.io/v1/_ping: dial tcp 74.125.204.82:443: getsockopt: connection refused
, error: exit status 1
[preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
https://blog.csdn.net/jinguangliu/article/details/82792617
kubeadm config images list 根据要求写名字，不能写错，或者根据错误写docker images
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
kubeadm reset
Unfortunately, an error has occurred:
timed out waiting for the condition
This error is likely caused by:
- The kubelet is not running
- The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
- 'systemctl status kubelet'
- 'journalctl -xeu kubelet'
Additionally, a control plane component may have crashed or exited when started by the container runtime.
To troubleshoot, list all containers using your preferred container runtimes CLI, e.g. docker.
Here is one example how you may list all Kubernetes containers running in docker:
- 'docker ps -a | grep kube | grep -v pause'
Once you have found the failing container, you can inspect its logs with:
- 'docker logs CONTAINERID'
error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
运行journalctl -xeu kubelet
Container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized
k8s.io/kubernetes/pkg/kubelet/kubelet.go:453: Failed to list *v1.Node: Get https://192.168.41.137:6443/api/v1/nodes?fieldSelector=metadata.name%3Dmaster&limit=500&resourceVersion=0: dial tcp 192.168.41.137:6443: connect: connection refused
eviction manager: failed to get get summary stats: failed to get node
https://blog.csdn.net/bbwangj/article/details/82024485
按照上面的方法始终不奏效，错误又是不停的循环滚动，到底也不知道哪里错了
后面http://blog.51cto.com/ghbsunny/2162205?source=dra
搜到最后这个错误，原来是内存小了，改成2G可以了
至于network那个后面kubeadm启动成功后会有系统解决方式告知
这里面的问题解决https://blog.csdn.net/u012286287/article/details/79716588
解决方案https://blog.csdn.net/zzq900503/article/details/81710319
kubelet 的cgroup dirver 与 docker的不一样。docker默认使用cgroupfs,keubelet 默认使用systemd没遇到，我是一致的
Your Kubernetes master has initialized successfully!
To start using your cluster, you need to run the following as a regular user:
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
https://kubernetes.io/docs/concepts/cluster-administration/addons/
You can now join any number of machines by running the following on each node
as root:
kubeadm join 192.168.41.137:6443 --token 2dva0q.rr3htfw7s4ofmjh4 --discovery-token-ca-cert-hash sha256:9384042d6812ff7631f0c3c5c5b827ccbefdbbac6d9213611c077799e89bfc6e
最后选了这个apply， 版本有点多
mkdir -p ~/k8s/
cd ~/k8s
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl apply -f kube-flannel.yml
[root@master k8s]# kubectl get pod --all-namespaces -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-system coredns-86c58d9df4-6m2vv 0/1 ContainerCreating 0 31m <none> master <none> <none>
kube-system coredns-86c58d9df4-xttbs 0/1 ContainerCreating 0 31m <none> master <none> <none>
kube-system etcd-master 1/1 Running 0 30m 192.168.41.137 master <none> <none>
kube-system kube-apiserver-master 1/1 Running 0 30m 192.168.41.137 master <none> <none>
kube-system kube-controller-manager-master 1/1 Running 0 30m 192.168.41.137 master <none> <none>
kube-system kube-flannel-ds-amd64-ftjsk 0/1 CrashLoopBackOff 3 6m4s 192.168.41.137 master <none> <none>
kube-system kube-proxy-bnzzx 1/1 Running 0 31m 192.168.41.137 master <none> <none>
kube-system kube-scheduler-master 1/1 Running 0 30m 192.168.41.137 master <none> <none>
journalctl -u kubelet -f
kubectl describe pod coredns-86c58d9df4-xttbs -n kube-system
kubectl describe pods --all-namespaces
kubectl logs coredns-86c58d9df4-6m2vv -n kube-system
kubectl get pods --all-namespaces
kubectl get pod --all-namespaces -o wide
kubectl get pods -n kube-system -o wide
kubeadm reset
The reset process does not reset or clean up iptables rules or IPVS tables.
If you wish to reset iptables, you must do so manually.
For example:
iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X
If your cluster was setup to utilize IPVS, run ipvsadm --clear (or similar)
to reset your system's IPVS tables
kubeadm init --kubernetes-version=v1.13.0 --pod-network-cidr=10.244.0.0/16
kubeadm join 192.168.41.137:6443 --token ycd1dl.xza4hi7b4prr0387 --discovery-token-ca-cert-hash sha256:15299a96ced577a2a865216b9240511d47e940e961ac461970a55f12e2b564be
docker pull quay.io/coreos/flannel:v0.10.0-amd64
mkdir -p /etc/cni/net.d/
cat <<EOF> /etc/cni/net.d/10-flannel.conf
{"name":"cbr0","type":"flannel","delegate": {"isDefaultGateway": true}}
EOF
mkdir /usr/share/oci-umount/oci-umount.d -p
mkdir /run/flannel/
cat <<EOF> /run/flannel/subnet.env
FLANNEL_NETWORK=172.100.0.0/16
FLANNEL_SUBNET=172.100.1.0/24
FLANNEL_MTU=1450
FLANNEL_IPMASQ=true
EOF
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/v0.9.1/Documentation/kube-flannel.yml
集群初始化如果遇到问题，可以使用下面的命令进行清理再重新初始化：
kubeadm reset
[root@master k8s]# iptables -F && iptables -t nat -F && iptables -t mangle -F && iptables -X
ifconfig cni0 down
ip link delete cni0
ifconfig flannel.1 down
ip link delete flannel.1
rm -rf /var/lib/cni/
[runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:docker: network plugin is not ready: cni config uninitialized]
mkdir -p ~/k8s/
cd ~/k8s
wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
kubectl apply -f kube-flannel.yml
问题解决参考
vim /etc/sysconfig/kubelet
KUBELET_EXTRA_ARGS='--runtime-cgroups=/systemd/system.slice --kubelet-cgroups=/systemd/system.slice'
https://blog.csdn.net/qq_34857250/article/details/82562514
open /run/flannel/subnet.env
http://dockone.io/question/1225
参考