Ingress简介
在Kubernetes中，服务和Pod的IP地址仅可以在集群网络内部使用，对于集群外的应用是不可见的。为了使外部的应用能够访问集群内的服务，在Kubernetes 目前提供了以下几种方案：
NodePort
LoadBalancer（负载均衡）
Ingress（入口）
Ingress 组成
ingress controller
将新加入的Ingress转化成Nginx的配置文件并使之生效
ingress服务
将Nginx的配置抽象成一个Ingress对象，每添加一个新的服务只需写一个新的Ingress的yaml文件即可
ingress工作原理
- ingress controller通过和kubernetes api交互，动态的去感知集群中ingress规则变化，
- 然后读取它，按照自定义的规则，规则就是写明了哪个域名对应哪个service，生成一段nginx配置
- 再写到nginx-ingress-control的pod里，这个Ingress controller的pod里运行着一个Nginx服务，控制器会把生成的nginx配置写入/etc/nginx.conf文件中
- 然后reload一下使配置生效。以此达到域名分配置和动态更新的问题
Ingress 可以解决什么问题
1.动态配置服务
如果按照传统方式, 当新增加一个服务时, 我们可能需要在流量入口加一个反向代理指向我们新的k8s服务. 而如果用了Ingress, 只需要配置好这个服务, 当服务启动时, 会自动注册到Ingress的中, 不需要而外的操作.
2.减少不必要的端口暴露
配置过k8s的都清楚, 第一步是要关闭防火墙的, 主要原因是k8s的很多服务会以NodePort方式映射出去, 这样就相当于给宿主机打了很多孔, 既不安全也不优雅. 而Ingress可以避免这个问题, 除了Ingress自身服务可能需要映射出去, 其他服务都不要用NodePort方式
一、实验需求：
| 资源 | 条件 |
|---|---|
| deployment1 | nginx镜像，replicas：3 |
| deploument2 | httpd镜像，replicas：4 |
| service1 | 绑定deployment1 |
| service2 | 绑定deployment2 |
//这里我们写两个yaml文件deployment和svc写成一个
[root@master yaml]# vim deployment1.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: deploy1
spec:
replicas: 3
template:
metadata:
labels:
app: test1
spec:
containers:
- name: test1
image: nginx
ports:
- containerPort: 80
---
kind: Service
apiVersion: v1
metadata:
name: svc1
spec:
selector:
app: test1
ports:
- protocol: TCP
port: 80
targetPort: 80
[root@master yaml]# vim deployment2.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: deploy2
spec:
replicas: 4
template:
metadata:
labels:
app: test2
spec:
containers:
- name: test2
image: httpd
ports:
- containerPort: 80
---
kind: Service
apiVersion: v1
metadata:
name: svc2
spec:
selector:
app: test2
ports:
- protocol: TCP
port: 80
targetPort: 80
二、部署Ingress
//部署Ingress-controller.可以在GitHub上找到，这里我们部署的是Ingress：0.35.0版本
//这里我们可以先保存Ingress的yaml文件，可以查看都做了什么，也方便我们后期管理
wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v0.35.0/deploy/static/provider/baremetal/deploy.yaml
//这里使用的镜像，是国外的镜像，需要科学上网获得
这里我使用提前准备好的ingress.yaml
[root@master yaml]# ls
deployment1.yaml deployment2.yaml deploy.yaml
[root@master yaml]# vim deploy.yaml
...
329 spec:
330 hostNetwork: true //这里添加字段
331 dnsPolicy: ClusterFirst
332 containers:
333 - name: controller334 image: quay.io/kubernetes-ingress-controller/nginx-ingress-c ontroller:0.30.0
335 imagePullPolicy: IfNotPresent
336 lifecycle:
337 preStop:
338 exec:
339 command:
340 - /wait-shutdown
...
运行
扫描二维码关注公众号，回复：
12461995 查看本文章
[root@master yaml]# kubectl apply -f deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
configmap/ingress-nginx-controller created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
service/ingress-nginx-controller-admission created
service/ingress-nginx-controller created
deployment.apps/ingress-nginx-controller created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
serviceaccount/ingress-nginx-admission created
[root@master yaml]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-vtwk8 0/1 Completed 0 5m36s
ingress-nginx-admission-patch-np2jk 0/1 Completed 0 5m36s
ingress-nginx-controller-674c958759-nrp98 1/1 Running 0 5m46s
[root@master yaml]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.100.63.52 <none> 80:31526/TCP,443:30262/TCP 7m4s
ingress-nginx-controller-admission ClusterIP 10.96.148.104 <none> 443/TCP 7m4s
查看Ingress-nginx-controller容器内部详情：可以看到，它现在已经有一个模板，用来描述Ingress资源能够收集到的信息了
[root@master yaml]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-vtwk8 0/1 Completed 0 10m
ingress-nginx-admission-patch-np2jk 0/1 Completed 0 10m
ingress-nginx-controller-674c958759-nrp98 1/1 Running 0 10m
[root@master yaml]# kubectl exec -it -n ingress-nginx ingress-nginx-controller-674c958759-nrp98 bash
bash-5.0$ ls
fastcgi.conf mime.types scgi_params
fastcgi.conf.default mime.types.default scgi_params.default
fastcgi_params modsecurity template
fastcgi_params.default modules uwsgi_params
geoip nginx.conf uwsgi_params.default
koi-utf nginx.conf.default win-utf
koi-win opentracing.json
lua owasp-modsecurity-crs
bash-5.0$ cat nginx.conf
...
location / {
set $namespace "";
set $ingress_name "";
set $service_name "";
set $service_port "";
set $location_path "/";
...
创建ingress规则
[root@master yaml]# vim ingress.yaml
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: bdqn-ingress
annotations:
nginx.ingress.kubernetes.io/rewrite-target: /
spec:
rules:
- host: ingress.bdqn.com
http:
paths:
- path: /nginx
backend:
serviceName: svc1
servicePort: 80
- path: /httpd
backend:
serviceName: svc2
servicePort: 80
[root@master yaml]# kubectl apply -f ingress.yaml
ingress.extensions/bdqn-ingress created
查看对应规则的详细信息
[root@master yaml]# kubectl describe ingresses. bdqn-ingress
Name: bdqn-ingress
Namespace: default
Address: 192.168.1.22
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
ingress.bdqn.com
/nginx svc1:80 (10.244.1.7:80,10.244.1.8:80,10.244.2.9:80)
/httpd svc2:80 (10.244.1.10:80,10.244.1.9:80,10.244.2.10:80 + 1 more...)
查看状态和容器里的模板
[root@master yaml]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-vtwk8 0/1 Completed 0 19m
ingress-nginx-admission-patch-np2jk 0/1 Completed 0 19m
ingress-nginx-controller-674c958759-nrp98 1/1 Running 0 19m
[root@master yaml]# kubectl exec -it -n ingress-nginx ingress-nginx-controller-674c958759-nrp98 bash
bash-5.0$ cat nginx.conf
location ~* "^/nginx" {
set $namespace "default";
set $ingress_name "bdqn-ingress";
set $service_name "svc1";
set $service_port "80";
set $location_path "/nginx";
location ~* "^/httpd" {
set $namespace "default";
set $ingress_name "bdqn-ingress";
set $service_name "svc2";
set $service_port "80";
set $location_path "/httpd";
修改本机域名解析，浏览器访问
//来模拟访问，这里我们用Windows的浏览器直接访问，因为我们是模拟的域名所以，需要我们在hosts文件内写入对应的域名解析。Windows10的域名解析文件地址：C:\Windows\System32\drivers\etc，更改这个文件的时候，需要提供管理员权限。
C:\Windows\System32\drivers\etc下的hosts文件右键–属性–安全–编辑
浏览器访问
[root@master yaml]# kubectl get svc -n ingress-nginx
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
ingress-nginx-controller NodePort 10.100.63.52 <none> 80:31526/TCP,443:30262/TCP 34m
ingress-nginx-controller-admission ClusterIP 10.96.148.104 <none> 443/TCP 34m
http://ingress.bdqn.com/nginx:31526
https://ingress.bdqn.com/httpd:31526
总结： Mandatory-svc,这个SVC,就等于是将nginx-ingress-controller做了一个NodePort类型的SVC,提供了一个统一的访问入口，这样可以避免暴露端口过多的情况
基于http实现虚拟机主机的访问
此实验的前提是： Ingress-nginx-controller服务在集群中，已经完成部署。
[root@master yaml]# kubectl get pod -n ingress-nginx
NAME READY STATUS RESTARTS AGE
ingress-nginx-admission-create-mf6sj 0/1 Completed 0 6m26s
ingress-nginx-admission-patch-h6fd9 0/1 Completed 2 6m26s
ingress-nginx-controller-674c958759-2dtfc 1/1 Running 0 6m37s
创建私有镜像和私有仓库
1.用docker容器运行registry私有仓库服务。
//下载需要的镜像
[root@master ~]# docker pull registry:2
2.运行私有仓库服务
[root@master ~]# docker run -itd --name registry --restart=always -p 5000:5000 registry:2
3.镜像重命名
[root@master ~]# docker pull httpd
[root@master ~]# docker tag httpd:latest 192.168.1.20:5000/httpd:v1
[root@master ~]# docker tag httpd:latest 192.168.1.20:5000/httpd:v2
4.编辑docker配置文件
三台节点都要修改
[root@master ~]# vim /usr/lib/systemd/system/docker.service
# for containers run by docker
ExecStart=/usr/bin/dockerd --insecure-registry 192.168.1.20:5000 //添加字段
ExecReload=/bin/kill -s HUP $MAINPID
TimeoutSec=0
RestartSec=2
Restart=always
[root@master ~]# systemctl daemon-reload
[root@master ~]# systemctl restart docker
5.上传私有仓库
[root@master ~]# docker push 192.168.1.20:5000/httpd:v1
[root@master ~]# docker push 192.168.1.20:5000/httpd:v2
//编辑ingress1.bdqn.io域名所需要的Deployment和SVC资源。
[root@master yaml]# vim dep1.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: deploy1
spec:
replicas: 2
template:
metadata:
labels:
version: v1
spec:
containers:
- name: httpd1
image: 192.168.1.20:5000/httpd:v1
---
kind: Service
apiVersion: v1
metadata:
name: svc1
spec:
selector:
version: v1
ports:
- port: 80
targetPort: 80
[root@master yaml]# kubectl apply -f dep1.yaml
deployment.extensions/deploy1 created
service/svc1 created
[root@master yaml]# kubectl get pod
NAME READY STATUS RESTARTS AGE
deploy1-6d69799bcf-cv9sz 1/1 Running 0 6s
deploy1-6d69799bcf-llx6c 1/1 Running 0 6s
//验证上述资源没有问题之后，可以直接复制其yaml文件，更改相应名称即可得到ingress2.bdqn.io这个域名所依赖的资源。
[root@master yaml]# cp dep1.yaml dep2.yaml
[root@master yaml]# vim dep2.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: deploy2
spec:
replicas: 2
template:
metadata:
labels:
version: v2
spec:
containers:
- name: httpd2
image: 192.168.1.20:5000/httpd:v2
---
kind: Service
apiVersion: v1
metadata:
name: svc2
spec:
selector:
version: v2
ports:
- port: 80
targetPort: 80
[root@master yaml]# kubectl apply -f dep2.yaml
deployment.extensions/deploy2 created
service/svc2 created
[root@master yaml]# kubectl get pod
NAME READY STATUS RESTARTS AGE
deploy1-6d69799bcf-cv9sz 1/1 Running 0 4m13s
deploy1-6d69799bcf-llx6c 1/1 Running 0 4m13s
deploy2-657686cf76-gfppd 1/1 Running 0 43s
deploy2-657686cf76-jpw8r 1/1 Running 0 43s
//访问各SVC资源的ClusterIP，验证服务
[root@master yaml]# kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 10.96.0.1 <none> 443/TCP 4d22h
svc1 ClusterIP 10.100.158.82 <none> 80/TCP 22s
svc2 ClusterIP 10.103.194.151 <none> 80/TCP 13s
[root@master yaml]# curl 10.100.158.82
<html><body><h1>It works!</h1></body></html>
[root@master yaml]# curl 10.103.194.151
<html><body><h1>It works!</h1></body></html>
//创建对应的Ingress规则，这个是最重要的一环。
[root@master yaml]# vim ing1.yaml
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: ingress1
spec:
rules:
- host: ingress1.bdqn.io
http:
paths:
- path: /
backend:
serviceName: svc1
servicePort: 80
---
kind: Ingress
apiVersion: extensions/v1beta1
metadata:
name: ingress2
spec:
rules:
- host: ingress2.bdqn.io
http:
paths:
- path: /
backend:
serviceName: svc2
servicePort: 80
[root@master yaml]# kubectl apply -f ing1.yaml
ingress.extensions/ingress1 created
ingress.extensions/ingress2 created
//查看对应Ingress规则的相信信息
[root@master yaml]# kubectl describe ingresses. ingress1
Name: ingress1
Namespace: default
Address: 192.168.1.22
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
ingress1.bdqn.io
/ svc1:80 (10.244.1.16:80,10.244.2.17:80)
[root@master yaml]# kubectl describe ingresses. ingress2
Name: ingress2
Namespace: default
Address: 192.168.1.22
Default backend: default-http-backend:80 (<none>)
Rules:
Host Path Backends
---- ---- --------
ingress2.bdqn.io
/ svc2:80 (10.244.1.17:80,10.244.2.18:80)
//在Windows上，用浏览器访问验证，当然，不要忘了，去做对应的域名解析。
浏览器访问
http://ingress1.bdqn.io/
浏览器访问
http://ingress2.bdqn.io/
基于https的访问
//创建证书
[root@master httpds]# pwd
/root/yaml/httpds
[root@master httpds]# openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=nginxsvc/O=nginxsvc"
Generating a 2048 bit RSA private key
.........+++
.........................................................+++
writing new private key to 'tls.key'
-----
[root@master httpds]# ls
tls.crt tls.key
/创建secret资源，将证书保存到k8s集群中
[root@master httpds]# kubectl create secret tls tls-secret --key=tls.key --cert tls.crt
secret/tls-secret created
创建新的Deploy5.yaml
[root@master httpds]# vim deploy5.yaml
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
name: deploy5
spec:
replicas: 2
template:
metadata:
labels:
app: nginx5
spec:
containers:
- name: nginx5
image: nginx
---
kind: Service
apiVersion: v1
metadata:
name: svc5
spec:
selector:
app: nginx5
ports:
- protocol: TCP
port: 80
targetPort: 80
[root@master httpds]# kubectl apply -f deploy5.yaml
deployment.extensions/deploy5 created
service/svc5 unchanged
//创建对应Ingress规则
[root@master httpds]# vim ing5.yaml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
name: https
spec:
tls:
- hosts:
- ingress5.bdqn.com
secretName: tls-secret
rules:
- host: ingress5.bdqn.com
http:
paths:
- path: /
backend:
serviceName: svc5
servicePort: 80
[root@master httpds]# kubectl apply -f ing5.yaml
ingress.extensions/https created
浏览器访问
选择高级
