kubernetes Ingress 之 Traefik

About traefik

Before writing a document reference:
https://blog.51cto.com/michaelkang/1918192

Version Introduction

traefik：v1.7
k8s：v1.15.1

Ingress

Ingress is a resource type version introduced since the kubernetes1.1. Ingress controller must be deployed in order to create Ingress resources, Ingress controller is a form of plug-ins available.

Generally have three components use Ingress:

反向代理负载均衡器
Ingress Controller
Ingress

Components Introduction

Reverse proxy load balancer

Reverse proxy load balancer is very simple, similar nginx, haproxy; reverse proxy load balancer in a cluster can be freely deployed, you can use Replication Controller, Deployment, DaemonSet and so on, the recommended way to deploy DaemonSet

Ingress Controller

Ingress Controller essentially be understood as a monitor, Ingress Controller by constantly dealing with kubernetes API, real-time perception back-end service, change pod, etc., such as add and reduce pod, service and other increases and decreases; when these changes get information after, Ingress controller Ingress recombination generated the following configuration, and then update the reverse proxy load balancer, and refresh its configuration, to the role of the service discovery

Ingress

Ingress is a simple to understand the rules defined above; for example, a domain name corresponding to a service, i.e., forwarded to a service when a request for a domain in; this rule will bind to the Ingress Controller, and the Controller Ingress dynamic write to it load balancer configuration in order to achieve the overall service discovery and load balancing

Deployment traefik ingress Service

If you are not familiar Kubernetes in Ingresses, you may want to read Kubernetes User's Guide.

Deployment Requirements

A normal working Kubernetes cluster. It can be minikube cluster.

Introduction cluster information

[root@kubm-02 traefik]# kubectl  get nodes -o wide
NAME         STATUS   ROLES    AGE   VERSION   INTERNAL-IP      
kubm-01      Ready    master   13d   v1.15.1   172.20.101.157   
kubm-02      Ready    master   13d   v1.15.1   172.20.101.164   
kubm-03      Ready    master   13d   v1.15.1   172.20.101.165   
kubnode-01   Ready    <none>   13d   v1.15.1   172.20.101.160   
kubnode-02   Ready    <none>   13d   v1.15.1   172.20.101.166   
kubnode-03   Ready    <none>   13d   v1.15.1   172.20.101.167  

Create a user access rules

Kubernetes introduces role-based access control (RBAC) in 1.6+, to allow Kubernetes API resources and fine-grained control.
If your cluster configuration of RBAC, you will need to authorize the use of Traefik Kubernetes API. There are two ways to set the appropriate permissions: ClusterRoleBinding RoleBindings in single or by a specific global namespace.
RoleBinding each namespace can limit the rights granted to only Traefik are monitoring the name space to use, in order to follow the principle of least privilege. If Traefik should not monitor all namespaces, and namespace set does not change the dynamic, then this is the preferred method. Otherwise, you must use a single ClusterRoleBinding.

ClusterRoleBinding：

---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
rules:
  - apiGroups:
      - ""
    resources:
      - services
      - endpoints
      - secrets
    verbs:
      - get
      - list
      - watch
  - apiGroups:
      - extensions
    resources:
      - ingresses
    verbs:
      - get
      - list
      - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: traefik-ingress-controller
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: traefik-ingress-controller
subjects:
- kind: ServiceAccount
  name: traefik-ingress-controller
  namespace: kube-system

[root@kubm-02 ~]# kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/traefik-rbac.yaml

return

clusterrole.rbac.authorization.k8s.io/traefik-ingress-controller created
clusterrolebinding.rbac.authorization.k8s.io/traefik-ingress-controller created

Deployment

difference

A copy of the Pod Deployment deployment will be distributed among the Node, each Node may run several copies.
The difference DaemonSet that can only run one copy on each Node.
The use of DaemonSet deployed

Deployment:
kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/traefik-deployment.yaml

DaemonSet :
kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/traefik-ds.yaml

Authentication Service to start:

View Information Services

[root@kubm-02 traefik]# kubectl get rc,services -n kube-system 

NAME                              TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE
...
service/traefik-ingress-service   ClusterIP   10.245.153.125   <none>        80/TCP,8080/TCP          3m42s

View container startup state

[root@kubm-02 ~]# kubectl get pods --all-namespaces  -o wide --selector=k8s-app=traefik-ingress-lb
NAMESPACE     NAME                               READY   STATUS    RESTARTS   AGE   IP             NODE         NOMINATED NODE   READINESS GATES
kube-system   traefik-ingress-controller-4d29d   1/1     Running   0          19m   10.244.3.107   kubnode-01   <none>           <none>
kube-system   traefik-ingress-controller-mgljm   1/1     Running   0          19m   10.244.5.143   kubnode-03   <none>           <none>
kube-system   traefik-ingress-controller-wcd5z   1/1     Running   0          19m   10.244.4.126   kubnode-02   <none>           <none>

Access traefik management page

The service will allow NodePort two public access entrance and Web interface.

80 业务端口，后端服务启动注册到traefik后，写hosts文件或者添加dns 解析才能访问；
8080 traefik 管理页面，访问node节点的IP地址：8080即可访问；
例如：http://172.20.101.167:8080

Registration Service Test Access:

First, create a public service and a will of Ingress Traefik Web UI

Deployment Services:

apiVersion: v1
kind: Service
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  selector:
    k8s-app: traefik-ingress-lb
  ports:
  - name: web
    port: 80
    targetPort: 8080
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: traefik-web-ui
  namespace: kube-system
spec:
  rules:
  - host: traefik-ui.minikube
    http:
      paths:
      - path: /
        backend:
          serviceName: traefik-web-ui
          servicePort: web

kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/ui.yaml

Task execution is complete, in / etc / hosts file will traefik-ui.minikube points to a node node
IP address, and browser access: HTTP: //traefik-ui.minikube
or
first open traefik management page, accessible through the curl, health see page request, refer to the following command:

[root@kubm-02 ~]# curl http://172.20.101.166 --user-agent "Mozilla/5.0" -H "Host:traefik-ui.minikube"
<a href="/dashboard/">Found</a>.

Use basic verification

Creating secret

A. htpasswd is used to create the file containing the user names and passwords MD5 coding:
yum -y install httpd
htpasswd -c ./auth MyUserName
you will be prompted to enter the password, you must enter the password twice. htpasswd will create a file with the following contents:

[root@kubm-02 traefik]# more auth 
myusername:$apr1$3yj4XbDF$4ekQISLfP8HyX9nYH3x9E.

B. Now use kubectl create monitoring of files created in a secret htpasswd namespace.

[root@kubm-02 traefik]# kubectl create namespace  monitoring
namespace/monitoring created

[root@kubm-02 traefik]# kubectl create secret generic mysecret --from-file auth --namespace=monitoring
secret/mysecret created

Note: Secret must be in the same name space with Ingress object.

C. The following notes attached to the Ingress objects:

traefik.ingress.kubernetes.io/auth-type: "basic"
traefik.ingress.kubernetes.io/auth-secret: "mysecret"

They specify Basic Authentication and that contains a reference Secret mysecret credentials.

Here is the complete Ingress example is based on Prometheus:

cat >prometheus-ingress.yaml<<EOF
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
 name: prometheus-dashboard
 namespace: monitoring
 annotations:
   kubernetes.io/ingress.class: traefik
   traefik.ingress.kubernetes.io/auth-type: "basic"
   traefik.ingress.kubernetes.io/auth-secret: "mysecret"
spec:
 rules:
 - host: dashboard.prometheus.example.com
   http:
     paths:
     - backend:
         serviceName: prometheus
         servicePort: 9090
EOF

kubectl create -f prometheus-ingress.yaml -n monitoring

Name-based routing

First start three web sites

---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: stilton
  labels:
    app: cheese
    cheese: stilton
spec:
  replicas: 2
  selector:
    matchLabels:
      app: cheese
      task: stilton
  template:
    metadata:
      labels:
        app: cheese
        task: stilton
        version: v0.0.1
    spec:
      containers:
      - name: cheese
        image: errm/cheese:stilton
        ports:
        - containerPort: 80
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: cheddar
  labels:
    app: cheese
    cheese: cheddar
spec:
  replicas: 2
  selector:
    matchLabels:
      app: cheese
      task: cheddar
  template:
    metadata:
      labels:
        app: cheese
        task: cheddar
        version: v0.0.1
    spec:
      containers:
      - name: cheese
        image: errm/cheese:cheddar
        ports:
        - containerPort: 80
---
kind: Deployment
apiVersion: extensions/v1beta1
metadata:
  name: wensleydale
  labels:
    app: cheese
    cheese: wensleydale
spec:
  replicas: 2
  selector:
    matchLabels:
      app: cheese
      task: wensleydale
  template:
    metadata:
      labels:
        app: cheese
        task: wensleydale
        version: v0.0.1
    spec:
      containers:
      - name: cheese
        image: errm/cheese:wensleydale
        ports:
        - containerPort: 80

kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/cheese-deployments.yaml

Verify that the server starts

[root@kubm-02 traefik]# kubectl get pods 
NAME                          READY   STATUS              RESTARTS   AGE
cheddar-845749dbd6-vht86      0/1     ContainerCreating   0          64s
cheddar-845749dbd6-z4zn6      1/1     Running             0          64s
curl-6bf6db5c4f-96nhg         1/1     Running             1          46h
stilton-f89c97cdb-dtgbx       1/1     Running             0          64s
stilton-f89c97cdb-nlhn5       1/1     Running             0          64s
wensleydale-7c5ff658b-lq5tn   0/1     ContainerCreating   0          64s
wensleydale-7c5ff658b-ps56g   1/1     Running             0          64s

Set up a service for each site.

---
apiVersion: v1
kind: Service
metadata:
  name: stilton
spec:
  ports:
  - name: http
    targetPort: 80
    port: 80
  selector:
    app: cheese
    task: stilton
---
apiVersion: v1
kind: Service
metadata:
  name: cheddar
spec:
  ports:
  - name: http
    targetPort: 80
    port: 80
  selector:
    app: cheese
    task: cheddar
---
apiVersion: v1
kind: Service
metadata:
  name: wensleydale
  annotations:
    traefik.backend.circuitbreaker: "NetworkErrorRatio() > 0.5"
spec:
  ports:
  - name: http
    targetPort: 80
    port: 80
  selector:
    app: cheese
    task: wensleydale

OR

kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/cheese-services.yaml

Verify that the server starts

[root@kubm-02 traefik]# kubectl get svc  --all-namespaces
NAMESPACE     NAME                      TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE
default       cheddar                   ClusterIP   10.245.80.192    <none>        80/TCP                   50s
default       stilton                   ClusterIP   10.245.177.114   <none>        80/TCP                   50s
default       wensleydale               ClusterIP   10.245.242.131   <none>        80/TCP                   50s

Note: We also set annotations on a back-end service set breaker expression traefik.backend.circuitbreaker to them.

The website is registered to mention traefik

kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/cheese-ingress.yaml

verification:

Completion of task execution management page can be seen in traefik new domain name has been registered to traefik, write hosts file can be accessed in the browser:

hosts file information

172.20.101.166 stilton.minikube
172.20.101.166 cheddar.minikube
172.20.101.166 wensleydale.minikube

Browser access:

http://stilton.minikube
http://cheddar.minikube
http://wensleydale.minikube

Based on the routing path

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: cheeses
  annotations:
    kubernetes.io/ingress.class: traefik
    traefik.frontend.rule.type: PathPrefixStrip
spec:
  rules:
  - host: cheeses.minikube
    http:
      paths:
      - path: /stilton
        backend:
          serviceName: stilton
          servicePort: http
      - path: /cheddar
        backend:
          serviceName: cheddar
          servicePort: http
      - path: /wensleydale
        backend:
          serviceName: wensleydale
          servicePort: http

OR

kubectl apply -f https://raw.githubusercontent.com/containous/traefik/v1.7/examples/k8s/cheeses-ingress.yaml

#写 host
echo "172.20.101.166 cheeses.minikube" | sudo tee -a /etc/hosts

#访问验证
http://cheeses.minikube/cheddar
http://cheeses.minikube/stilton
http://cheeses.minikube/wensleydale

Note: Traefik configured to use the prefix is ​​removed from the annotation traefik.frontend.rule.type url path, so that we can use the previous example of a container without modification.

Route preference

Sometimes you need to specify the priority entrance route, especially when dealing with a wildcard route. This can be done by adding traefik.frontend.priority comment, namely:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: wildcard-cheeses
  annotations:
    traefik.frontend.priority: "1"
spec:
  rules:
  - host: *.minikube
    http:
      paths:
      - path: /
        backend:
          serviceName: stilton
          servicePort: http

kind: Ingress
metadata:
  name: specific-cheeses
  annotations:
    traefik.frontend.priority: "2"
spec:
  rules:
  - host: specific.minikube
    http:
      paths:
      - path: /
        backend:
          serviceName: stilton
          servicePort: http

Note that you must reference priority value to avoid the digital interpretation (for annotation is illegal).

Forwarded to the domain name extension

When specifying the domain name extension, Traefik accordingly will forward the request to a given host and use HTTPS when serving port 443 matches. This is still a need to set the correct port mapping port from Ingress to (external) service on port services.

Disable passing host header case, Traefik will be passed to the incoming host header upstream resources by default. However, sometimes you may not want that. For example, if your service is a need to use many types of domain names.

Prohibited Method

1: Global Disabled:
Add the following to TOML configuration file:

disablePassHostHeaders = true

2: Each Ingress Disabled:
To disable each resource entry pass host header traefik.frontend.passHostHeader, comment setting set on the entrance to "false".

Examples of definitions:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example
  annotations:
    kubernetes.io/ingress.class: traefik
    traefik.frontend.passHostHeader: "false"
spec:
  rules:
  - host: example.com
    http:
      paths:
      - path: /static
        backend:
          serviceName: static
          servicePort: https
以及一个示例服务定义：

apiVersion: v1
kind: Service
metadata:
  name: static
spec:
  ports:
  - name: https
    port: 443
  type: ExternalName
  externalName: static.otherdomain.com

If you want to access example.com/static the request, then the request is received and passed to static.otherdomain.com/static,static.otherdomain.com static.otherdomain.com with Host header.

Note:
Each entry comments are set to cover the global value of anything. So, you can set disablePassHostHeaders true to your TOML profile, then access by each domain name you want.

Deploy multiple Traefik

General deployment of two different types of traefik:

Interior-facing (internal) services traefik, recommended deployment can be used the way
for outside (external) traefik service, we recommend daemonset ways you can use

We recommend the use of traffic-type tag

traffic-type: external
traffic-type: internal

traefik used accordingly labelSelector

traffic-type=internal
traffic-type=external

Load weight distribution

You can use the service weights between multiple deployments in a fine-grained way split Ingress traffic. Let newer versions will receive the initial deployment of a small but growing portion of the requests over time. In Traefik ways to do this is to specify the percentage of each incoming request to be deployed.

For example, suppose that an application running in the my-app version 1. The newer version of the upcoming release 2, but the robustness and reliability of confidence in the new version of the production run of only gradually gain. Therefore, my-app-canary create a new deployment and scaling enough to get a copy of 1% traffic share count. At the same time, create a Service object as usual.

Ingress specification looks like this:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    traefik.ingress.kubernetes.io/service-weights: |
      my-app: 99%
      my-app-canary: 1%
  name: my-app
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: my-app
          servicePort: 80
        path: /
      - backend:
          serviceName: my-app-canary
          servicePort: 80
        path: /

traefik.ingress.kubernetes.io/service-weights Note: the service requests between the rear end of the specified reference distribution that, my-app and my-app-canary. According to this definition, Traefik 99% of the request is routed to the my-app deployment support pod, and 1% of the request is routed to the support pod my-app-canary. Over time, the ratio may gradually turn canary deployed until it was considered to replace the previous major applications, such as Step 5% / 95%, 10% / 90%, 50% / 50%, and finally 100% / 0%.

Load weight distribution, optimal allocation

When specifying the service weights, for convenience reasons, a service can be omitted.
For example, the following definitions show how to publish a request to split the case along the baseline deployed in canary in order to more easily make comparison or automated index canary analysis:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  annotations:
    traefik.ingress.kubernetes.io/service-weights: |
      my-app-canary: 10%
      my-app-baseline: 10%
  name: app
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: my-app-canary
          servicePort: 80
        path: /
      - backend:
          serviceName: my-app-baseline
          servicePort: 80
        path: /
      - backend:
          serviceName: my-app-main
          servicePort: 80
        path: /
此配置my-app-main自动分配80％的流量，从而使用户无需手动完成百分比值。当连续增加金丝雀释放的份额时，这变得很方便。

Use Traefik Tudor deployment ¶
Note

Helm Chart maintained by the community, rather than Traefik project maintenance personnel.

You can also use Traefik Helm chart, rather than directly through the installation Traefik Kubernetes object.

Installation Traefik chart in the following ways:

helm install stable / traefik
use values.yaml file to install Traefik chart.

helm install --values values.yaml stable/traefik

dashboard:
enabled: true
domain: traefik-ui.minikube
kubernetes:
namespaces:

• default
• kube-system
  For more information, please view the document.

Reference documents:

https://docs.traefik.io/user-guide/kubernetes/
https://github.com/containous/traefik/tree/v1.7.14
https://juejin.im/post/5b46119ce51d455d94713787