Production environment, k8s cluster of external exposure and Ingress LoadBalancer services are mainly in two ways:
- LoadBalancer: Cloud vendors need to support, use k8s service load balancing capabilities, that is, relying on iptables / ipvs ability, it can be used for a variety of protocols
- Ingress: relatively more flexible, load balancing through the reverse proxy server, used only for http / https protocol, the need for additional services as well as reverse proxy ingress controller In this scenario, nginx reverse proxy is well-known, in the era of k8s , there has been nginx-ingress , is nginx + ingress combination of controller, ingress controller is responsible for generating nginx configuration of the ingress resource, when the configuration changes is to restart nginx. There has also been a cloud native reverse proxy traefik, which corresponds to the ingress controller which comprises as one, and can dynamically change aware routing rules, need to restart.
traefik is a relatively new reverse proxy, Internet-related information is not particularly rich, study for several days before a successful visit to k8s dashboard, will be one of the key points in this record.
Installation traefik
Use helm installation, traefik 1.7.19 latest chart used:
helm install stable/traefik -f traefik-values.yaml
traefik-values.yaml:
rbac:
enabled: true
dashboard:
enabled: true # 启用traefik dashboard
ingress:
annotations:
traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip
deployment:
hostPort:
httpEnabled: true # traefik pod所在node上开启80端口
httpsEnabled: true # traefik pod所在node上开启443端口
dashboardEnabled: true # traefik pod所在node上开启8080端口,共traefik dashboard使用
ssl:
insecureSkipVerify: true # frontend不验证https的benkend
enabled: true # 启用https入口
extraVolumes:
- name: traefik-ssl
hostPath:
path: /share/k8s/traefik/ssl # 其中存放https入口的证书和key,名字必须为tls.crt,tls.key
type: DirectoryOrCreate
extraVolumeMounts:
- name: traefik-ssl
mountPath: /ssl # traefik pod从/ssl目录读取上述tls.crt,tls.key
Detailed configuration methods, see the official documents , the above-mentioned key points are as follows:
- Https inlet opening provided ssl.enabled = true, then a certificate and key, the above-mentioned node by node from the local directory mount the pod to the embodiment, each node first node and put certificate key, better way by k8s secret, then mount the pod to create a secret
- How to access to the entrance, I was through the open port on the node, then you can access to the entrance through the node pod where, through http: // nodeip or https: // nodeip; you can also use NodePort type of service, so at http: // any-nodeip: http-nodeport or https: // any-nodeip: https-nodeport access, value set serviceType: nodePort
- PathPrefixStrip routes matching rules I use the default name is the host match
Because traefik dashboard is enabled, the installation will automatically create a dashboard traefik of ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip
labels:
app: traefik
chart: traefik-1.82.1
heritage: Tiller
release: traefik
name: traefik-dashboard
namespace: default
spec:
rules:
- host: traefik.example.com
http:
paths:
- backend:
serviceName: traefik-dashboard
servicePort: dashboard-http
traefik by label app: Select the required perception of ingress traefik. You added ingress note contains the tag. Above annotations and host are derived from the value. Because I do not want with host, so use PathPrefixStrip routing rules, I modified the above ingress as follows:
spec:
rules:
- http:
paths:
- backend:
serviceName: traefik-dashboard
servicePort: dashboard-http
path: /traefik
So that when using http: // nodeip / traefik can access the dashboard, because the node also opens the dashboard port, or through http: // nodeip: 8080 visit.
Acting k8s dashboard
The latest k8s dashboard (v2.0.0-beta6) mounted kubernetes-dashboard namespace:
kubectl get svc -n kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.254.238.13 <none> 8000/TCP 21d
kubernetes-dashboard LoadBalancer 10.254.253.226 <pending> 443:30223/TCP 21d
Increased ingress:
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
annotations:
traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip
labels:
app: traefik
name: kubernetes-dashboard
namespace: default
spec:
rules:
- http:
paths:
- backend:
serviceName: kubernetes-dashboard
servicePort: 443
path: /k8s
Https proxy backend
k8s dashboard only supports https access, the first question is how stuck https proxy service, frontend to backend routing will appear the following situations:
http->http
http->https
https->http
https->https
When the backend is https, whether frontend http or https, that is, 2 and 4, will be reported 500 errors, can not be verified because the frontend backend, this time Solution:
- Either set insecureSkipVerify, this is relatively simple, if this is always the best way frontend using https, which is set redirect
- Either set the ingress tls, configure host of tls certificate information
Set insecureSkipVerify method I used. Generally the best way is to use the entrance always use https, then end tls, whether the backend https unimportant.
Acting different namespace service
After solving the above problems, the next encounter k8s dashboard service can not access issues, in traefik dashboard displays in red, because the helm traefik installed by default in the default namespace rather k8s dashboard mounted kubernetes-dashboard namespace, you can not access across namespace to service, the solution:
Either the traefik installed in the same space and k8s dashboard
Either the dashboard service into the default namespace by ExternalName
apiVersion: v1 kind: Service metadata: name: kubernetes-dashboard namespace: default spec: ports: - name: https port: 443 protocol: TCP targetPort: 443 sessionAffinity: None type: ExternalName externalName: kubernetes-dashboard.kubernetes-dashboard.svc.cluster.local
ExternalName method I used. The full domain name service is servicename.namespace.svc.cluster.local, cluster.local kubelet is configured.
Based on path routing
Services can be visited, but appeared MIME type is not a supported stylesheet MIME type errors.
Initially I thought it was traefik joined the X-Content-Type-Options in reponse header in: nosniff, but found traefik default is not added.
Finally found url path, I use only the ingress path routing, do not use host.
When using https: when // nodeip / k8s access k8s dashboard, because the routing rules are PathPrefixStrip, to the back-end request is https: // nodeip, then get home, the file name is k8s, the main page k8s in css, js, etc. file path is relative to the current document path, so the request url is https: //nodeip/xxx.css, then you do not match the rules of the road, the above error.
If you use https: // nodeip / k8s / access dashboard, on all normal.
So there is a certain risk when using path matching the route, and resource path in the definition of home:
Home page css, js and other resource path defined way | Explanation |
---|---|
Not defined base, not the path to resources ./ or ../ and / or at the beginning or <base href="./"> resource path beginning with ./ |
1. Match / path, only through https: // xxxx / path / access 2. Match / path, and redirected to a rear end sub /, this time through https: // xxxx / path / or https: // xxxx / path can be accessed |
<base href="/"> , Resource path not to ./ or ../ or / beginning |
Only match / matches other paths are not working really encountered such a situation, such as monocular |
So the best way to match or host route.
dashboard certification
After understanding the above-mentioned problems, and finally into the dashboard login screen:
At first I was using http entrance, using Token landing approach, there is no response, through the development of tools to view, find the problem in the use http entrance, header does not carry jweToken, authentication fails, you must use https entrance.
Recall before the adoption of kubectl proxy, that is http://localhost:8001/api/v1/namespaces/kubernetes-dashboard/services/https:kubernetes-dashboard:/proxy/
also not landing, in fact, is the same problem.
Use http entrance landing failed:
When using the https entry, jweToken is carrying, landing success:
)
So decisive set frontend always https, values increase traefik.ingress.kubernetes.io/redirect-entry-point: https, then helm upgrade, increase the ingress own needs its own modification:
dashboard:
enabled: true
ingress:
annotations:
traefik.ingress.kubernetes.io/rule-type: PathPrefixStrip
traefik.ingress.kubernetes.io/redirect-entry-point: https
# 不要使用ingress.kubernetes.io/ssl-redirect: "true",因为会丢掉path
Whether such use http: // nodeip / k8s / or https: // nodeip / k8s / can be successfully landed.
Welcome to Zhong Pan's blog