Solve the proxy problem of project migration to Kubernetes cluster

Others 2020-04-22 16:13:51 views: null

Solve the proxy problem of project migration to Kubernetes cluster

As Kubernetes technology matures, more and more enterprises choose to use Kubernetes clusters to manage projects. The new project is okay, you can choose the appropriate cluster size to build the project from scratch; the old project needs to consider many factors when moving into the Kubernetes cluster, after all, the project cannot be interrupted for too long.

Source of the problem

Recently, when doing a project migration to a Kubernetes cluster, I encountered an interesting problem: because the development version of dubbo is too low, it is not registered in zookeeper, you need to develop and upgrade dobbo, and then package it into a mirror, so you must first migrate nodejs Enter the Kubernets cluster. Because part of the business is migrated into the Kubernets cluster, it is necessary to add a layer of proxy Nginx in front of traefik (Nginx is the entrance of the old business, the micro service behind the reverse proxy, Alibaba Cloud's slb points to nginx, and wait until the business is completely migrated, slb points to traefik). This kind of architecture is a two-layer proxy, namely Slb-> Nginx-> Traefik-> Service.

Graphic

solution:

  1. The business of migrating to k8s cluster takes Nodeport, Nginx-> Nodeport. The business application is directly Nodeport, which is not easy to manage. If 10,000 machines are used, you can't also use Nodeport. You have to plan the port yourself. If there are more machines, each machine will still expose the port. It is not realistic to think about it.
  2. The business of migrating to k8s cluster takes Clusterip, Nginx-> Traefik-> Service. This way is reasonable.

Solve the problem

You can't use the production environment to write blog posts. In fact, there is a difference between the virtual machine and the production machine in terms of the network environment.

Mental analysis

  1. Deploy k8s cluster
  2. Deploy nginx
  3. Deploy traefik
  4. Deploy the application
  5. Joint Commissioning Test

Deploy k8s cluster

Use my previous blog post deployment method: https://www.cnblogs.com/zisefeizhu/p/12505117.html

Deploy nginx

Download the necessary components

# hostname -I
20.0.0.101
# cat /etc/redhat-release 
CentOS Linux release 7.6.1810 (Core) 
# uname -a
Linux fuxi-node02-101 4.4.186-1.el7.elrepo.x86_64 #1 SMP Sun Jul 21 04:06:52 EDT 2019 x86_64 x86_64 x86_64 GNU/Linux
# wget http://nginx.org/download/nginx-1.10.2.tar.gz
# wget http://www.openssl.org/source/openssl-fips-2.0.10.tar.gz
# wget http://zlib.net/zlib-1.2.11.tar.gz
# wget https://ftp.pcre.org/pub/pcre/pcre-8.40.tar.gz
# yum install gcc-c++

Configure-compile-install software

# tar zxvf openssl-fips-2.0.10.tar.gz
# cd openssl-fips-2.0.10/
# ./config && make && make install
# cd ..
# ll
tar zxvf pcre-8.40.tar.gz
# cd pcre-8.40/
# ./configure && make && make install
# tar zxvf zlib-1.2.11.tar.gz
# cd zlib-1.2.11/
# ./configure && make && make install
# tar zxvf nginx-1.10.2.tar.gz
# cd nginx-1.10.2/
#./configure --with-http_stub_status_module --prefix=/opt/nginx
# make && make install

Start Nginx

# pwd
/opt/nginx
# ll
总用量 4
drwx------ 2 nobody root    6 4月  22 11:30 client_body_temp
drwxr-xr-x 2 root   root 4096 4月  22 12:53 conf
drwx------ 2 nobody root    6 4月  22 11:30 fastcgi_temp
drwxr-xr-x 2 root   root   40 4月  22 11:29 html
drwxr-xr-x 2 root   root   41 4月  22 14:24 logs
drwx------ 2 nobody root    6 4月  22 11:30 proxy_temp
drwxr-xr-x 2 root   root   19 4月  22 11:29 sbin
drwx------ 2 nobody root    6 4月  22 11:30 scgi_temp
drwx------ 2 nobody root    6 4月  22 11:30 uwsgi_temp
# sbin/nginx

traefik deployment

https://www.cnblogs.com/zisefeizhu/p/12692979.html

Environmental inspection

# kubectl get pods,svc -A | grep traefik
kube-system   pod/traefik-ingress-controller-z5qd7             1/1     Running       0          136m
kube-system   service/traefik                     ClusterIP   10.68.251.132   <none>        80/TCP,443/TCP,8080/TCP        4h14m

Browser access

Deploy the application

The test application here chooses the containous / whoami image

Test application deployment

# cat whoami.yaml 
##########################################################################
#Author:                     zisefeizhu
#QQ:                         2********0
#Date:                       2020-04-22
#FileName:                   whoami.yaml
#URL:                        https://www.cnblogs.com/zisefeizhu/
#Description:                The test script
#Copyright (C):              2020 All rights reserved
###########################################################################
apiVersion: v1
kind: Service
metadata:
  name: whoami
spec:
  ports:
    - protocol: TCP
      name: web
      port: 80
  selector:
    app: whoami
---
kind: Deployment
apiVersion: apps/v1
metadata:
  name: whoami
  labels:
    app: whoami
spec:
  replicas: 2
  selector:
    matchLabels:
      app: whoami
  template:
    metadata:
      labels:
        app: whoami
    spec:
      containers:
        - name: whoami
          image: containous/whoami
          ports:
            - name: web
              containerPort: 80

# kubectl get svc,pod
NAME                 TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)             AGE
service/whoami       ClusterIP   10.68.109.151   <none>        80/TCP              3h30m

NAME                                 READY   STATUS    RESTARTS   AGE
pod/whoami-bd6b677dc-jvqc2           1/1     Running   0          3h30m
pod/whoami-bd6b677dc-lvcxp           1/1     Running   0          3h30m

Joint Commissioning Test

Because the selected solution to the problem is: nginx-> traefik-> service

  1. traefik -->service
  2. nginx --> traefik
  3. nginx --> service

traefik --> service

Use the traefik agent to test the application's resource list:

# cat traefik-whoami.yaml 
##########################################################################
#Author:                     zisefeizhu
#QQ:                         2********0
#Date:                       2020-04-22
#FileName:                   traefik-whoami.yaml
#URL:                        https://www.cnblogs.com/zisefeizhu/
#Description:                The test script
#Copyright (C):              2020 All rights reserved
###########################################################################
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: simpleingressroute
spec:
  entryPoints:
    - web
  routes:
  - match: Host(`who.linux.com`) && PathPrefix(`/notls`)
    kind: Rule
    services:
    - name: whoami
      port: 80

Local hosts parse the

traefik interface and observe that the proxy is successful:

visit who.linux.com/notls nginx-

> traefik

# cat conf/nginx.conf
user nobody;
worker_processes 4;
events {
	use epoll;
	worker_connections 2048;
}
http {
	upstream app {
		server  20.0.0.202;  
	}

	server {
		listen 80;
#		server_name who2.linux.com;
  	access_log logs/access.log;
  	error_log logs/error.log;
  	location / {
   		proxy_set_header X-Forwarded-For $remote_addr;
    	proxy_set_header X-Real-IP $remote_addr;
    	proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    	proxy_set_header Host $host;
			proxy_headers_hash_max_size 51200;
			proxy_headers_hash_bucket_size 6400;
    	proxy_redirect off;
    	proxy_read_timeout 600;
    	proxy_connect_timeout 600;
    	proxy_pass http://app;
  	}
	}
}

# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
20.0.0.202  who.linux.com   //k8s集群traefik所落节点，其实K8s任意节点都随便拉
# curl -iL who.linux.com/notls
HTTP/1.1 200 OK
Content-Length: 388
Content-Type: text/plain; charset=utf-8
Date: Wed, 22 Apr 2020 07:33:52 GMT

Hostname: whoami-bd6b677dc-lvcxp
IP: 127.0.0.1
IP: 172.20.46.67
RemoteAddr: 172.20.177.153:58168
GET /notls HTTP/1.1
Host: who.linux.com
User-Agent: curl/7.29.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 20.0.0.101
X-Forwarded-Host: who.linux.com
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-ingress-controller-z5qd7
X-Real-Ip: 20.0.0.101

If you are not familiar with nginx, please read this big man's blog post: https://www.cnblogs.com/kevingrace/p/6095027.html

nginx --> service

# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
20.0.0.101  who.linux.com
# curl -iL who.linux.com/notls
HTTP/1.1 200 OK       //响应信息
Server: nginx/1.10.2    //响应服务    
Date: Wed, 22 Apr 2020 07:27:46 GMT
Content-Type: text/plain; charset=utf-8
Content-Length: 389
Connection: keep-alive

Hostname: whoami-bd6b677dc-jvqc2
IP: 127.0.0.1
IP: 172.20.46.111
RemoteAddr: 172.20.177.153:38298
GET /notls HTTP/1.1
Host: who.linux.com
User-Agent: curl/7.29.0
Accept: */*
Accept-Encoding: gzip
X-Forwarded-For: 20.0.0.101   
X-Forwarded-Host: who.linux.com
X-Forwarded-Port: 80
X-Forwarded-Proto: http
X-Forwarded-Server: traefik-ingress-controller-z5qd7
X-Real-Ip: 20.0.0.101

nginx日志
# tail -f access.log
20.0.0.101 - - [22/Apr/2020:15:28:28 +0800] "GET /notls HTTP/1.1" 200 389 "-" "curl/7.29.0"

Browser testing


Continue testing

Turn off the traefik application, and then test

# kubectl delete -f .
configmap "traefik-config" deleted
customresourcedefinition.apiextensions.k8s.io "ingressroutes.traefik.containo.us" deleted
customresourcedefinition.apiextensions.k8s.io "ingressroutetcps.traefik.containo.us" deleted
customresourcedefinition.apiextensions.k8s.io "middlewares.traefik.containo.us" deleted
customresourcedefinition.apiextensions.k8s.io "tlsoptions.traefik.containo.us" deleted
customresourcedefinition.apiextensions.k8s.io "traefikservices.traefik.containo.us" deleted
ingressroute.traefik.containo.us "traefik-dashboard-route" deleted
service "traefik" deleted
daemonset.apps "traefik-ingress-controller" deleted
serviceaccount "traefik-ingress-controller" deleted
clusterrole.rbac.authorization.k8s.io "traefik-ingress-controller" deleted
clusterrolebinding.rbac.authorization.k8s.io "traefik-ingress-controller" deleted


# kubectl delete -f traefik-whoami.yaml   //关闭whoami traefik代理
ingressroute.traefik.containo.us "simpleingressroute" deleted


Not to mention that the test results are very clear: visit who.linux.com traffic trend: nginx-> traefik-> service.

Guess you like

Origin www.cnblogs.com/zisefeizhu/p/12752566.html
Recommended
Ranking
2019 JD pen questions - Choir
Grafana installation & basic configuration
How to solve the high gas fee of Filecoin? The official attitude is bright!
Informatization development 59
Date,DateFormat,Calendar,
Introduction to MySQL - Understanding MySQL
win10 conexão de área de trabalho remota ubuntu20 (RDP)
52. Customize generics on methods
Detailed explanation of CAP, ACID, BASE theory and NWR practice strategy
UnknownHostException：XXX：Name or service not know
Daily
More
2024-08-01(0)
2024-07-31(0)
2024-07-30(0)
2024-07-29(0)
2024-07-28(0)
2024-07-27(0)
2024-07-26(0)
2024-07-25(0)
2024-07-24(0)
2024-07-23(0)

Code World

© 2018 codetd.com