Kubernetes Notes (04) - Why not directly use Pod, Job/CronJob definition, use YAML to describe Job/CronJob, operate Job/CronJob in Kubernetes

1. Why not use Pod directly

KubernetesThe core object Podis used to arrange one or more containers, so that these containers share resources such as network and storage, and are always scheduled together, so as to work closely together.

Because is more representative of actual applications Podthan containers, it Kubernetesis not used to orchestrate services at the container level, but is used Podas the smallest unit for scheduling O&M in the cluster.

Although object-oriented design ideas are mostly used in software development, it is unexpectedly appropriate to put them Kubernetesin . Because Kubernetesusing YAMLto describe resources, the business is simplified into individual objects, which have attributes internally and external connections, and also need to cooperate with each other, but we don't need programming, and it is completely handled Kubernetesautomatically Kubernetes.Go object-oriented).

There are many basic principles in object-oriented design, two of which I think describe Kubernetesthe object , one is "single responsibility" and the other is "composition is better than inheritance".

• "Single responsibility" means that the object should only focus on doing one thing well, don't be greedy for everything, and keep the granularity small enough to make it easier to reuse and manage.
• "Combination is better than inheritance" means that objects should be connected at runtime as much as possible to keep loose coupling, instead of fixing the relationship of objects in a hard-coded way.

Applying these two principles, the resource object we will Kuberneteslook will be very clear. Because it is Podalready a relatively complete object, which is responsible for managing containers, so we should not blindly expand its functions by "adding to the snake", but to maintain its independence. Functions outside the container need to define other objects , "combined" Podas one of its members.

2. Job/CronJob

KubernetesTwo new objects in: Joband CronJob, they are combined Podto realize the processing of offline business.

• There are many "online business" types of applications, such as Nginx, Node.js, MySQL, Redisand so on. Once they are running, they will basically not stop, that is, they will always be online.
• The characteristic of "offline business" is that it will definitely exit and will not run indefinitely, so its scheduling strategy is very different from "online business". It needs to consider running timeout, status check, failure retry, and acquisition Management matters such as calculation results.

However, these business features are not necessarily related to container management. If they are implemented Podby will assume unnecessary obligations and violate the "single responsibility". Therefore, we should separate this part of the function to another object and let this object go. PodControl the operation and complete additional work.

"Offline business" can also be divided into two types. One is a "temporary task", which is finished after running, and will be rescheduled next time if there is a need; the other is a "scheduled task", which can run on time and periodically without too much intervention.

Corresponds to Kuberneteswhere :

• The "temporary task" is APIthe object Job;
• "Timed task" is APIthe object CronJob;

Using these two objects, you can schedule and manage any offline business Kubernetesin .

3. Use YAML to describe the job

JobThe YAML"file header" part of the file is still the required fields, briefly:

• apiVersionNo v1, but batch/v1.
• kindYes Job, this is consistent with the name of the object.
• metadataYou still have to have namethe tag , and you can labelsadd arbitrary tags with .

job/CronjobThe apiVersionfield is batch/v1, indicating that they are not part of the core object group core group, but the batch object group batch group.

You can also use the command kubectl explain jobto view its field descriptions. However, if you want to generate a YAMLtemplate file, you cannot use it kubectl run, because kubectl runyou can only create it Pod. To Podcreate APIobjects other than , you need to use the command kubectl createand add the type name of the object.

For example, to busyboxcreate one with echo-job, the command is like this:

export out="--dry-run=client -o yaml"              # 定义Shell变量
kubectl create job echo-job --image=busybox $out

A basic YAMLfile , and after saving and making some modifications, there will be a Jobobject:

apiVersion: batch/v1
kind: Job
metadata:
  name: echo-job

spec:
  template:
    spec:
      restartPolicy: OnFailure
      containers:
      - image: busybox
        name: echo-job
        imagePullPolicy: IfNotPresent
        command: ["/bin/echo"]
        args: ["hello", "world"]

You will notice Jobthat the description of Podis very similar to , but there are some differences. The main difference is in specthe field . There is one more templatefield, and then another spec, which seems a bit strange.

If you understand the object-oriented design ideas just mentioned, you will understand the reason for this approach. It actually applies the composite mode in Jobthe object , templateand the field defines an "application template", which embeds one Pod, so Jobthat can be created from this template Pod.

And Podbecause is under Jobthe management control of , it does not directly apiserverdeal with , so there is no need to repeat apiVersionthe "header fields" such as . It only needs to define the key specand describe the information related to the container. It can be said to be a "headless" Podobject .

In fact, there are not many extra functions echo-jobin , just Poda simple package:

Image source: https://time.geekbang.org/column/article/531566

In general, Podthe work is very simple, containerswrite the name and image in , commandexecute /bin/echo, and output "hello world".

However, because of the particularity of Jobthe business , we also need specto add an additional field in restartPolicyto determine the strategy when Podthe operation fails, OnFailurewhich is to restart the container on failure, Neverwhile does not restart the container and let Jobto reschedule to generate a new one Pod.

4. Operate Job in Kubernetes

Now let's create Jobthe object and run this simple offline job, again with the command kubectl apply:

kubectl apply -f job.yml

After creation Kubernetes, will be YAMLextracted from the template definition of andPod run under the control of , you can use to view the status of and respectively:JobPodkubectl get jobkubectl get podJobPod

$ kubectl apply -f job.yml
job.batch/echo-job created
wohu@dev:~/k8s$ kubectl get pod
NAME             READY   STATUS      RESTARTS   AGE
echo-job-lt2sq   0/1     Completed   0          6s
ngx              1/1     Running     0          5h37m
wohu@dev:~/k8s$ kubectl get job
NAME       COMPLETIONS   DURATION   AGE
echo-job   1/1           2s         11s
wohu@dev:~/k8s$

It can be seen that because Podis Jobmanaged by , it will not repeatedly restart and report an error, but will be displayed as to Completedindicate that the task is completed, and Jobwill also list the number of jobs that ran successfully. There is only one job here, so it is 1/1.

You can also see Podthat a name is automatically associated Jobwith the name ( )echo-job plus a random string (lt2sq). JobYou can use the command kubectl logsto get Podthe running result of :

$ kubectl logs echo-job-lt2sq
hello world

KubernetesThis framework for YAMLdescribing objects provides a lot of flexibility. You can add arbitrary fields at Jobthe level and Podlevel to customize the business. This advantage is incomparable with simple container technology. For example, the following fields, other more detailed information can refer to JobDocumentation

• activeDeadlineSeconds, set the timeout for Podrunning .
• backoffLimit, sets the number Podof failed retries for .
• completions, Jobhow many times to complete Pod, the default is 1.
• parallelism, which is completionsrelated indicates Podthe number of allowed concurrent operations to avoid excessive resource occupation.

It should be noted that these 4 fields are not under templatethe field , but specunder the field, so they belong to Jobthe level and are used to control Podthe objects in the template.

Next, I will create another Jobobject named "sleep-job", which sleeps for a random period of time and then exits, simulating a long-running job (for example MapReduce). JobThe parameter is set to a 15-second timeout, and a maximum of 2 retries is required. A total of 4 operations need to be completed Pod, but at most 2 concurrent operations at the same time Pod:

apiVersion: batch/v1
kind: Job
metadata:
  name: sleep-job

spec:
  activeDeadlineSeconds: 15
  backoffLimit: 2
  completions: 4
  parallelism: 2

  template:
    spec:
      restartPolicy: OnFailure
      containers:
      - image: busybox
        name: echo-job
        imagePullPolicy: IfNotPresent
        command:
          - sh
          - -c
          - sleep $(($RANDOM % 10 + 1)) && echo done

Job configuration details:
job.spec.containers.template.spec.containers.image cannot specify the image version number, only the image: complete image: the version number can only be defined by the pod, otherwise the image will be pulled from the Internet, If you can connect to the Internet, of course it’s fine. In the offline environment, an error will be reported directly and the image cannot be pulled, although you do have this version of the image locally and the imagePullPolicy is set to Never or IfNotPresent.
For example, I am in an offline environment, and the image configuration in the job is: - image: busybox:1.35.0, then an error will be reported and the image cannot be pulled.

After using kubectl applyCreate Job, we can use kubectl get pod -wto observe Podthe status of in real time, and see the process of being queued, created, and running Pod continuously :

kubectl apply -f sleep-job.yml
kubectl get pod -w

When Podall finished running, we will use kubectl getto look Jobat Podthe status of and , and

we will see Jobthat the number of completed is 4 as we expected, and all 4 Podare also in the completed state.

JobIt will not be deleted immediately after the running, this is for the convenience of obtaining the calculation results, but if too many completed Jobwill consume system resources, you can use the field ttlSecondsAfterFinishedto set a retention time limit.

5. Use YAML to describe CronJob

1. Because the name CronJobof is a bit long, Kubernetesa shorthand is provided cj, which can also kubectl api-resourcesbe seen ;
2. CronJobIt needs to be run regularly, so we also need to specify parameters on the command line --schedule.

Use the command directly kubectl createto create CronJobtemplates for .

export out="--dry-run=client -o yaml"              # 定义Shell变量
kubectl create cj echo-cj --image=busybox --schedule="" $out

Then we edit the YAMLboilerplate to generate CronJobthe object:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: echo-cj

spec:
  schedule: '*/1 * * * *'
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          containers:
          - image: busybox
            name: echo-cj
            imagePullPolicy: IfNotPresent
            command: ["/bin/echo"]
            args: ["hello", "world"]

We still focus on its specfields , and you will find that it has three consecutive specnesting levels:

• The first specis CronJobthe own object specification declaration.
• The second specbelongs to jobTemplate, which defines a Jobobject .
• The third specis subordinate template, which defines what runs Jobin Pod.

Therefore, CronJobin fact, the new object Jobgenerated by

In addition to Jobdefining jobTemplatethe field of the object, CronJobthere is also a new field that is scheduleused to define the rules for the periodic operation of the task. It uses the standard Cronsyntax to specify minutes, hours, days, months, weeks, theLinux same as on .crontab

Except for the different names, theCronJob usage is almost the same as , use to create , use , to view the status:Jobkubectl applyCronJobkubectl get cjkubectl get pod

kubectl apply -f cronjob.yml
kubectl get cj
kubectl get pod

For the sake of saving resources, it CronJobwill not keep the running ones indefinitely Job, it only keeps the 3 most recent execution results by default, but it can successfulJobsHistoryLimitbe changed .

CronTime setting syntax: https://crontab.guru/

6. Summary

Through this nesting method, Kubernetesthese APIobjects in form a "control chain":

CronJobUse timing rules to control Job, Jobuse concurrent quantity control Pod, Poddefine parameters to control the container, and the container then isolates the control process, and the process finally realizes the business function. The progressive form is a bit like the design mode Decorator(decoration mode), each in the chain Each link performs its own duties and completes the task under the unified command Kubernetesof the leader.

1. PodKubernetesis the smallest scheduling unit of , but in order to maintain its independence, no redundant functionality should be added to it.
2. KubernetesTwo objects, Joband , are provided for offline business , which deal with "temporary tasks" and "scheduled tasks" respectively.CronJobAPI
3. JobThe key field is spec.template, which defines Podthe template , and other important fields are completions, parallelismetc.
4. CronJobThe key fields of are spec.jobTemplateand spec.schedule, which define Jobthe template .