Get to know the Kubernetes multi-cluster service API

sunset-g55251915a_1280

For various reasons, enterprises using Kubernetes have several, dozens or even hundreds of clusters within them. For example, considering the R&D process, there are independent clusters in different environments; considering the regulatory level, user data stored on-site needs to be matched with application clusters; the capacity of a single cluster cannot meet the business volume; availability requirements are multi-cloud, Multiple locations and multiple centers; Kubernetes in-situ upgrade costs a lot, so new clusters are considered; and other reasons.

These clusters appear to be independent of each other, but are inextricably linked. For example, high-availability multi-clusters achieve cluster-level disaster recovery, but how do services in the cluster achieve cross-cluster fault migration?

Let's first look at how applications within the cluster provide services outside the cluster. Due to the network isolation feature of Kubernetes, there is a natural network boundary, and some solutions (such as Ingress, NodePort) are needed to expose services outside the cluster. Although the connectivity problem is solved, the registration and discovery of services cannot be solved yet.

k8s-svc-exposing

Usually we regard Service as the service registration and discovery of the Kubernetes platform. The Multi-Cluster Service (Multi-Cluster Service API, MCS API for short) to be introduced today can be regarded as service registration and discovery across Kubernetes clusters .

Introduction to MCS

MCS API comes from Kubernetes Enhancement Proposal KEP-1645: Multi-Cluster Services API is currently in the proposal stage.

The goal of MCS is to define a set of APIs to realize cross-cluster registration and discovery of services since multi-clusters are inevitable, and this set of APIs is lightweight enough to be able to access services of other clusters like accessing local services.

Note that the MCS API only provides design guidance for API and implementation, but does not provide specific implementation.

the term

clusterset: cluster set, shared services between clusters in the set.
mcs-controller: Multi-cluster service controller, which completes service registration and discovery among multiple clusters.
cluster name: The unique identifier of the cluster, used to identify which cluster the registered service comes from.

CRD

ServiceExportDefine services exposed to other clusters in the cluster. Like Servicemust be created in the same namespace as the load and have Servicethe same name as .
ServiceImportDefines other imported clusters Service.

type ServiceExport struct {
  metav1.TypeMeta `json:",inline"`
  metav1.ObjectMeta `json:"metadata,omitempty"`
  Status ServiceExportStatus `json:"status,omitempty"`
}

type ServiceImport struct {
  metav1.TypeMeta `json:",inline"`
  metav1.ObjectMeta `json:"metadata,omitempty"`
  Spec ServiceImportSpec `json:"spec,omitempty"`
  Status ServiceImportStatus `json:"status,omitempty"`
}
type ServiceImportSpec struct {
  Ports []ServicePort `json:"ports"`
  IPs []string `json:"ips,omitempty"`
  Type ServiceImportType `json:"type"`
  SessionAffinity corev1.ServiceAffinity `json:"sessionAffinity"`
  SessionAffinityConfig *corev1.SessionAffinityConfig `json:"sessionAffinityConfig"`
}

Cross-cluster service registration

If multiple clusters declare services with the same name and namespace, these services will be treated as a single composite service. barFor example, there are 4 clusters under 1 cluster set, and 1 cluster has a name under foothe namespace Service. If other clusters want to use this cluster Service, the cluster needs to declare to the outside world that it Servicecan provide services across clusters and complete the registration of multi-cluster services.

mcs-service

Registration of multi-cluster services is completed by creating resources Servicewith the same name in the same namespace . It can be created manually, or it can realize automatic registration of supported services: automatically register services in the cluster or in the specified namespace according to the identifier.ServiceExportServiceExport

For more design details, please refer to Service Exporting Design Details .

Cross-cluster service discovery

After ServiceExportthe resource is created, the service will be "discovered" by other clusters under the cluster set: a ServiceImportresource with the same name appears under the namespace with the same name, and the is associated ServiceImportwith the registered service .Endpoint

mcs-exported

Similarly, if a service goes offline or stops cross-cluster services, ServiceExportafter deleting the resource, the resources in other clusters ServiceImportwill be destroyed accordingly. ServiceImportThe management is completed by mcs-controller in MCS implementation.

It is worth noting that regarding ServiceImportthe use of , the specific implementation details are not defined in the standard. Since Pod is a non-permanent resource in Kubernetes, we usually use it when accessing applications Service. The ideal situation for multi-cluster services is that Servicethey can automatically use other clusters when using them Endpoints, so that they are unaware of the application.

One solution is to modify Service APIthe implementation of (for example kube-proxy) to use ServiceImport.

Another solution is to Service APIcreate a shadow service by mcs-controller without modifying the current implementation, and this service is associated with the newly created one EndpointSlice, which aggregates all of the cross-services Endpoints. ServiceImportThere may be data from multiple clusters under a EndpointSlice, and each EndpointSlicehas the identity of its own cluster.

mcs-endpoints

ClusterIPThe solution with DNS is suggested and details provided in the official design details , but it is not the only implementation solution.

Summarize

MCS API hopes to provide the definition of multi-cluster services at the API level, but due to the complexity of its implementation, the progress of this proposal has been slow. From the proposal in 2020.2 to the alpha implementation of the API in 2020.8, there has not been much progress.

Even so, the registration and discovery mechanism of multi-cluster services can bring certain reference value to subsequent implementations.

Articles are published uniformly on the public account云原生指北