Control Plane Controller

The Control Plane controller’s main responsibilities are:

• Managing a set of machines that represent a Kubernetes control plane.
• Provide information about the state of the control plane to downstream consumers.
• Create/manage a secret with the kubeconfig file for accessing the workload cluster.

A reference implementation is managed within the core Cluster API project as the Kubeadm control plane controller (KubeadmControlPlane). In this document, we refer to an example ImplementationControlPlane where not otherwise specified.

Contracts

Control Plane Provider

The general expectation of a control plane controller is to instantiate a Kubernetes control plane consisting of the following services:

Required Control Plane Services

• etcd
• Kubernetes API Server
• Kubernetes Controller Manager
• Kubernetes Scheduler

Optional Control Plane Services

• Cloud controller manager
• Cluster DNS (e.g. CoreDNS)
• Service proxy (e.g. kube-proxy)

Prohibited Services

• CNI - should be left to user to apply once control plane is instantiated.

Relationship to other Cluster API types

The Cluster controller will set an OwnerReference on the Control Plane. The Control Plane controller should normally take no action during reconciliation until it sees the ownerReference.

A Control Plane controller implementation should exit reconciliation until it sees cluster.spec.controlPlaneEndpoint populated.

The Cluster controller bubbles up status.ready into status.controlPlaneReady and status.initialized into a controlPlaneInitialized condition from the Control Plane CR.

The ImplementationControlPlane must rely on the existence of status.controlplaneEndpoint in its parent Cluster object.

CRD contracts

The CRD name must have the format produced by sigs.k8s.io/cluster-api/util/contract.CalculateCRDName(Group, Kind). The same applies for the name of the corresponding ControlPlane template CRD.

Required spec fields for implementations using replicas

• replicas - is an integer representing the number of desired replicas. In the KubeadmControlPlane, this represents the desired number of control plane machines.

• scale subresource with the following signature:

scale:
  labelSelectorPath: .status.selector
  specReplicasPath: .spec.replicas
  statusReplicasPath: .status.replicas
status: {}

More information about the scale subresource can be found in the Kubernetes documentation.

Required spec fields for implementations using version

• version - is a string representing the Kubernetes version to be used by the control plane machines. The value must be a valid semantic version; also if the value provided by the user does not start with the v prefix, it must be added.

Required spec fields for implementations using Machines

• machineTemplate - is a struct containing details of the control plane machine template.

• machineTemplate.metadata - is a struct containing info about metadata for control plane machines.

• machineTemplate.metadata.labels - is a map of string keys and values that can be used to organize and categorize control plane machines.

• machineTemplate.metadata.annotations - is a map of string keys and values containing arbitrary metadata to be applied to control plane machines.

• machineTemplate.infrastructureRef - is a corev1.ObjectReference to a custom resource offered by an infrastructure provider. The namespace in the ObjectReference must be in the same namespace of the control plane object.

• machineTemplate.nodeDrainTimeout - is a *metav1.Duration defining the total amount of time that the controller will spend on draining a control plane node. The default value is 0, meaning that the node can be drained without any time limitations.

• machineTemplate.nodeVolumeDetachTimeout - is a *metav1.Duration defining how long the controller will spend on waiting for all volumes to be detached. The default value is 0, meaning that the volume can be detached without any time limitations.

• machineTemplate.nodeDeletionTimeout - is a *metav1.Duration defining how long the controller will attempt to delete the Node that is hosted by a Machine after the Machine is marked for deletion. A duration of 0 will retry deletion indefinitely. It defaults to 10 seconds on the Machine.

Required status fields

The ImplementationControlPlane object must have a status object.

The status object must have the following fields defined:

Required status fields for implementations using replicas

Where the ImplementationControlPlane has a concept of replicas, e.g. most high availability control planes, then the status object must have the following fields defined:

Required status fields for implementations using version

• version - is a string representing the minimum Kubernetes version for the control plane machines in the cluster. NOTE: The minimum Kubernetes version, and more specifically the API server version, will be used to determine when a control plane is fully upgraded (spec.version == status.version) and for enforcing Kubernetes version skew policies in managed topologies.

Optional status fields

The status object may define several fields:

• failureReason - is a string that explains why an error has occurred, if possible.
• failureMessage - is a string that holds the message contained by the error.
• externalManagedControlPlane - is a bool that should be set to true if the Node objects do not exist in the cluster. For example, managed control plane providers for AKS, EKS, GKE, etc, should set this to true. Leaving the field undefined is equivalent to setting the value to false.

Example usage

apiVersion: controlplane.cluster.x-k8s.io/v1beta1
kind: KubeadmControlPlane
metadata:
  name: kcp-1
  namespace: default
spec:
  machineTemplate:
    infrastructureRef:
      apiVersion: infrastructure.cluster.x-k8s.io/v1beta1
      kind: DockerMachineTemplate
      name: docker-machine-template-1
      namespace: default
  replicas: 3
  version: v1.21.2

Kubeconfig management

Control Plane providers are expected to create and maintain a Kubeconfig secret for operators to gain initial access to the cluster. The given secret must be labelled with the key-pair cluster.x-k8s.io/cluster-name=${CLUSTER_NAME} to make it stored and retrievable in the cache used by CAPI managers. If a provider uses client certificates for authentication in these Kubeconfigs, the client certificate should be kept with a reasonably short expiration period and periodically regenerated to keep a valid set of credentials available. As an example, the Kubeadm Control Plane provider uses a year of validity and refreshes the certificate after 6 months.