Deploy Cloud Controller Manager

The configurations for using Azure Cloud Controller Manager.

azure-cloud-controller-manager is a Kubernetes component which provides interoperability with Azure API, and will be used by Kubernetes clusters running on Azure. It runs together with other components to provide the Kubernetes cluster’s control plane.

Using cloud-controller-manager is a new alpha feature for Kubernetes since v1.14. cloud-controller-manager runs cloud provider related controller loops, which used to be run by controller-manager.

azure-cloud-controller-manager is a specialization of cloud-controller-manager. It depends on cloud-controller-manager app and azure cloud provider.

Deployment

To deploy Azure cloud controller manager, the following components need to be configured.

kubelet

FlagValueRemark
--cloud-providerexternalcloud-provider should be set external
--azure-container-registry-config/etc/kubernetes/azure.jsonUsed for Azure credential provider

kube-controller-manager

FlagValueRemark
--cloud-providerexternalcloud-provider should be set external
--external-cloud-volume-pluginazureOptional*

*: Since cloud controller manager does not support volume controllers, it will not provide volume capabilities compared to using previous built-in cloud provider case. You can add this flag to turn on volume controller for in-tree cloud providers. This option is likely to be removed with in-tree cloud providers in future.

kube-apiserver

Do not set flag --cloud-provider.

azure-cloud-controller-manager

FlagValueRemark
--cloud-providerazurecloud-provider should be set azure
--cloud-config/etc/kubernetes/azure.jsonPath for cloud provider config
--controllers*,-cloud-nodecloud node controller should be disabled
--configure-cloud-routes“false” for Azure CNI and “true” for other network pluginsUsed for non-AzureCNI clusters

For other flags such as --allocate-node-cidrs, --cluster-cidr and --cluster-name, they are moved from kube-controller-manager. If you are migrating from kube-controller-manager, they should be set to same value.

For details of those flags, please refer to this doc.

azure-cloud-node-manager

azure-cloud-node-manager should be run as daemonsets on both Windows and Linux nodes, and the following configurations should be set:

FlagValueRemark
--node-nameThe node name for the PodKubernetes Downward API could be used to get Pod’s name
--wait-routesonly set to true when --configure-cloud-routes=true in cloud-controller-managerUsed for non-AzureCNI clusters

Please refer examples here for sample deployment manifests for above components.

Alternatively, you can use aks-engine to deploy a Kubernetes cluster running with cloud-controller-manager. It supports deploying Kubernetes azure-cloud-controller-manager for Kubernetes v1.16+.

AzureDisk and AzureFile

AzureDisk and AzureFile volume plugins are not supported with in-tree cloud provider (See kubernetes/kubernetes#71018 for explanations).

Hence, azuredisk-csi-driver and azurefile-csi-driver should be used for persistent volumes. Please refer the installation guides here and here for their deployments.

Change default storage class

Follow the steps bellow if you want change the current default storage class to AzureDisk CSI driver.

First, delete the default storage class:

kubectl delete storageclass default

Then create a new storage class named default:

cat <<EOF | kubectl apply -f-
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  annotations:
    storageclass.beta.kubernetes.io/is-default-class: "true"
  name: default
provisioner: disk.csi.azure.com
parameters:
  skuname: Standard_LRS # available values: Standard_LRS, Premium_LRS, StandardSSD_LRS and UltraSSD_LRS
  kind: managed         # value "dedicated", "shared" are deprecated since it's using unmanaged disk
  cachingMode: ReadOnly
reclaimPolicy: Delete
volumeBindingMode: Immediate
EOF

Last modified July 26, 2021 : docs: fix broken links (6f8ebbd0)