Kubernetes cloud controller managerPreview

ankra-cloud-ccm is the cloud controller manager for Kubernetes clusters that run on Ankra Cloud servers. It uses the provider name ankracloud and talks only to the public API, with an API token. It does two jobs:

• Nodes: it links every node to its server, sets the node's addresses, instance type, zone and region, and notices when a server stops or is deleted.
• Services of type LoadBalancer: it gives each one an Ankra load balancer, or serves it from a combined network edge.

It does not program routes; your CNI routes pod traffic between nodes.

Before you install#

Every kubelet must run with --cloud-provider=external. A node started that way carries the taint node.cloudprovider.kubernetes.io/uninitialized until the controller has initialised it, so workloads wait for real addresses and topology labels. On k3s, start the servers with --disable-cloud-controller and every node with --kubelet-arg=cloud-provider=external.

Create an API token with operate permission (Settings → API tokens).

Install with Helm#

The chart is in the Ankra Helm repository. Three commands install it:

helm repo add ankra https://ankraio.github.io/ankra-charts
helm repo update
helm install ankra-cloud-ccm ankra/ankra-cloud-ccm -n kube-system --set api.token=<token>

For production, keep the token in a Secret you manage instead of in Helm values, name the cluster and put load balancers on your private network:

kubectl -n kube-system create secret generic ankra-cloud-ccm --from-literal=token="$ANKRA_CLOUD_TOKEN"
helm install ankra-cloud-ccm ankra/ankra-cloud-ccm -n kube-system \
  --set api.existingSecret=ankra-cloud-ccm \
  --set clusterName=production \
  --set loadBalancer.networkID=<network-id>

The same chart is published as an OCI artifact, oci://share.ankra.cloud/charts/ankra-cloud-ccm. The controller image is share.ankra.cloud/library/ankra-cloud-ccm (linux/amd64 and linux/arm64), which pulls without credentials. The source, issues and releases are at github.com/ankraio/ankra-cloud-ccm, under the Apache 2.0 licence.

The chart installs:

• a Deployment with leader election;
• the cloud controller manager RBAC;
• tolerations for the uninitialized and control-plane taints;
• host networking, because the controller has to run before any CNI is ready.

Without Helm, create the Secret above and apply the chart rendered for kube-system:

kubectl apply -f https://raw.githubusercontent.com/ankraio/ankra-cloud-ccm/main/deploy/ankra-cloud-ccm.yaml

Nodes#

The controller finds a node's server by its providerID, ankracloud://<zone>/<server-id>. A node that has no providerID yet is looked up by hostname, and the controller then sets the providerID.

A stopped or stopping server marks its node as shut down. A deleted server removes its node.

Services of type LoadBalancer#

Each Service gets one load balancer named k8s-<cluster>-<namespace>-<name> and labelled ccm.ankra.cloud/service=<namespace>/<name>. The controller finds it again by that label, so a restart never creates a second one.

• Every TCP port of the Service becomes a frontend on the same port, backed by the nodes' NodePort.
• Members are the nodes' addresses in the Service's IP families (spec.ipFamilies), IPv6 first. On a private network the members are the nodes' private addresses; without one, they are their public IPv6 addresses.
• When nodes join or leave, each backend's member list is replaced in one call.
• Deleting the Service deletes the load balancer.

The Service's status.loadBalancer.ingress lists the load balancer's IPv6 address, and its IPv4 address when it has one:

apiVersion: v1
kind: Service
metadata:
  name: web
  annotations:
    load-balancer.ankra.cloud/ipv4: "true"
spec:
  type: LoadBalancer
  ipFamilyPolicy: PreferDualStack
  ipFamilies: [IPv6, IPv4]
  selector:
    app: web
  ports:
    - name: http
      port: 80
      targetPort: 8080

Annotations#

If you change ipv4 or network-id after the load balancer exists, the controller records an ImmutableSetting event on the Service. To apply the new value, recreate the Service. Ports that are not TCP are skipped with an UnsupportedPort event.

Serving from a combined edge#

With load-balancer.ankra.cloud/edge, the Service's ports are added as members of the edge's load_balancer role. The edge listens on the port over IPv4 and IPv6. This is the cheapest way to reach a cluster from the IPv4 internet, because the edge's single IPv4 address serves the whole network. Members of other Services, and members you added by hand, are left alone. A port that another member already serves is refused.

A zone with one compute node#

A zone that starts on one server cannot place a load balancer pair on two hosts. While the zone reports that stage, the controller creates a single-VM load balancer (high_availability: false) and records a SingleNodeLoadBalancer event on the Service. That load balancer stays single after the zone grows. To get a pair, recreate the Service.