API Information for Tanzu Kubernetes Clusters for VMware Cloud Director

The Container Service Extension 4.0 has been launched with a number of vital enhancements and extra use circumstances, together with Cluster API, lifecycle administration via a person interface, GPU assist for Kubernetes clusters, and integration with VMware Cloud Director as infrastructure. With its feature-rich person interface, clients can carry out operations similar to creation, scaling, and upgrading on Tanzu Kubernetes clusters. Nevertheless, some clients could search automation assist for these similar operations.

This weblog submit is meant for purchasers who want to automate the provisioning of Tanzu Kubernetes clusters on the VMware Cloud Director Tenant portal utilizing the VMware Cloud Director API. Though the VCD API is supported, the weblog submit is critical as a result of the Cluster API is used to create and handle TKG clusters on VCD. The payload required to carry out operations on TKG clusters requires some work to offer the Cluster API-generated payload. The weblog submit outlines the step-by-step course of for producing the proper payload for purchasers utilizing their VCD infrastructure.

Model Help:

This API information is relevant to clusters created by CSE 4.0 and CSE 4.0.1 Tanzu Kubernetes Clusters.

The prevailing stipulations for purchasers to create TKG clusters of their organizations additionally apply to the automation stream. These stipulations are summarized right here and will be discovered within the official documentation to onboard Supplier and Tenant Admin customers. The next sections present an outline of the necessities for each cloud supplier directors and Tenant Admin customers.

Cloud Supplier Admin Steps

The Steps to onboard the shoppers is demonstrated on this video and documented right here. As soon as buyer group and its customers are onboarded, they will use subsequent part to make use of APIs, or eat it to create automated Cluster operations.

As a fast abstract following steps are anticipated to be carried out by cloud supplier to onboard and put together the shopper:

1. Assessment Interoperability Matrix to assist Container Service Extension 4.0 and 4.0.1
2. Enable obligatory communication for CSE server
3. Begin CSE server and Onboard buyer group (Reference Demo and Official Documentation)

Buyer Org Admin Steps

Enter	Instance worth for this weblog
VCD_SITE	VCD Handle (https://vcd-01a.native)
VCD_ORGANIZATION	Buyer Group title(ACME)
VCD_ORGANIZATION_VDC	Buyer OVDC title (ACME_VDC_T)
VCD_ORGANIZATION_VDC_NETWORK	Community title in buyer org (172.16.2.0)
VCD_CATALOG	CSE shared catalog title (cse)

Enter	Instance worth for this weblog
VCD_TEMPLATE_NAME	Kubernetes and TKG model of the cluster(Ubuntu 20.04 and Kubernetes v1.22.9+vmware.1)
VCD_CONTROL_PLANE_SIZING_POLICY	Sizing coverage of management aircraft vms(TKG small)
VCD_CONTROL_PLANE_STORAGE_PROFILE	Storage profile for management aircraft of the cluster (Capability)
VCD_CONTROL_PLANE_PLACEMENT_POLICY	Non-compulsory – Go away empty if not utilizing
VCD_WORKER_SIZING_POLICY	Sizing coverage of employee nodes vms(TKG small)
VCD_WORKER_PLACEMENT_POLICY	Non-compulsory – Go away empty if not utilizing
VCD_WORKER_STORAGE_PROFILE	Storage profile for management aircraft of the cluster (Capability)
CONTROL_PLANE_MACHINE_COUNT	1
WORKER_MACHINE_COUNT	1
VCD_REFRESH_TOKEN_B64	“MHB1d0tXSllVb2twU2tGRjExNllCNGZnVWZqTm5UZ2U=” Ref VMware Doc to Generate token earlier than reworking it to Base64

• Set up required instruments to generate the capiyaml. Consumer can use any Working System or a Digital Machine(together with Linux, Mac or Home windows) to generate the payload.
• As soon as the tenant person has collected all the data, person must set up following elements similar to  Clusterctl 1.1.3, Sort(0.17.0), and Docker (20.10.21) on finish person’s machine. The next step requires above collected data, and never the entry to VCD Infrastructure to generate capiyaml payload.
• Copy TKG CRS Information domestically. Incase the TKG model is lacking from the folder, ensure you have the templates created for the specified TKG variations. The Following desk gives supported listing of and so forth, coredns, tkg, tkr variations for CSE 40 and CSE 4.0.1 launch. Alternatively this script to fetch the identical values from Tanzu Kubernetes Grid assets.

Kubernetes Model	Etcd ImageTag	CoreDNS ImageTag	Full Distinctive Model	OVA	TKG Product Model	TKr model
v1.22.9+vmware.1	v3.5.4_vmware.2	v1.8.4_vmware.9	v1.22.9+vmware.1-tkg.1	ubuntu-2004-kube-v1.22.9+vmware.1-tkg.1-2182cbabee08edf480ee9bc5866d6933.ova	1.5.4	v1.22.9—vmware.1-tkg.1
v1.21.11+vmware.1	v3.4.13_vmware.27	v1.8.0_vmware.13	v1.21.11+vmware.1-tkg.2	ubuntu-2004-kube-v1.21.11+vmware.1-tkg.2-d788dbbb335710c0a0d1a28670057896.ova	1.5.4	v1.21.11—vmware.1-tkg.3
v1.20.15+vmware.1	v3.4.13_vmware.23	v1.7.0_vmware.15	v1.20.15+vmware.1-tkg.2	ubuntu-2004-kube-v1.20.15+vmware.1-tkg.2-839faf7d1fa7fa356be22b72170ce1a8.ova	1.5.4	v1.20.15—vmware.1-tkg.2

<br /> mkdir ~/infrastructure-vcd/<br /> cd ~/infrastructure-vcd<br /> mkdir v1.0.0<br /> cd v1.0.0

mkdir ~/infrastructure–vcd/   cd ~/infrastructure–vcd mkdir v1.0.0 cd v1.0.0

<br /> v1.0.0% ls -lrta<br /> whole 280<br /> drwxr-xr-x 3 bhatts employees 96 Jan 30 16:41 ..<br /> drwxr-xr-x 6 bhatts employees 192 Jan 30 16:42 crs<br /> -rw-r–r– 1 bhatts employees 9073 Jan 30 16:56 cluster-template-v1.20.8-crs.yaml<br /> -rw-r–r– 1 bhatts employees 9099 Jan 30 16:56 cluster-template-v1.20.8.yaml<br /> -rw-r–r– 1 bhatts employees 9085 Jan 30 16:57 cluster-template-v1.21.8-crs.yaml<br /> -rw-r–r– 1 bhatts employees 9023 Jan 30 16:57 cluster-template-v1.21.8.yaml<br /> -rw-r–r– 1 bhatts employees 9081 Jan 30 16:57 cluster-template-v1.22.9-crs.yaml<br /> -rw-r–r– 1 bhatts employees 9019 Jan 30 16:57 cluster-template-v1.22.9.yaml<br /> -rw-r–r– 1 bhatts employees 9469 Jan 30 16:57 cluster-template.yaml<br /> -rw-r–r– 1 bhatts employees 45546 Jan 30 16:58 infrastructure-components.yaml<br /> -rw-r–r– 1 bhatts employees 165 Jan 30 16:58 metadata.yaml<br /> -rw-r–r– 1 bhatts employees 3355 Jan 30 18:53 clusterctl.yaml<br /> drwxr-xr-x 13 bhatts employees 416 Jan 30 18:53 .</p> <p>crs % ls -lrta<br /> whole 0<br /> drwxr-xr-x 6 bhatts employees 192 Jan 30 16:42 .<br /> drwxr-xr-x 4 bhatts employees 128 Jan 30 16:42 tanzu<br /> drwxr-xr-x 4 bhatts employees 128 Jan 30 16:51 cni<br /> drwxr-xr-x 4 bhatts employees 128 Jan 30 16:54 cpi<br /> drwxr-xr-x 6 bhatts employees 192 Jan 30 16:55 csi<br /> drwxr-xr-x 13 bhatts employees 416 Jan 30 18:53 ..

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

v1.0.0% ls –lrta whole 280 drwxr–xr–x   3 bhatts  employees     96 Jan 30 16:41 .. drwxr–xr–x   6 bhatts  employees    192 Jan 30 16:42 crs –rw–r—r—   1 bhatts  employees   9073 Jan 30 16:56 cluster–template–v1.20.8–crs.yaml –rw–r—r—   1 bhatts  employees   9099 Jan 30 16:56 cluster–template–v1.20.8.yaml –rw–r—r—   1 bhatts  employees   9085 Jan 30 16:57 cluster–template–v1.21.8–crs.yaml –rw–r—r—   1 bhatts  employees   9023 Jan 30 16:57 cluster–template–v1.21.8.yaml –rw–r—r—   1 bhatts  employees   9081 Jan 30 16:57 cluster–template–v1.22.9–crs.yaml –rw–r—r—   1 bhatts  employees   9019 Jan 30 16:57 cluster–template–v1.22.9.yaml –rw–r—r—   1 bhatts  employees   9469 Jan 30 16:57 cluster–template.yaml –rw–r—r—   1 bhatts  employees  45546 Jan 30 16:58 infrastructure–elements.yaml –rw–r—r—   1 bhatts  employees    165 Jan 30 16:58 metadata.yaml –rw–r—r—   1 bhatts  employees   3355 Jan 30 18:53 clusterctl.yaml drwxr–xr–x  13 bhatts  employees    416 Jan 30 18:53 .   crs % ls –lrta whole 0 drwxr–xr–x   6 bhatts  employees  192 Jan 30 16:42 . drwxr–xr–x   4 bhatts  employees  128 Jan 30 16:42 tanzu drwxr–xr–x   4 bhatts  employees  128 Jan 30 16:51 cni drwxr–xr–x   4 bhatts  employees  128 Jan 30 16:54 cpi drwxr–xr–x   6 bhatts  employees  192 Jan 30 16:55 csi drwxr–xr–x  13 bhatts  employees  416 Jan 30 18:53 ..

• Copy the ~/infrastructure-vcd/v1.0.0/clusterctl.yaml to ~/.cluster-api/clusterctl.yaml.
• The ‘clusterctl‘ command makes use of clusterctl.yaml from ~/.cluster-api/clusterctl.yaml to create the capiyaml payload. Replace the infrastructure particulars from step one on this doc.
• Replace the suppliers.url in ~/.cluster-api/clusterctl.yaml to ~/infrastructure-vcd/v1.0.0/infrastructure-components.yaml.

<br /> suppliers:<br /> – title: “vcd”<br /> url: “~/infrastructure-vcd/v1.0.0/infrastructure-components.yaml”<br /> kind: “InfrastructureProvider”

suppliers:   – title: “vcd”     url: “~/infrastructure-vcd/v1.0.0/infrastructure-components.yaml”     kind: “InfrastructureProvider”

• At this level your ~/cluster-api/clusterctl.yaml values ought to look as follows

At this level, we are going to want a sort cluster to put in clusterctl to generate the payload. On this step, create Sort cluster to generate capiyaml payload and initialize clusterctl as follows:

<br /> cat > kind-cluster-with-extramounts.yaml &lt;&lt;EOF<br /> variety: Cluster<br /> apiVersion: variety.x-k8s.io/v1alpha4<br /> nodes:<br /> – function: control-plane<br /> extraMounts:<br /> – hostPath: /var/run/docker.sock<br /> containerPath: /var/run/docker.sock<br /> EOF</p> <p>Create a neighborhood cluster on mac // This may be equally executed on alternative of your working system.</p> <p>variety create cluster –config kind-cluster-with-extramounts.yaml<br /> kubectl cluster-info –context kind-kind<br /> kubectl config set-context kind-kind<br /> kubectl get po -A -owide<br /> clusterctl init –core cluster-api:v1.1.3 -b kubeadm:v1.1.3 -c kubeadm:v1.1.3 vcd:v1.0.0

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17

cat > variety–cluster–with–extramounts.yaml <<EOF variety: Cluster apiVersion: variety.x–k8s.io/v1alpha4 nodes: – function: management–aircraft   extraMounts:     – hostPath: /var/run/docker.sock       containerPath: /var/run/docker.sock EOF   Create a native cluster on mac // This may be equally executed on alternative of your working system.   variety create cluster —config variety–cluster–with–extramounts.yaml kubectl cluster–data —context variety–variety kubectl config set–context variety–variety kubectl get po –A –owide clusterctl init —core cluster–api:v1.1.3 –b kubeadm:v1.1.3 –c kubeadm:v1.1.3 vcd:v1.0.0

Replace the beneath tkg labels to “Sort: Cluster” object and annotations.

<br /> OLD Metadata: </p> <p>apiVersion: cluster.x-k8s.io/v1beta1<br /> variety: Cluster<br /> metadata:<br /> labels:<br /> ccm: exterior<br /> cni: antrea<br /> csi: exterior<br /> title: api5<br /> namespace: default</p> <p>New Metadata: </p> <p>apiVersion: cluster.x-k8s.io/v1beta1<br /> variety: Cluster<br /> metadata:<br /> labels:<br /> cluster-role.tkg.tanzu.vmware.com/administration: “”<br /> tanzuKubernetesRelease: v1.21.8—vmware.1-tkg.2<br /> tkg.tanzu.vmware.com/cluster-name: api5<br /> annotations:<br /> osInfo: ubuntu,20.04,amd64<br /> TKGVERSION: v1.4.3<br /> title: api5<br /> namespace: api5-ns

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

OLD Metadata:   apiVersion: cluster.x–k8s.io/v1beta1 variety: Cluster metadata:   labels:     ccm: exterior     cni: antrea     csi: exterior   title: api5   namespace: default   New Metadata:   apiVersion: cluster.x–k8s.io/v1beta1 variety: Cluster metadata:   labels:     cluster–function.tkg.tanzu.vmware.com/administration: “”     tanzuKubernetesRelease: v1.21.8—vmware.1–tkg.2     tkg.tanzu.vmware.com/cluster–title: api5   annotations:     osInfo: ubuntu,20.04,amd64     TKGVERSION: v1.4.3   title: api5   namespace: api5–ns

• At this level, the capiyaml is able to be consumed by VCD APIs to carry out varied operations. For verification, make certain cluster title, namespace values are constant. Copy the content material of capiyaml to generate jsonstring utilizing related instrument as right here.

Following part describes all supported API operations for Tanzu Kubernetes Cluster on VMware Cloud Director:

Record Clusters

Record all clusters within the buyer group. for CSE 4.0 launch the CAPVCD model is 1.

Data Cluster

Filter Cluster by title

Get cluster by ID:

Get Kubeconfig of the cluster:

The Kubeconfig will be discovered as follows at: entity.standing.capvcd.non-public.kubeconfig

Create a brand new Cluster

Resize a Cluster

• Fetch the Cluster ID("id": "urn:vcloud:entity:vmware:capvcdCluster:<ID>) from the above API name’s output.
• Copy the whole output of the API response.
• Notedown eTag Worth from API response header
• Modify “capiyaml” with following values:
  • To resize Management Aircraft VMs Modify kubeadmcontrolplane.spec.replicas with desired variety of management aircraft vms. Word solely odd numbers of management aircraft are supported.
  • To resize Employee Aircraft VMS Modify MachineDeployment.spec.replicas with desired variety of employee aircraft VMs
• Whereas performing the PUT API name, guarantee to incorporate fetched eTag worth as If-Match

Improve a Cluster

To Improve a cluster, Supplier admin must publish desired the Tanzu Kubernetes templates to the shopper group in catalog utilized by Container Service Extension.

accumulate the GET API response for the cluster to be upgraded as follows:

• Fetch the Cluster ID("id": "urn:vcloud:entity:vmware:capvcdCluster:<ID>) from the above API name’s output.
• Copy the whole output of the API response.
• Notedown eTag Worth from API response header
• The client person performing cluster improve would require entry to Desk 3 data. Modify Following values matching the goal TKG model. The Following desk reveals Improve for TKG model 1.5.4 from v1.20.15+vmware.1 to v1.22.9+vmware.1

• Whereas performing the PUT API name, guarantee to incorporate fetched eTag worth as If-Match

Delete a Cluster

• Fetch the Cluster ID("id": "urn:vcloud:entity:vmware:capvcdCluster:<ID>) from the above API name’s output.
• Copy the whole output of the API response.
• Notedown eTag Worth from API response header
• Add or modify the next fields to delete or forcefully delete the cluster beneath entity.spec.vcdke:
  • “markForDelete”: true, –> Set the worth to true to delete the cluster
  • “forceDelete”: true, –> Set this worth to true for Forceful deletion of a cluster

Advice for API Utilization throughout automation

• DO NOT hardcode API urls with RDE variations. ALWAYS parameterize RDE variations. For instance:

POST https://{{vcd}}/cloudapi/1.0.0/entityTypes/urn:vcloud:kind:vmware:capvcdCluster:1.1.0 Guarantee to declare 1.1.0 as a variable. This may guarantee simple API shopper upgrades to future variations of CSE.

• Make sure the API shopper code ignores any unknown/extra properties whereas unmarshaling the API response

1. Generate API token utilizing VMware Cloud Director
2. CSE 4.0 Official Documentation
3. Cluster API for VMware Cloud Director Platform official Documentation