TKG cluster can be deployed in a many ways. You can use simple YAML file and run it from the Supervisor. You can use some automation tool like Jenkins or create Kubernetes nodes from the TMC.
Apart from that, Terraform has a dedicated Tanzu Mission Control provider (plugin) to automate many processes.
In this article I will show you how to deploy TKGs cluster (v1beta1) with Terraform on TMC (SaaS).
Prerequisites:
- vSphere with Tanzu – in my case it’s a vSphere 8.0 U2a with AVI LB
- vSphere with Tanzu connected with TMC
- Linux/Mac/Windows machine with Terraform installed
1. Generate API Token from the TMC
1. At first, we need to generate an API Token from the TMC. It will be used to connect Terraform to TMC. Log in to your Organization: https://console.cloud.vmware.com/
2. Go to My Account under User Settings.
3. Click Generate Token to move to the token settings.
4. Type a Token Name, Token TTL and Define Scopes. For testing purposes, I selected all available roles. Click Generate to create a token.
5. Token generated! Copy somewhere this data.
2. Terraform installation
Depending on your operating system, you will need to install right version of the Terraform. Check documentation – https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli and choose a appropriate binaries. In this tutorial, I have installed Terraform v1.6.6 on MacOS.
3. Create Terraform configuration files
Before we begin, remember to setup vSphere Namespace with your parameters.
I’ve created a dedicated catalog: tkgs-cluster-terraform-tmc where I will store all the files.
1. Create a first file: tmc-conf.tf
It’s a Terraform provider file. Configuration describes name, version and TMC SasS parameters – endpoint and API token.
!! REMEMBER to use minimum v1.4.1 version of this provider !!
More info about Terraform Tanzu Mission Control provider:
https://registry.terraform.io/providers/vmware/tanzu-mission-control/latest/docs
terraform {
required_providers {
tanzu-mission-control = {
source = "vmware/tanzu-mission-control"
version = "1.4.1"
}
}
}
# Provider configuration for TMC SaaS
provider "tanzu-mission-control" {
endpoint = var.endpoint
vmw_cloud_api_token = var.vmw_cloud_api_token
}
2. Create a second file: terraform.tfvars
Here, you need to paste generated API Token from the 1st step and enter endpoint name. Endpoint is a FQDN of the TMC SaaS infrastructure.
vmw_cloud_api_token = "Generated_API_Token_From_1st_Step"
endpoint = "NAME.tmc.cloud.vmware.com"
3. Create a third file: cluster.tf
This is a configuration of the new TKGs cluster.
Bolded data can be different than yours. Fill this parameters according to your needs.
Default Cluster template can be found here:
https://registry.terraform.io/providers/vmware/tanzu-mission-control/latest/docs/resources/cluster#tanzu-kubernetes-grid-service-workload-cluster
resource "tanzu-mission-control_tanzu_kubernetes_cluster" "tkgs_cluster" {
name = "cl01-terraform-tmc" // name of the TKGs cluster
management_cluster_name = "vmware-vcs-01" // name of the Management Cluster on the TMC
provisioner_name = "ns01" // vSphere namespace
spec {
cluster_group_name = "default"
topology {
version = "v1.26.5+vmware.2-fips.1-tkg.1"
cluster_class = "tanzukubernetescluster"
cluster_variables = jsonencode(local.tkgs_cluster_variables)
control_plane {
replicas = 1
os_image {
name = "photon"
version = "3"
arch = "amd64"
}
}
nodepool {
name = "worker"
description = "vmattroman-workers"
spec {
worker_class = "node-pool"
replicas = 3
overrides = jsonencode(local.tkgs_nodepool_a_overrides)
os_image {
name = "photon"
version = "3"
arch = "amd64"
}
}
}
network {
pod_cidr_blocks = [
"172.20.0.0/16",
]
service_cidr_blocks = [
"10.96.0.0/16",
]
service_domain = "cluster.local"
}
}
}
timeout_policy {
timeout = 60
wait_for_kubeconfig = true
fail_on_timeout = true
}
}
4. Create a fourth file: variables.tf
At the beginning, there are variables regarding to the endpoint and API token.
After that, there are variables regarding to the configuration of the ClusterClass. If you want to deploy TKGs cluster with an API v1beta1 it’s a strong requirement.
Bolded data can be different than yours. Fill this parameters according to your needs. Fill this parameters according to your needs.
Default template can be found here:
https://registry.terraform.io/providers/vmware/tanzu-mission-control/latest/docs/resources/tanzu_kubernetes_cluster#example-usage-1
variable "endpoint" {
description = "endpoint"
type = string
}
variable "vmw_cloud_api_token" {
description = "vmw_cloud_api_token"
type = string
}
// ClusterClass configuration below
locals {
tkgs_cluster_variables = {
"controlPlaneCertificateRotation" : {
"activate" : true,
"daysBefore" : 30
},
"defaultStorageClass" : "tanzu-vmware-ssd",
"defaultVolumeSnapshotClass" : "tanzu-vmware-ssd",
"nodePoolLabels" : [
],
"nodePoolVolumes" : [
{
"capacity" : {
"storage" : "20G"
},
"mountPath" : "/var/lib/containerd",
"name" : "containerd",
"storageClass" : "tanzu-vmware-ssd"
},
{
"capacity" : {
"storage" : "20G"
},
"mountPath" : "/var/lib/kubelet",
"name" : "kubelet",
"storageClass" : "tanzu-vmware-ssd"
}
],
"ntp" : "0.pl.pool.ntp.org",
"storageClass" : "tanzu-vmware-ssd",
"storageClasses" : [
"tanzu-vmware-ssd"
],
"vmClass" : "best-effort-xsmall"
}
tkgs_nodepool_a_overrides = {
"nodePoolLabels" : [
{
"key" : "vmattroman-worker-label",
"value" : "terraform-tmc"
}
],
"storageClass" : "tanzu-vmware-ssd",
"vmClass" : "best-effort-xsmall"
}
}
4. TKGs cluster deployment
Now, when all things are ready, we can move on and create a new cluster!
1. Open shell in the place, where files are created. Type a command terraform init to initialize Terraform provider plugins.
terraform init
2. To check that everything is OK before deployment, run a command terraform plan.
The output generates execution plan and description about “installation” steps.
terraform plan
3. Run a command terraform apply -auto-approve to start deployment.
Now, Terraform starts cluster installation. You can check progress in the shell, vCenter and Tanzu Mission Control.
terraform apply -auto-approve
4. Done! After a few minutes, TKGs cluster was created and attached to the TMC! 🥳
4. Logging in to the TKGs Cluster
Newly created cluster is accessible through the shell and looks healthy! 😀
5. Destroy cluster
To delete and detach TKGs cluster from the vCenter and TMC, type a command: terraform destory -auto-approve
terraform destroy -auto-approve
Wrapping up
And that’s it. We deployed TKGs cluster using Terraform in Tanzu Mission Control.
This is the beginning and basic possibilities offered by Terraform TMC provider.
In the future, I will show more options that can be used when setting up a cluster.