Terraform K8S cluster creation

Planning ahead for deploying our microservices at scale.

In our article on Lifting Kubernetes up to the Cloud, we followed a step-by-step approach to get our Kubernetes clusters running in GCP. This approach is fine when you have a onesy or twosy microservices. But, when you have tens or thousands or tens of thousands we’ll need a better automated approach that scales. That’s where Teraform comes in.

With Teraform we define our infrastructure as code and can deploy into Azure, AWS, Google, Oracle and many other clouds. Teraform allows you to deploy, update and destroy infrastructure solutions without touching a web console. It integrates seamlessly with CICD pipelines. To get started you’ll need to download Teraform for your OS. After you’ve installed and configured Teraform to your path, run teraform with no options to see the available commands. We’ll play with a handful of these when we create our GKE cluster.

Lets get started by logging in to the Google Cloud Platform and creating a new project. Click on Select a project then New Project. For the project name enter terraform and keep the unique number Google provides so that the result looks something like this: terraform-15209. The parent organization can be left blank. Click Create and GCP will create your new project.

Next we’ll need to create a Service Account so that we can interact with GCP from a gcloud shell using Teraform. Service Accounts can be found under the IAM & Admin menu. In our service account we’re going to bind some access permissions. Under service accounts, select Create Service Account, then enter terraform for the service account name and leave the service account id as is. Go ahead and enter a Description and click Create.

Click Role and choose the Editor role, Add another role choose Kubernetes Engine –> Kubernetes Engine Admin. Select Continue and Create a Key, create the JSON key type and it will download to your machine. Be sure to keep this key private, it provides access to your kingdom.

With your secret key downloaded, create a folder for your project and move your key into it. Rename the key to: service-account.json. Create the file below and name it main.tf.

If you don’t already have the Google Cloud SDK (gcloud) installed go ahead and install it now. Open a gcloud shell and change into your project folder. Run the gcloud command to get a list of your projects, you should see the terraform project you created earlier.

# Get a list of your projects
$ gcloud projects list
PROJECT_ID                      NAME                     PROJECT_NUMBER
terraform-15209                 terraform-15209          849873418072

Create the Terraform script below for creating a cluster from the shell. Be sure to replace all instances (3) of my project name terraform-28892 with your from the command above.

Main.tf – Terraform script to create GCP Cluster

provider "google" {
  # rename your saved credentials to: service-account.json
  credentials = "${file("service-account.json")}"
  project = "terraform-28892"
  # change your region and zone to suit your location preference
  region  = "us-east1"
  zone    = "us-east1-c"
}

resource "google_container_cluster" "primary" {
  name        = "my-terraformed-k8s"
  project     = "terraform-28892"
  description = "Demo GKE Cluster"
  location    = "us-east1-c"

  # We can't create a cluster with no node pool defined, but we want to only use
  # separately managed node pools. So we create the smallest possible default
  # node pool and immediately delete it.
  remove_default_node_pool = true
  initial_node_count = "1"

  # Setting an empty username and password explicitly disables basic auth
  master_auth {
    username = ""
    password = ""

    client_certificate_config {
      issue_client_certificate = false
    }
  }
}

resource "google_container_node_pool" "primary_preemptible_node" {
  name       = "default-node-pool"
  project     = "terraform-28892"
  location   = "us-east1-c"
  cluster    = google_container_cluster.primary.name
  node_count = 1

  node_config {
    preemptible  = true
    machine_type = "n1-standard-1"

    metadata = {
      disable-legacy-endpoints = "true"
    }

    oauth_scopes = [
		"https://www.googleapis.com/auth/devstorage.read_only",
		"https://www.googleapis.com/auth/logging.write",
		"https://www.googleapis.com/auth/monitoring",
		"https://www.googleapis.com/auth/servicecontrol",
		"https://www.googleapis.com/auth/service.management.readonly",
		"https://www.googleapis.com/auth/trace.append",
    ]
  }
}

With the main.tf created lets initialize Teraform to pull down the artifacts it needs to build a cluster in the GKE. After we run init, we’ll run terraform plan to get an idea of what Terraform is going to create for us. At this point the command is run locally and won’t interact with the cloud.

# Download Terraform components needed to work with Google CLoud
terraform init
* provider.google: version = "~> 3.12"

Terraform has been successfully initialized!

# Run 
$ terraform plan

If all goes according to plan you’re ready to apply your plan and create a cluster.

# execute the plan
$ terraform apply

You should get an error telling you that the Kubernetes SDK hasn’t been used yet in your new project, which is true, it hasn’t. Follow the link Google provides and the remedy they recommend, then run the apply command again when the Kubernetes Engine is enabled.

It should take several minutes, or so, to create your cluster and node. You should get a message indicating: Apply complete! Resources: 2 added, 0 changed, 0 destroyed.

At this point, you may need to run the kubectl version command. If you get an x509 certificate error, you can reload your cert for the new cluster you created.

$ kubectl version
Unable to connect to the server: x509: certificate signed by unknown authority

# Change you zone if needed and project name using your project name (from above)
$ gcloud container clusters get-credentials my-terraformed-k8s \
    --zone us-east1-c --project terraform-28892

Fetching cluster endpoint and auth data.
kubeconfig entry generated for my-terraformed-k8s.

# rerun kubectl 
$ kubectl version

At this point you can go back to the article Lifting Kubernetes up to the Cloud, picking up at section: Deploy our basic service, if you would like to deploy our test dummy container, create gateway and scale it up.

When you’re done playing don’t forget to run the cleanup so the billing stops. To delete your cluster you can run terraform destroy.

$ terraform destroy

# after several minutes or a tall pint
Destroy complete! Resources: 2 destroyed.

I hope you’ve gotten a good feel for the power and capabilities of Terraform for creating infrastructure as code. the CLI API guide has an extensive rich set of commands you can apply across a number of different clouds helping make you a better devsecops profesional.