Comet Infrastructure Deployment: Terraform on GCP

Terraform module for deploying infrastructure components to run CometML.

Deployment

Prerequisites:

• Git installed
• gcloud CLI
  • Install
  • Configure
• Terraform installed

Infrastructure Deployment:

• Follow the steps below to deploy directly from the GitHub repository.
  • Clone the repository to your local machine: git clone https://github.com/comet-ml/terraform-gcp-comet.git
  • Move into the deployment directory: cd terraform-gcp-comet
  • Initialize the directory: terraform init
  • Within terraform.tfvars, set your module toggles to enable the desired infrastructure components and set any required environment variables
  • Provision the resources: terraform apply

A note on state management:

• This configuration stores the Terraform state locally by default. To store the state file remotely in GCS, a backend block can be nested within the terraform block inside versions.tf if applying directly from this configuration, or within your terraform block if calling the module. Below is an example of such a configuration:

terraform {
  backend "gcs" {
    bucket  = "tf-state-prod"
    prefix  = "terraform/state"
  }
}

• More on state management in GCS can be found here

Requirements

Name	Version
terraform	>= 1.0
google	~> 4.51
kubernetes	~> 2.10
null	~> 3.1
random	~> 3.0

Providers

Name	Version
google	4.74.0

Modules

Name	Source	Version
comet_gke	./modules/comet_gke	n/a
comet_lb	./modules/comet_lb	n/a
comet_memorystore	./modules/comet_memorystore	n/a
comet_mysql	./modules/comet_mysql	n/a
comet_s3	./modules/comet_s3	n/a
comet_vm	./modules/comet_vm	n/a
comet_vpc	./modules/comet_vpc	n/a

Resources

Name	Type
google_client_config.default	data source
google_compute_regions.available	data source

Inputs

Name	Description	Type	Default	Required
comet_vpc_name	Name for the existing VPC that resources will be provisioned in	string	null	no
comet_vpc_subnet_name	Name for the existing subnet that resources will be provisioned in	string	null	no
comet_vpc_zones	Compute zones to use within the region	list(string)	null	no
db_assign_public_ip	Enables giving the master instance a public IP address	bool	true	no
db_deletion_protection	Used to block Terraform from deleting a SQL Instance	bool	false	no
db_instance_tier	The tier for the master instance	string	"db-n1-standard-16"	no
db_password	Password for database user account	string	null	no
db_random_instance_name	Sets random suffix at the end of the Cloud SQL resource name	bool	true	no
db_user	Name for database user account	string	"cometml"	no
db_version	The database version to use	string	"MYSQL_8_0"	no
enable_gke	Toggles the comet_gke module, to provision GKE resources for running Comet	bool	n/a	yes
enable_lb	Toggles the comet_lb module, to provision an application load balancer for the Comet VM	bool	n/a	yes
enable_memorystore	Toggles the comet_memorystore module, to provision a Redis instance for Comet	bool	n/a	yes
enable_mysql	Toggles the comet_mysql module, to provision Google Cloud SQL resources for Comet	bool	n/a	yes
enable_s3	Toggles the comet_s3 module, to provision a Google Cloud Storage bucket for Comet S3	bool	n/a	yes
enable_vm	Toggles the comet_vm module, to provision a Google Compute Engine VM for Comet	bool	n/a	yes
enable_vpc	Toggles the comet_vpc module, to provision a new VPC for hosting the Comet resources	bool	n/a	yes
environment	Name for the Comet environment, for use in resource naming	string	"prod"	no
gke_create_service_account	Enables creation of a GKE cluster service account	bool	true	no
gke_enable_cost_allocation	Enables cost allocation tracking in the GKE cluster	bool	false	no
gke_enable_vertical_pod_autoscaling	Enables vertical pod autoscaling in the GKE cluster	bool	false	no
gke_filestore_csi_driver	Enables the Filestore CSI driver in the GKE cluster	bool	false	no
gke_horizontal_pod_autoscaling	Enables horizontal pod autoscaling in the GKE cluster	bool	true	no
gke_http_load_balancing	Enables the load balancer controller in the GKE cluster	bool	true	no
gke_kubernetes_version	Kubernetes version to use in the GKE cluster	string	"latest"	no
gke_network_policy	Enables network policy usage in the GKE cluster	bool	false	no
gke_network_policy_provider	Network policy provider to use in the GKE cluster	string	"CALICO"	no
gke_nodepool_auto_repair	Enables GKE nodepool autorepair	bool	true	no
gke_nodepool_auto_upgrade	Enables GKE nodepool autoupgrade	bool	true	no
gke_nodepool_disk_size_gb	Disk size for GKE nodes	number	100	no
gke_nodepool_disk_type	Disk type for GKE nodes	string	"pd-standard"	no
gke_nodepool_enable_gcfs	Enables GCFS on GKE nodepool	bool	false	no
gke_nodepool_enable_gvnic	Enables GVNIC on GKE nodepool	bool	false	no
gke_nodepool_image_type	OS image type for GKE nodes	string	"COS_CONTAINERD"	no
gke_nodepool_local_ssd_count	Number of local SSDs to provision with GKE nodes	number	0	no
gke_nodepool_machine_type	VM machine type for the GKE nodes	string	"n1-standard-8"	no
gke_nodepool_max_count	Maximum count of nodes in the GKE nodepool	number	3	no
gke_nodepool_min_count	Minimum and target count of nodes in the GKE nodepool	number	1	no
gke_nodepool_preemptible	Sets GKE nodes as preemptible	bool	false	no
gke_nodepool_spot	Enables using spot VMs for GKE nodes	bool	false	no
gke_pods_cidr	Secondary network CIDR block for the VPC, to be used for GKE pods	string	"10.1.0.0/16"	no
gke_pods_cidr_name	Name of the secondary CIDR block to be used for GKE pods	string	null	no
gke_regional	Enables provisioning the GKE cluster with the regional architecture	bool	true	no
gke_release_channel	Sets the release channel for the cluster	string	"STABLE"	no
gke_remove_default_node_pool	Sets the cluster default GKE nodepool to be removed when the cluster is created	bool	false	no
gke_services_cidr	Secondary network CIDR block for the VPC, to be used for GKE services	string	"10.2.0.0/20"	no
gke_services_cidr_name	Name of the secondary CIDR block to be used for GKE services	string	null	no
lb_backend_enable_cdn	Enables using CDN load balancer	bool	false	no
lb_backend_instance_group	Compute managed instance group to serve as backend for the load balancer	string	null	no
lb_backend_service_logging	Enables logging for the load balancer requests to backend	bool	false	no
lb_backend_timeout_sec	Timeout in seconds for backend connections	number	10	no
lb_cloudnat_router_name	Name of the router used for Cloud NAT	string	null	no
lb_enable_iap_config	Enables using identity-aware proxy with the load balancer	bool	false	no
lb_enable_ssl	Enables SSL support on the load balancer	bool	false	no
lb_health_check_logging	Enables logging for the load balancer health checks	bool	false	no
lb_logging_sample_rate	Sample rate for the backend logging	number	1	no
lb_managed_ssl_certificate_domains	Google-managed SSL certificates for specified domains. Requires lb_enable_ssl to be set to true and lb_use_provided_ssl_certificates set to false	list(string)	[]	no
lb_provided_ssl_certificates	List of existing SSL certificates to use for the load balancer	list(string)	[]	no
lb_use_provided_ssl_certificates	If true, use the certificates provided from lb_provided_ssl_certificates	bool	false	no
memorystore_instance_tier	Memorystore instance tier	string	"BASIC"	no
memorystore_maintenance_day	Day to allow maintenance window for Memorystore instance	string	"SATURDAY"	no
memorystore_maintenance_start_hour	Starting hour for Memorystore maintenance window	number	0	no
memorystore_maintenance_start_min	Starting minute for Memorystore maintenance window	number	30	no
memorystore_memory_size_gb	Amount of memory for the Memorystore instance	number	32	no
memorystore_redis_version	Redis version for the Memorystore instance	string	"REDIS_6_X"	no
project_id	ID of the project that resources will be provisioned in	string	n/a	yes
region	Region that resources will be launched in	string	"us-east1"	no
s3_existing_bucket_name	Name of an existing storage bucket to use with Comet; only used with enable_s3 set to false	string	"no-bucket"	no
s3_force_destroy	Setting to allow storage bucket to be deleted while still holding objects	bool	false	no
s3_storage_class	Storage bucket class	string	"STANDARD"	no
s3_uniform_bucket_level_access	Enables uniform bucket level access	bool	true	no
s3_versioning	Enables versioning for objects in the storage bucket	bool	false	no
vm_disk_auto_delete	Enables the storage disk on the VM to be automatically deleted on VM deletion	bool	false	no
vm_disk_size_gb	Size in GB for storage disk	string	"1000"	no
vm_disk_type	Storage disk type for the VM	string	"pd-standard"	no
vm_enable_ssh	Set to allow SSH traffic to the VM	bool	true	no
vm_machine_type	Machine type for the VM	string	"n1-standard-16"	no
vm_source_image_family	Operating system family for the VM	string	"ubuntu-2204-lts"	no
vpc_subnet_cidr	Network CIDR block for the provisioned VPC subnet	string	"10.0.0.0/16"	no
vpc_subnet_flow_logs	Enables flow logs for the provisioned VPC	bool	false	no
vpc_subnet_private_access	Enables resources in VPC without public IPs to access Google APIs and services	bool	true	no

Outputs

Name	Description
comet_gke_cluster	Name of the GKE cluster
comet_gke_configure_kubectl	Configure kubectl: run the following command to update your kubeconfig with the newly provisioned cluster
comet_gke_nodepools	Names of the nodepools in the GKE cluster
comet_gke_sa_hmac_access_id	Access ID of the HMAC key created for the cluster service account
comet_gke_sa_hmac_secret	Secret of the HMAC key created for the cluster service account
comet_gke_service_account	Service account used by the GKE nodes
comet_lb_external_ip	External IP address for the VM load balancer
comet_memorystore_host	Hostname or IP address of the exposed Redis endpoint used by clients to connect to the service
comet_memorystore_name	ID of the Redis instance
comet_memorystore_port	The port number of the exposed Redis endpoint
comet_mysql_connection_name	The connection name of the MySQL instance to be used in connection strings
comet_s3_storage_bucket_name	Name of the storage bucket
comet_s3_storage_bucket_url	Base URL of the storage bucket
comet_vm_instance_group	URL of the managed instance group created for the Comet VM
comet_vm_instance_sa_hmac_access_id	Access ID of the HMAC key created for the instance service account
comet_vm_instance_sa_hmac_secret	Secret of the HMAC key created for the instance service account
comet_vm_instance_service_account	Service account used by the VM
comet_vm_instance_template	Name of the VM instance template created for the Comet VM
comet_vpc_name	Name of the VPC created for Comet
comet_vpc_subnet_name	Name of the subnet created in the Comet VPC