Skip to content

Latest commit

 

History

History
474 lines (394 loc) · 77.5 KB

README.md

File metadata and controls

474 lines (394 loc) · 77.5 KB

terraform-rds

Terraform modules to manage RDS resources

rds

Creates a RDS instance, security_group, subnet_group and parameter_group

Requirements

Name Version
terraform >= 0.12

Providers

Name Version
aws n/a

Modules

No modules.

Resources

Name Type
aws_db_instance.rds resource
aws_db_parameter_group.rds resource
aws_db_subnet_group.rds resource
aws_security_group.sg_rds resource
aws_security_group_rule.rds_cidr_in resource
aws_security_group_rule.rds_sg_in resource

Inputs

Name Description Type Default Required
rds_password RDS root password any n/a yes
security_groups Security groups that are allowed to access the RDS list(string) n/a yes
security_groups_count Number of security groups provided in security_groups variable any n/a yes
subnets Subnets to deploy in list(string) n/a yes
vpc_id ID of the VPC where to deploy in any n/a yes
allowed_cidr_blocks CIDR blocks that are allowed to access the RDS list(string) [] no
apply_immediately Apply changes immediately bool true no
auto_minor_version_upgrade Indicates that minor engine upgrades will be applied automatically to the DB instance during the maintenance window. bool true no
availability_zone The availability zone where you want to launch your instance in string "" no
backup_retention_period How long do you want to keep RDS backups string "14" no
default_parameter_group_family Parameter group family for the default parameter group, according to the chosen engine and engine version. Defaults to mysql5.7 string "mysql5.7" no
deletion_protection If the DB instance should have deletion protection enabled. The database can't be deleted when this value is set to true bool false no
enabled_cloudwatch_logs_exports List of log types to enable for exporting to CloudWatch logs. You can check the available log types per engine in the AWS RDS documentation. list(string) [] no
engine RDS engine: mysql, oracle, postgres. Defaults to mysql string "mysql" no
engine_version Engine version to use, according to the chosen engine. You can check the available engine versions using the AWS CLI. Defaults to 5.7.17 for MySQL. string "5.7.25" no
environment How do you want to call your environment, this is helpful if you have more than 1 VPC. string "production" no
extra_tags A mapping of extra tags to assign to the resource map(string) {} no
maintenance_window The window to perform maintenance in. Syntax: "ddd:hh24:mi-ddd:hh24:mi". Eg: "Mon:00:00-Mon:03:00" string null no
max_allocated_storage When configured, the upper limit to which Amazon RDS can automatically scale the storage of the DB instance. Configuring this will automatically ignore differences to allocated_storage. Must be greater than or equal to allocated_storage or 0 to disable Storage Autoscaling. string "0" no
monitoring_interval The interval, in seconds, between points when Enhanced Monitoring metrics are collected for the DB instance. To disable collecting Enhanced Monitoring metrics, specify 0. Valid Values: 0, 1, 5, 10, 15, 30, 60. string "0" no
multi_az Multi AZ true or false bool true no
name The name of the RDS instance string "" no
number number of the database default 01 string "01" no
performance_insights_enabled Specifies whether Performance Insights is enabled or not. bool false no
performance_insights_kms_key_id Custom KMS key to use to encrypt the performance insights data string null no
performance_insights_retention_period Amount of time in days to retain Performance Insights data. Valid values are 7, 731 (2 years) or a multiple of 31. When specifying performance_insights_retention_period number 7 no
project The current project string "" no
publicly_accessible Bool to control if instance is publicly accessible bool false no
rds_custom_parameter_group_name A custom parameter group name to attach to the RDS instance. If not provided a default one will be used string "" no
rds_username RDS root user string "root" no
size Instance size string "db.t2.small" no
skip_final_snapshot Skip final snapshot when destroying RDS bool false no
snapshot_identifier Specifies whether or not to create this database from a snapshot. This correlates to the snapshot ID you'd find in the RDS console, e.g: rds:production-2015-06-26-06-05. string "" no
storage How many GBs of space does your database need? string "10" no
storage_encrypted Encrypt RDS storage bool true no
storage_kms_key_id Custom KMS key to use to encrypt the storage. Will use the AWS key if left null (default) string null no
storage_type Type of storage you want to use string "gp2" no
subnet_group_name_override Override the name of the created subnet group string null no
tag A tag used to identify an RDS in a project that has more than one RDS string "" no

Outputs

Name Description
aws_db_subnet_group_id The subnet group id of the RDS instance
rds_address The hostname of the RDS instance
rds_arn The arn of the RDS instance
rds_id The id of the RDS instance
rds_identifier The identifier of the RDS instance
rds_port The port of the RDS instance
rds_sg_id The security group id of the RDS instance

Example

module "rds" {
  source                          = "github.com/skyscrapers/terraform-rds//rds"
  vpc_id                          = "vpc-e123bc45"
  subnets                         = ["subnet-12345d67", "subnet-12345d68", "subnet-12345d69"]
  project                         = "myproject"
  environment                     = "production"
  size                            = "db.t2.small"
  security_groups                 = ["sg-12be345678905ebf1", "sg-1234567890aef"]
  enabled_cloudwatch_logs_exports = ["audit", "error", "slowquery"]
  security_groups_count           = 2
  rds_password                    = "supersecurepassword"
  multi_az                        = "false"
}

Aurora

Creates a Aurora cluster + instances, security_group, subnet_group and parameter_group

Requirements

Name Version
terraform >= 0.12

Providers

Name Version
aws n/a

Modules

No modules.

Resources

Name Type
aws_db_parameter_group.aurora_mysql resource
aws_db_subnet_group.aurora resource
aws_rds_cluster.aurora resource
aws_rds_cluster_instance.cluster_instances resource
aws_security_group.sg_aurora resource
aws_security_group_rule.sg_aurora_in resource
aws_availability_zones.available data source
aws_subnet.subnet data source

Inputs

Name Description Type Default Required
password RDS root password any n/a yes
security_groups Security groups that are allowed to access the RDS on port 3306 list(string) n/a yes
subnets Subnets to deploy in list(string) n/a yes
amount_of_instances The amount of Aurora instances you need, for HA you need minumum 2 number 1 no
apply_immediately Apply changes immediately bool true no
backup_retention_period How long do you want to keep RDS backups string "14" no
cluster_parameter_group_name Optional parameter group you can set for the RDS Aurora cluster string "" no
default_ports n/a map 
{
  "aurora": "3306",
  "aurora-postgresql": "5432"
}
 no
enabled_cloudwatch_logs_exports List of log types to enable for exporting to CloudWatch logs. You can check the available log types per engine in the AWS Aurora documentation. list(string) [] no
engine Optional parameter to set the Aurora engine string "aurora" no
engine_version Optional parameter to set the Aurora engine version string "5.6.10a" no
environment How do you want to call your environment, this is helpful if you have more than 1 VPC. string "production" no
extra_tags A mapping of extra tags to assign to the resource map(string) {} no
family n/a string "aurora5.6" no
instance_parameter_group_name Optional parameter group you can set for the RDS instances inside an Aurora cluster string "" no
instance_promotion_tiers Set promotion tier for each instance in the cluster. The size of the list must be equal to var.amount_of_instances. If ommitted or set to [], the default of 0 will be used. list(number) [] no
instance_size_override Provide different instance sizes for each individual aurora instance in the cluster. The size of the list must be equal to var.amount_of_instances. If ommitted or set to [], this module will use var.size for all the instances in the cluster. list(string) [] no
performance_insights_enabled Specifies whether Performance Insights is enabled or not. bool false no
project The current project string "" no
rds_instance_name_overrides List of names to override the default RDS instance names / identifiers. list(string) null no
rds_username RDS root user string "root" no
size Instance size string "db.t2.small" no
skip_final_snapshot Skip final snapshot when destroying RDS bool false no
snapshot_identifier Specifies whether or not to create this cluster from a snapshot. You can use either the name or ARN when specifying a DB cluster snapshot, or the ARN when specifying a DB snapshot string null no
storage_encrypted Encrypt RDS storage bool true no
tag A tag used to identify an RDS in a project that has more than one RDS string "" no

Outputs

Name Description
aurora_cluster_id n/a
aurora_cluster_instances_id n/a
aurora_port n/a
aurora_sg_id n/a
endpoint n/a
reader_endpoint n/a

Example

module "aurora" {
  source                          = "github.com/skyscrapers/terraform-rds//aurora"
  project                         = "myproject"
  environment                     = "production"
  size                            = "db.t2.small"
  password                        = "supersecurepassword"
  subnets                         = ["subnet-12345d67", "subnet-12345d68", "subnet-12345d69"]
  amount_of_instances             = 1
  security_groups                 = ["sg-12be345678905ebf1", "sg-1234567890aef"]
  enabled_cloudwatch_logs_exports = ["audit", "error", "slowquery"]
}

rds-replica

Creates an RDS read replica instance, the replica security_group and a subnet_group if not passed as parameter

Inputs

Name Description Type Default Required
engine string n/a yes
subnets Subnets to deploy in list(string) n/a yes
vpc_id ID of the VPC where to deploy in string n/a yes
allowed_cidr_blocks CIDR blocks that are allowed to access the RDS list(string) [] no
auto_minor_version_upgrade Indicates that minor engine upgrades will be applied automatically to the DB instance during the maintenance window. string "true" no
backup_retention_period How long do you want to keep RDS Slave backups number "14" no
custom_parameter_group_name A custom parameter group name to attach to the RDS instance. If not provided it will use the default from the master instance string "" no
enabled_cloudwatch_logs_exports List of log types to enable for exporting to CloudWatch logs. You can check the available log types per engine in the [AWS RDS documentation](https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/USER\_LogAccess.html#USER\_LogAccess.Procedural.UploadtoCloudWatch\). list(string) [] no
environment How do you want to call your environment, this is helpful if you have more than 1 VPC. string "production" no
name An optional custom name to give to the module's resources string "" no
number_of_replicas number of database repliacs default 1 string "1" no
multi_az Multi AZ true or false bool false no
ports map { "mysql": "3306", "oracle": "1521", "postgres": "5432" } no
project The current project string "" no
replicate_source_db RDS source to replicate from string "" no
publicly_accessible Bool to control if instance is publicly accessible false no
security_groups Security groups that are allowed to access the RDS list(string) [] no
size Instance size string "db.t2.small" no
storage_encrypted Encrypt RDS storage string "true" no
tag A tag used to identify an RDS in a project that has more than one RDS string "" no
max_allocated_storage When configured, the upper limit to which Amazon RDS can automatically scale the storage of the DB instance. Configuring this will automatically ignore differences to allocated_storage. Must be greater than or equal to allocated_storage or 0 to disable Storage Autoscaling. If not set the default of the master instance is set. string null no
allocated_storage How many GBs of space does your database need? If not set the default of the master instance is set. string null no

Outputs

Name Description
rds_address
rds_arn
rds_sg_id

Example

module "rds" {
  source              = "github.com/skyscrapers/terraform-rds//rds-replica"
  project             = "myproject"
  environment         = "production"
  size                = "db.t2.small"
  engine              = "postgres"
  security_groups     = ["sg-12be345678905ebf1", "sg-1234567890aef"]
  replicate_source_db = "arn:aws:rds:eu-west-1:123456789012:db:myproject-production-something-rds01"
  vpc_id              = "vpc-e123bc45"
  subnets             = ["subnet-12345d67", "subnet-12345d68", "subnet-12345d69"]
}

snapshot-cross-account-replicator

This module creates snapshots of RDS instances based on a configured frequency, and replicates them to a different region in a different AWS account. To achieve this it creates several Lambda functions that take care of the copy operations in the different steps.

As an example, let's say we want to back up an RDS instance in AWS account 111111111111 in region eu-west-1 to the AWS account 222222222222 in region eu-central-1. The whole replication process takes place in 4 steps:

  1. A snapshot is created from the RDS instance, in the account 111111111111 in region eu-west-1 . If the instance is KMS encrypted, the snapshot will be encrypted with the same key
  2. The initial snapshot is copied to region eu-central-1 within the source account 111111111111. Snapshots cannot be copied to a different AWS account and region in the same copy operation, so it needs to happen in two steps. In this step, the snapshot is re-encrypted using a KMS key in the target AWS account and region (222222222222 & eu-central-1)
  3. The resulting snapshot from step (2) is then copied over to its final destination, in account 222222222222 in region eu-central-1.

There are Lambda functions in place that will take care of cleaning up the initial and intermediate snapshots resulting from steps (1) and (2).

There's another Lambda function running in account 222222222222 in region eu-central-1 that will periodically run and delete those snapshots that are older than the configured retention period.

For monitoring, the module creates two SNS topics where CloudWatch will post alert messages in case there's problems running the Lambda functions. These SNS topics can be subscribed to upstream monitoring services like OpsGenie.

Take into account that for the copy operation and re-encryption process to work properly, the policy of the provided KMS key in the target account needs to allow usage access to the root user of the source account. IAM policies to further grant access to the Lambda functions will be created within the module. Check this AWS documentation page to know more about how encrpyted snapshots can be shared between different accounts.

Example

data "aws_caller_identity" "source" {
  provider = aws.source
}

module "rds_replication" {
  source = "github.com/skyscrapers/terraform-rds//snapshot-cross-account-replicator?ref=6.1.0"

  name                      = "AuroraReplicator"
  is_aurora_cluster         = true
  rds_instance_ids          = var.rds_cluster_ids
  snapshot_schedule_period  = 12
  retention_period          = 4
  target_account_kms_key_id = aws_kms_key.rds_target.id

  providers = {
    aws.source       = aws.source
    aws.intermediate = aws.intermediate
    aws.target       = aws.target
  }
}

resource "aws_kms_key" "rds_target" {
  provider = aws.target

  description = "KMS key used to encrypt RDS"
  policy      = data.aws_iam_policy_document.rds_replication_key.json

  lifecycle {
    prevent_destroy = true
  }
}

data "aws_iam_policy_document" "rds_replication_key" {
  provider = aws.target

  statement {
    sid       = "Enable IAM policies in source & target accounts"
    effect    = "Allow"
    actions   = ["kms:*"]
    resources = ["*"]

    principals {
      type             = "AWS"
      identifiers = [
        "arn:aws:iam::${data.aws_caller_identity.source.account_id}:root",
        "arn:aws:iam::${var.target_aws_account}:root"
      ]
    }
  }
}

Requirements

Name Version
terraform ~> 1.0
aws ~> 3.61

Providers

Name Version
archive n/a
aws.intermediate ~> 3.61
aws.source ~> 3.61
aws.target ~> 3.61

Modules

Name Source Version
cleanup_intermediate_lambda_monitoring github.com/skyscrapers/terraform-cloudwatch//lambda_function 2.0.1
cleanup_source_lambda_monitoring github.com/skyscrapers/terraform-cloudwatch//lambda_function 2.0.1
cleanup_target_lambda_monitoring github.com/skyscrapers/terraform-cloudwatch//lambda_function 2.0.1
step_1_lambda_monitoring github.com/skyscrapers/terraform-cloudwatch//lambda_function 2.0.1
step_2_lambda_monitoring github.com/skyscrapers/terraform-cloudwatch//lambda_function 2.0.1
step_3_lambda_monitoring github.com/skyscrapers/terraform-cloudwatch//lambda_function 2.0.1

Resources

Name Type
aws_cloudwatch_event_rule.invoke_cleanup_intermediate_lambda resource
aws_cloudwatch_event_rule.invoke_cleanup_source_lambda resource
aws_cloudwatch_event_rule.invoke_cleanup_target_lambda resource
aws_cloudwatch_event_rule.invoke_step_1_lambda resource
aws_cloudwatch_event_rule.invoke_step_2_lambda resource
aws_cloudwatch_event_rule.invoke_step_3_lambda resource
aws_cloudwatch_event_target.invoke_cleanup_intermediate_lambda resource
aws_cloudwatch_event_target.invoke_cleanup_source_lambda resource
aws_cloudwatch_event_target.invoke_cleanup_target_lambda resource
aws_cloudwatch_event_target.invoke_step_1_lambda resource
aws_cloudwatch_event_target.invoke_step_2_lambda resource
aws_cloudwatch_event_target.invoke_step_3_lambda resource
aws_iam_role.source_lambda resource
aws_iam_role.target_lambda resource
aws_iam_role_policy.source_lambda resource
aws_iam_role_policy.source_lambda_kms resource
aws_iam_role_policy.target_lambda resource
aws_iam_role_policy.target_lambda_kms resource
aws_iam_role_policy_attachment.source_lambda_exec_role resource
aws_iam_role_policy_attachment.target_lambda_exec_role resource
aws_lambda_function.cleanup_intermediate resource
aws_lambda_function.cleanup_source resource
aws_lambda_function.cleanup_target resource
aws_lambda_function.step_1 resource
aws_lambda_function.step_2 resource
aws_lambda_function.step_3 resource
aws_lambda_permission.invoke_cleanup_intermediate_lambda resource
aws_lambda_permission.invoke_cleanup_source_lambda resource
aws_lambda_permission.invoke_cleanup_target_lambda resource
aws_lambda_permission.invoke_step_1_lambda resource
aws_lambda_permission.invoke_step_2_lambda resource
aws_lambda_permission.invoke_step_3_lambda resource
aws_sns_topic.source_region_topic resource
aws_sns_topic.target_region_topic resource
aws_sns_topic_policy.source_region_topic resource
aws_sns_topic_policy.target_region_topic resource
archive_file.lambda_zip data source
aws_caller_identity.source data source
aws_caller_identity.target data source
aws_db_instance.rds data source
aws_iam_policy_document.lambda_kms_permissions data source
aws_iam_policy_document.source_lambda_assume_role_policy data source
aws_iam_policy_document.source_lambda_permissions data source
aws_iam_policy_document.source_retion_sns_policy data source
aws_iam_policy_document.target_lambda_assume_role_policy data source
aws_iam_policy_document.target_lambda_permissions data source
aws_iam_policy_document.target_retion_sns_policy data source
aws_kms_key.target_key data source
aws_rds_cluster.rds data source
aws_region.intermediate data source
aws_region.source data source
aws_region.target data source

Inputs

Name Description Type Default Required
name Name of the setup string n/a yes
rds_instance_ids List of IDs of the RDS instances to back up. If using Aurora, provide the cluster IDs instead list(string) n/a yes
target_account_kms_key_id KMS key to use to encrypt replicated RDS snapshots in the target AWS account string n/a yes
is_aurora_cluster Whether we're backing up Aurora clusters instead of RDS instances bool false no
lambda_monitoring_metric_period The metric period to use for the Lambdas CloudWatch alerts for monitoring. This should be equal or higher than the snapshoting period number 21600 no
retention_period Snapshot retention period in days number 14 no
snapshot_schedule_expression Snapshot frequency specified as a CloudWatch schedule expression. Can either be a rate() or cron() expression. Check the AWS documentation on how to compose such expression. string "cron(0 */6 * * ? *)" no
storage_type Determine storage type for your Aurora RDS. Valid values for Single-AZ: aurora, "" (default, both refer to Aurora Standard), aurora-iopt1 (Aurora I/O Optimized). Valid values for Multi-AZ: io1 (default). string null no

Outputs

Name Description
source_region_sns_topic_arn SNS topic ARN for the lambdas in the source region
target_region_sns_topic_arn SNS topic ARN for the lambdas in the target region

rds-proxy

Create an RDS proxy and configure IAM role to use for reading AWS Secrets to access the database.

Inputs

Name Description Type Default Required
db_cluster_identifier ID of the database instance to set as the proxy target any n/a yes
db_secret_arns AWS Secret Manager ARNs to use to access the database credentials list n/a yes
engine RDS engine: MYSQL or POSTGRES any n/a yes
environment The current environment any n/a yes
project The current project any n/a yes
security_groups Security groups that are allowed to access the RDS list(string) n/a yes
subnets Subnets to deploy in list(string) n/a yes
proxy_connection_timeout The number of seconds for a proxy to wait for a connection to become available in the connection pool number 120 no
proxy_max_connection_percent The maximum size of the connection pool for each target in a target group number 100 no
idle_client_timeout The number of seconds that a connection to the proxy can be inactive before the proxy disconnects it. number 1800 no
debug_logging Whether the proxy includes detailed information about SQL statements in its logs. bool false no
reader_endpoint Whether to include an extra reader endpoint. bool false no

Outputs

Name Description
proxy_endpoint Endpoint of the created proxy

Example

module "rds_proxy" {
  source = "github.com/skyscrapers/terraform-rds//rds_proxy"
  subnets                    = data.terraform_remote_state.networking.outputs.private_db_subnets
  project                    = var.project
  environment                = terraform.workspace
  engine                     = "MYSQL"
  security_groups            = ["sg-aaaaa", "sg-bbbb"]
  db_cluster_identifier     = module.rds_database.rds_id
  db_secret_arns             = [aws_secretsmanager_secret.db_user_rw.arn, aws_secretsmanager_secret.db_user_ro.arn]
}