app-rails: Add based on template-application-rails

[doshitan]

Ticket

Resolves navapbc/template-infra#762

Changes

Add application based on template-application-rails, following its guide[1].

This required some edits of infra configuration, callouts:

• Ignore HTTPS errors in E2E tests (due to PR Environments)
• Support for curl in container healthchecks
• Support for non-read-only container filesystems

[1] https://github.com/navapbc/template-application-rails/tree/881cc71e062e5e7c44b55d28946a84ce13123d9d/template-only-docs/Deployment.md

Context for reviewers

TODOs:

• Set up the CI/CD dependent on the app_has_dev_env_setup answer
• The rails server requires write access to the filesystem on startup, so the infra service module is modified to support this, should these changes be upstreamed into template-infra first? We'll upstream, but can carry the custom changes here for now.

Current status: service running

Do some basic manual tests at: https://app-rails.platform-test-dev.navateam.com/

(create account, login, etc)

Setup

Code

Before all the new releases were tagged:

nava-platform infra update --template-uri ../template-infra --version=doshitan/platform-cli .
nava-platform infra add-app --template-uri ../template-infra . app-rails
nava-platform app install --commit --template-uri ../template-application-rails --version platform-cli . app-rails

After the releases:

nava-platform infra add-app . app-rails
nava-platform app install --commit --template-uri gh:navapbc/template-application-rails . app-rails

Infra/Setting up dev environment

make infra-update-current-account
No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration and found no differences, so no changes are needed.

Apply complete! Resources: 0 added, 0 changed, 0 destroyed.

Outputs:

account_id = "430004246987"
project_name = "platform-test"
region = "us-east-1"
tf_locks_table_name = "platform-test-430004246987-us-east-1-tf-state-locks"
tf_log_bucket_name = "platform-test-430004246987-us-east-1-tf-logs"
tf_state_bucket_name = "platform-test-430004246987-us-east-1-tf"

❯ make infra-configure-network NETWORK_NAME=dev
./bin/create-tfbackend infra/networks dev
====================================
Create terraform backend config file
====================================
Input parameters
  module_dir=infra/networks
  backend_config_name=dev

Created file: infra/networks/dev.s3.tfbackend
------------------ file contents ------------------
bucket         = "platform-test-430004246987-us-east-1-tf"
key            = "infra/networks/dev.tfstate"
dynamodb_table = "platform-test-430004246987-us-east-1-tf-state-locks"
region         = "us-east-1"
----------------------- end -----------------------

make infra-update-network NETWORK_NAME=dev

# it wanted to remove some notification VPC endpoints, said no

❯ make infra-configure-app-database APP_NAME=app-rails ENVIRONMENT=dev
./bin/create-tfbackend "infra/app-rails/database" "dev"
====================================
Create terraform backend config file
====================================
Input parameters
  module_dir=infra/app-rails/database
  backend_config_name=dev

Created file: infra/app-rails/database/dev.s3.tfbackend
------------------ file contents ------------------
bucket         = "platform-test-430004246987-us-east-1-tf"
key            = "infra/app-rails/database/dev.tfstate"
dynamodb_table = "platform-test-430004246987-us-east-1-tf-state-locks"
region         = "us-east-1"
----------------------- end -----------------------

❯ make infra-update-app-database APP_NAME=app-rails ENVIRONMENT=dev

# create a bunch of resources

module.database.aws_rds_cluster_instance.primary: Creation complete after 11m56s [id=app-rails-dev-primary]

Apply complete! Resources: 23 added, 0 changed, 0 destroyed.

Outputs:

role_manager_function_name = "app-rails-dev-role-manager"

❯ make infra-update-app-database-roles APP_NAME=app-rails ENVIRONMENT=dev            

./bin/create-or-update-database-roles app-rails dev

================================                                                                                                                                        
Creating/updating database users                                                                                                                                        
================================                                                                                                                                        
Input parameters                                                                                                                                                        
  app_name=app-rails                                                                                                                                                    
  environment=dev                                                                                                                                                       
                                                                                                                                                                        
Invoking Lambda function: app-rails-dev-role-manager                                                                                                                    
  Payload: {"action":"manage","config":{                                                                                                                                
  "app_access_policy_name": "app-rails-dev-app-access",                                                                                                                 
  "app_username": "app",                                                                                                                                                
  "cluster_name": "app-rails-dev",                                                                                                                                      
  "migrator_access_policy_name": "app-rails-dev-migrator-access",                                                                                                       
  "migrator_username": "migrator",                                                                                                                                      
  "region": "us-east-1",                                                                                                                                                
  "schema_name": "app",                                                                                                                                                 
  "superuser_extensions": {}                                                                                                                                            
}}

START RequestId: 4e97b221-1e7e-4f89-960c-03b5f0aeb46e Version: $LATEST
-- Running command 'manage' to manage database roles, schema, and privileges
Fetching password from parameter store:
/aws/reference/secretsmanager/rds!cluster-bd1053c3-1a07-431e-b2f0-379c2c39512e
Connecting to database: user='postgres' host='app-rails-dev.cluster-cluxgx4shg5c.us-east-1.rds.amazonaws.com' port='5432' database='app'
-- Current database configuration
---- Roles
------ Role postgres
---- Schema privileges                                                                                                                                                  
------ Schema name='public' acl='{pg_database_owner=UC/pg_database_owner,=U/pg_database_owner}'                                                                         
-- Configuring database                                                                                                                                                 
---- Revoking default access on public schema                                                                                                                           
postgres> REVOKE CREATE ON SCHEMA public FROM PUBLIC
---- Revoking database access from public role
postgres> REVOKE ALL ON DATABASE app FROM PUBLIC
---- Setting default search path to schema app
postgres> ALTER DATABASE app SET search_path TO app
---- Configuring roles
------ Configuring role: username='migrator'
postgres> 
DO $$
BEGIN
CREATE USER migrator;
EXCEPTION WHEN DUPLICATE_OBJECT THEN
RAISE NOTICE 'user already exists';
END
$$;
postgres> GRANT rds_iam TO migrator
postgres> GRANT CONNECT ON DATABASE app TO migrator
------ Configuring role: username='app'
postgres> 
DO $$
BEGIN
CREATE USER app;
EXCEPTION WHEN DUPLICATE_OBJECT THEN
RAISE NOTICE 'user already exists';
END
$$;
postgres> GRANT rds_iam TO app
postgres> GRANT CONNECT ON DATABASE app TO app
---- Configuring schema
------ Creating schema: schema_name='app'
postgres> CREATE SCHEMA IF NOT EXISTS app
------ Changing schema owner: new_owner=migrator
postgres> ALTER SCHEMA app OWNER TO migrator
------ Granting schema usage privileges: grantee=app
postgres> GRANT USAGE ON SCHEMA app TO app 
---- Configuring superuser extensions
-- Current database configuration
---- Roles
------ Role postgres
------ Role migrator
------ Role app
---- Schema privileges
------ Schema name='public' acl='{pg_database_owner=UC/pg_database_owner,=U/pg_database_owner}'
------ Schema name='app' acl='{migrator=UC/migrator,app=U/migrator}'
Connecting to database: user='migrator' host='app-rails-dev.cluster-cluxgx4shg5c.us-east-1.rds.amazonaws.com' port='5432' database='app'
------ Granting privileges for future objects in schema: grantee=app
migrator> ALTER DEFAULT PRIVILEGES IN SCHEMA app GRANT ALL ON TABLES TO app
migrator> ALTER DEFAULT PRIVILEGES IN SCHEMA app GRANT ALL ON SEQUENCES TO app
migrator> ALTER DEFAULT PRIVILEGES IN SCHEMA app GRANT ALL ON ROUTINES TO app
END RequestId: 4e97b221-1e7e-4f89-960c-03b5f0aeb46e
REPORT RequestId: 4e97b221-1e7e-4f89-960c-03b5f0aeb46e  Duration: 17514.20 ms   Billed Duration: 17515 ms       Memory Size: 128 MB     Max Memory Used: 89 MB  Init Duration: 519.32 ms
XRAY TraceId: 1-6789685d-164bf74e54e594b25e8db9b9       SegmentId: 6fcd95d165ce2c81     Sampled: true

Lambda function response:
{"roles": ["postgres", "migrator", "app"], "roles_with_groups": {"rds_superuser": "rds_password", "pg_monitor": "pg_read_all_settings,pg_read_all_stats,pg_stat_scan_tables", "postgres": "rds_superuser", "app": "rds_iam", "migrator": "rds_iam"}, "schema_privileges": {"public": "{pg_database_owner=UC/pg_database_owner,=U/pg_database_owner}", "app": "{migrator=UC/migrator,app=U/migrator}"}}

❯ make infra-configure-app-build-repository APP_NAME=app-rails
./bin/create-tfbackend "infra/app-rails/build-repository" shared
====================================
Create terraform backend config file
====================================
Input parameters
  module_dir=infra/app-rails/build-repository
  backend_config_name=shared

Created file: infra/app-rails/build-repository/shared.s3.tfbackend
------------------ file contents ------------------
bucket         = "platform-test-430004246987-us-east-1-tf"
key            = "infra/app-rails/build-repository/shared.tfstate"
dynamodb_table = "platform-test-430004246987-us-east-1-tf-state-locks"
region         = "us-east-1"
----------------------- end -----------------------

❯ make infra-update-app-build-repository APP_NAME=app-rails                                                                                                             
./bin/terraform-init-and-apply infra/app-rails/build-repository shared
module.container_image_repository.aws_kms_key.ecr_kms: Creating...
module.container_image_repository.aws_kms_key.ecr_kms: Creation complete after 10s [id=417c1a40-416b-40d4-8c73-74ecb73c5fca]
module.container_image_repository.aws_ecr_repository.app: Creating...
module.container_image_repository.aws_ecr_repository.app: Creation complete after 0s [id=platform-test-app-rails]
module.container_image_repository.aws_ecr_lifecycle_policy.image_retention: Creating...
module.container_image_repository.aws_ecr_repository_policy.image_access: Creating...
module.container_image_repository.aws_ecr_lifecycle_policy.image_retention: Creation complete after 0s [id=platform-test-app-rails]
module.container_image_repository.aws_ecr_repository_policy.image_access: Creation complete after 0s [id=platform-test-app-rails]

Apply complete! Resources: 4 added, 0 changed, 0 destroyed.

❯ make infra-configure-app-service APP_NAME=app-rails ENVIRONMENT=dev

./bin/create-tfbackend "infra/app-rails/service" "dev"
====================================
Create terraform backend config file
====================================
Input parameters
  module_dir=infra/app-rails/service
  backend_config_name=dev

Created file: infra/app-rails/service/dev.s3.tfbackend
------------------ file contents ------------------
bucket         = "platform-test-430004246987-us-east-1-tf"
key            = "infra/app-rails/service/dev.tfstate"
dynamodb_table = "platform-test-430004246987-us-east-1-tf-state-locks"
region         = "us-east-1"
----------------------- end -----------------------

❯ make release-build APP_NAME=app-rails
❯ make release-publish APP_NAME=app-rails
./bin/publish-release app-rails platform-test-app-rails 510ea04e49d91f8f1b84748d5a77cf29460264f7
---------------
Publish release
---------------
Input parameters:
  app_name=app-rails
  image_name=platform-test-app-rails
  image_tag=510ea04e49d91f8f1b84748d5a77cf29460264f7
Build repository info:
  region=us-east-1
  image_registry=430004246987.dkr.ecr.us-east-1.amazonaws.com
  image_repository_name=platform-test-app-rails
  image_repository_url=430004246987.dkr.ecr.us-east-1.amazonaws.com/platform-test-app-rails

510ea04e49d91f8f1b84748d5a77cf29460264f7: digest: sha256:79bed89119a8c118f56e19aac876f007b8927b70da702810dbc8b2c67a0e518b size: 2629

❯ TF_CLI_ARGS_apply="-var=image_tag=510ea04e49d91f8f1b84748d5a77cf29460264f7" make infra-update-app-service APP_NAME=app-rails ENVIRONMENT=dev 

│ Warning: Argument is deprecated
│ 
│   with module.service.aws_iam_role.events,
│   on ../../modules/service/events_role.tf line 10, in resource "aws_iam_role" "events":
│   10:   managed_policy_arns = [aws_iam_policy.run_task.arn]
│ 
│ The managed_policy_arns argument is deprecated. Use the aws_iam_role_policy_attachment resource instead. If Terraform should exclusively manage all managed policy
│ attachments (the current behavior of this argument), use the aws_iam_role_policy_attachments_exclusive resource as well.
│ 
│ (and 2 more similar warnings elsewhere)
╵

Apply complete! Resources: 45 added, 0 changed, 0 destroyed.

Outputs:

application_log_group = "service/app-rails-dev"
application_log_stream_prefix = "app-rails-dev"
image_tag = "510ea04e49d91f8f1b84748d5a77cf29460264f7"
migrator_role_arn = "arn:aws:iam::430004246987:role/app-rails-dev-migrator"
service_cluster_name = "app-rails-dev"
service_endpoint = "http://app-rails-dev-1762374690.us-east-1.elb.amazonaws.com"
service_name = "app-rails-dev"

---

Configuring network after Cognito/HTTPS config

❯ make infra-update-network NETWORK_NAME=dev                                                                                                                            
terraform -chdir="infra/networks" init -input=false -reconfigure -backend-config="dev.s3.tfbackend" 

[...]

certificate_arns = {
  "app-rails.platform-test-dev.navateam.com" = "arn:aws:acm:us-east-1:430004246987:certificate/7fe1502b-3228-4fa4-bacd-f368d75ff7e4"
  "platform-test-dev.navateam.com" = "arn:aws:acm:us-east-1:430004246987:certificate/aab24593-7531-45ac-bb90-d83e9d99668a"
}
certificate_domains = [
  "app-rails.platform-test-dev.navateam.com",
  "platform-test-dev.navateam.com",
]
hosted_zone_name_servers = tolist([
  "ns-1431.awsdns-50.org",
  "ns-1643.awsdns-13.co.uk",
  "ns-687.awsdns-21.net",
  "ns-80.awsdns-10.com",
])

---

Accidentally running against "prod" account the first time

make infra-configure-app-database APP_NAME=app-rails ENVIRONMENT=dev

./bin/create-tfbackend "infra/app-rails/database" "dev"
====================================
Create terraform backend config file
====================================
Input parameters
  module_dir=infra/app-rails/database
  backend_config_name=dev

Created file: infra/app-rails/database/dev.s3.tfbackend
------------------ file contents ------------------
bucket         = "platform-test-288761770570-us-east-1-tf"
key            = "infra/app-rails/database/dev.tfstate"
dynamodb_table = "platform-test-288761770570-us-east-1-tf-state-locks"
region         = "us-east-1"
----------------------- end -----------------------

make infra-update-app-database APP_NAME=app-rails ENVIRONMENT=dev

Testing

Dev environment: https://app-rails.platform-test-dev.navateam.com/

Created account, logged in. Not much else to test.

Preview environment

♻️ Environment destroyed ♻️

Preview environment

♻️ Environment destroyed ♻️