How to use aws-auth in CI/CD pipeline

[gruntwork-support]

This message was extracted from a discussion that originally took place in Gruntwork Community Slack. Names and URLs have been removed where appropriate

From a customer

Hi, I am trying to execute infrastructure-deployer in our CI server, but I am struggling with assuming the correct role (and hence running in the correct destination account). Locally I am using aws-vault, and everything works (I can invoke the lambda) but on Jenkins I set AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_PROFILE. The ~/.aws/config contains a profile defining the arn.

[profile development]
credential_source = Environment
role_arn = arn:aws:iam::$DEVELOPMENT_ACCOUNT:role/allow-auto-deploy-from-other-accounts

This is what executing infrastrucure-deployer gives me:

AccessDeniedException: User arn:aws:iam::$SECURITY_ACCOUNT:user/jenkins is not authorized to perform lambda:InvokeFunction on resource arn:aws:lambda:eu-central-1:$SECURITY_ACCOUNT:function:ecs-deploy-runner

So it seems like the role_arn is ignored, and it also tries to call the lambda in the security account, where it does not exist. How can I tell infrastructure-deployer to assume a role first, before invoking the lambda?

I might have found the trick — that’s what aws-auth is for, isn’t it?

[gruntwork-support]

From a grunt

Yup that is correct! The idea is to use aws-auth to assume the role you want to use for deploying. You can take a look at our pipeline example from the reference architecture:

• Reference architecture infra live: https://github.com/gruntwork-io/terraform-aws-service-catalog/tree/master/examples/for-production/infrastructure-live
• Helper function to assume role using aws-auth: https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/examples/for-production/infrastructure-live/_ci/scripts/helpers.sh#L15
• Section of pipeline CI script where we take advantage of that: https://github.com/gruntwork-io/terraform-aws-service-catalog/blob/master/examples[…]/for-production/infrastructure-live/_ci/scripts/deploy-infra.sh

[gruntwork-support]

From a customer

got it working, thank you 🙂

[ecs-deploy-runner][2021-07-09T15:20:45+0000] [INFO] [infrastructure-deploy-script] 2021-07-09 15:20:45 "terragrunt plan" exited with code 0 🎉

[gruntwork-support]

From a grunt

Awesome news! 🎉
Thanks for closing the loop!

[gruntwork-support]

From a customer

One last question, since the deployer consists of quite a lot of components: could one not get rid of the fargate tasks by pulling their functionality into lambda, instead of using lambda only as a trigger for task creation?

[gruntwork-support]

From a grunt

That would be ideal, but unfortunately Lambda has a 15 minute task timeout and most deployments take longer than that.

[gruntwork-support]

From a customer

👍