S3 - Profile with role_arn are not working

[alexvanbelle]

Enhancement Suggestion / Bug Report

AWS recommends to use assume roles for cross-accounts management (which is the standard approach in a company). Rome can support profiles but the role_arn seems ignored and therefore, users get access denied.

Steps which explain the enhancement or reproduce the bug
For a matter of simplicity of repro steps, I give steps with admin rights while in real scenario, users would be granted only some roles and the roles would have less privileges too.
Note that this tutorial can be useful for people that are not used to this mechanism.

1. Create two AWS accounts (named A and B below)
2. On account A, create a user account "myuser" and gives him admin rights
3. On account B, create a S3 bucket for Rome (region=us-east-1)
4. On account B, create a role that can be assumed from account A and gives him admin rights
5. Login on account A with "myuser" and click on switch role to assume the new role on account B: validate you are able to browse/read/write from the console
6. Configure the CLI to have a profile which assume this role (profile with a line role_arn='<full_arn>').
7. export AWS_PROFILE='<yourprofilename', export AWS_REGION='us-east-1'. Validate you can list the buckets with "aws s3 ls". You should see the buckets from account B and use it properly
8. Use Rome with this profile. You'll get access denied. (which makes me believe Rome is not ignoring the role_arn while the AWS Cli is using it)

Current behavior
Access denied -> Rome doesn't seem to assume role.

The workaround is to create a shared service account on "account B" and to provide the access key to our users (we don't want to create user accounts on "account B"). This workaround is a bad practice.

Suggested behavior
Role should be assumed.

Why would the enhancement be useful to most users
This is how most organizations are/will be using AWS in production.

Rome version:
0.19.0.55 - Romam uno die non fuisse conditam.

OS and version:
MACOS, Mojave
OS is probably not important here.

[tmspzz]

Can this be a duplicate of #159 ? I appreciate the explanation

[tmspzz]

Is it this? http://hackage.haskell.org/package/amazonka-sts-1.6.1/docs/Network-AWS-STS-AssumeRole.html

[alexvanbelle]

Hi,
I've seen #159 before opening this one. I believe they are related but slightly different:

1. This one is about users assuming roles cross-accounts.
2. Add support for S3 IAM Role #159 is (as I understand latest answer on it) about EC2 IAM Role

Both covers different scenario (in respective order):

1. If you have developers that should use Rome on their local computer, they should have a personal account and generate an access key for it. Then, they configure their local computer with this access key and assume the role they are granted to.
2. If you have a build machine that is running on EC2, you should use EC2 role to avoid generating access keys. This is one way on AWS to avoid creating a special service user. By using an EC2 role, you grants the same right and you don't have to rotate keys.

Let me know if you need more details, IAM can be hard to manage.

[alexvanbelle]

Is it this? http://hackage.haskell.org/package/amazonka-sts-1.6.1/docs/Network-AWS-STS-AssumeRole.html

I've seen that. It looks ok but I don't have enough knowledge on this Haskell library. The documentation in itself is relevant. However, the package is misleading: why is it in the network one?

[tmspzz]

from what I gather this would work in the following way:

• specify ARN
• rome makes request to STS, get the new keys
• rome uses the new keys

correct?

[tmspzz]

also I don't think possible to run rome on EC2 (it requires Xcode) so I'm quite puzzle by #159

[alexvanbelle]

from what I gather this would work in the following way:

* specify ARN

* rome makes request to STS, get the new keys

* rome uses the new keys

correct?

Yes it is, kind of. Actually, when you use STS (Security Token Service), it generates one additional parameter compared to "normal" access keys. You'll get a token.

So, it's more:

1. Use the permanent Access key and secret to use STS to get a new temporary access key, its associated secret and the session token
2. Romes uses the temporary credentials (access key, secret and token) to do S3 requests

It's what I can see from the library documentation, it seems you can specify the session rather than the access keys. See here

On other languages such as Python, the official SDK provides that (boto3 is the official library provided by AWS)

To show you
FULL_SESSION_TOKEN=$(aws sts get-session-token --duration-seconds $DURATION_TOKEN_SECONDS)

TEMP_ACCESS_KEY=$(echo $FULL_SESSION_TOKEN | jq .Credentials.AccessKeyId)
TEMP_SECRET_KEY=$(echo $FULL_SESSION_TOKEN | jq .Credentials.SecretAccessKey)
TEMP_SESSION_TOKEN=$(echo $FULL_SESSION_TOKEN | jq .Credentials.SessionToken)
TEMP_EXPIRATION=$(echo $FULL_SESSION_TOKEN | jq .Credentials.Expiration)`

[tmspzz]

So would it be enough rome to support auth via Session?

As far as I understand the session itself can be retrieved beforehand with the AWS CLI for example.

Sorry I'm really not familiar with this part of aws.

[alexvanbelle]

Would say no because it would not be userfriendly. Session last 1h in general.

I know understading this part can be tricky and it takes time to get used to it, so no worries, you're not alone :)

You should propose two approaches:

1. The profile is using role_arn: you should use session.
2. The profile is not using role_arn: you should use the permanent access keys

May you point me to the place in your code where you're managing current authentication (reading the access keys). I might provide better guidance :)

[tmspzz]

Here is the bit: https://github.com/blender/Rome/blob/master/src/Lib.hs#L72
This uses "Discover" strategy:

• Explanation: https://github.com/brendanhay/amazonka/blob/248f7b2a7248222cc21cef6194cd1872ba99ac5d/amazonka/src/Network/AWS/Auth.hs#L215
• Implementation: https://github.com/brendanhay/amazonka/blob/248f7b2a7248222cc21cef6194cd1872ba99ac5d/amazonka/src/Network/AWS/Auth.hs#L354

I don't think it would be hard for me to make an additional call honestly to get the new credentials from STS.

[tmspzz]

Other questions:

should ARN be:

• a CLI option?
• and or an ENV variable?
• stored in some file?

[alexvanbelle]

Other questions:

should ARN be:

* a CLI option?

* and or an ENV variable?

* stored in some file?

My understanding is that you want to mimic the behavior of AWS CLI (as I can read in the README of the project).

You can read this.

So, to answer your question: role_arn should be in the ~/.aws/config file, under the AWS_PROFILE users will use for Rome.

[alexvanbelle]

Do you an AWS account on which you are admin? I can provide guidance on how to create a role in order to help you try it. (while we use roles mainly for cross accounts, you can assume roles that are in the same account).

[tmspzz]

Yes, I have a AWS account I can play around. I'll try to follow the tutorial you posted and see how far I get.

Following the AWS CLI convention sounds solid.

[alexvanbelle]

Ok, then, on a single account, you can do the following.

Gather the information:
Any AWS account has an ID. This is a big integer. You can get yours from there. Keep it because you will use it in different steps below.

Create the role
These steps create a role on your account that can be assumed by users of your account. You need to use a user with high privilege to create such a role.

1. Go on IAM
2. Click on "Roles" and then, "Create role"
3. For the type of trusted entity, select "Another AWS account"
4. For the account ID, use the one of your account, then click on next
5. On permissions, select the one you need. On real life, you would put a least privilege here. For this repro steps, you can select "AmazonS3FullAccess" and then click next
6. For tags, keep them empty
7. For name, let's put "RoleForRomeSTS" then click "Create role"

Use the role from the console
These steps show you how to assume the role on the console. You need a user with a lot of privilege (sts:* as an example). In real life, you would grant only the right sts policies for the role you grant to that users. If you have a user with admin rights, he's granted sts:*.

1. Once logged on the console, click on the username
2. Click on "Switch Role"
3. In account, put the account id (the big integer). In role, put "RoleForRomeSTS". You can keep the display name (or adjust it to your needs).
4. You should now have assumed the role and browse S3 but get permissions denied for other services (according to the permissions you granted to the role).

Use the role from the CLI

1. Go on IAM, select the role you want to assume.
2. Note its ARN
3. Use this [configuration] to add a new profile(https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-role.html)
4. export AWS_PROFILE="<yourprofilename" and then, "aws s3 ls" should be OK while "aws ec2 describe-instances"

Please let me know. After your test, I recommend to delete this role.

[tmspzz]

Thanks! I will follow your instructions.

Unfortunately this completely breaks credentials discovery and I have to do it from scratch.

About the discovery of the credentials, can you please validate the following:

• AWS_PROFILE env variable will be set to the name of the profile where the role_arn is specified. E.g. AWS_PROFILE="profile s3"
• in case AWS_PROFILE is not set, use default
• whatever the profiles ends up being it should also contain role_arn but may of may not contain source_profile.
• If no source_profile then then current profile can assume role via aws_access_key_id and aws_secret_access_key
• make STS request with source_profile credentials
• on success, use credentials provided form STS for further calls

[alexvanbelle]

Sorry for the delay. Please find my answers below.

* `AWS_PROFILE` env variable will be set **to the name of the profile where the role_arn is specified**. E.g. AWS_PROFILE="profile s3"

[alexvan] Correct.

* in case AWS_PROFILE is not set, use `default`

[alexvan] Correct.

* whatever the profiles ends up being it should also contain `role_arn` but may of may not contain `source_profile`.

[alexvan] I don't know. I've never used this kind of configuration.

* If no `source_profile` then then current profile can assume role via `aws_access_key_id` and `aws_secret_access_key`

[alexvan] I don't know. I've never used this kind of configuration.

* make STS request with `source_profile` credentials

[alexvan] I believe it's correct. However, I believe users could have different configurations.

* on success, use credentials provided form STS for further calls

[alexvan] Correct. Credentials are temporary access keys + secrets + token. However, you should manage the expiration (by default, it's one hour if I remember properly)

My 2 cents:
I believe reading the configuration yourself might be hard to maintain. AWS might change things and you'll notice too late. Why don't you provide your own configuration file? You put three optional parameters: Access key(mandatory), secret(mandatory), rolearn (optional). This way: you keep it simple and easy to maintain.

[tmspzz]

WIP at #179

[tmspzz]

@alexvanbelle please try https://github.com/blender/Rome/releases/tag/v0.21.0.57

[tmspzz]

This seems to work fine! Hopefully I didn't break any other mechanism and there are enough tests to prove that.

[tmspzz]

closed by https://github.com/blender/Rome/releases/tag/v0.21.0.58