Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

copy_object() is slow #166

Closed
leeper opened this issue Sep 26, 2017 · 2 comments
Closed

copy_object() is slow #166

leeper opened this issue Sep 26, 2017 · 2 comments

Comments

@leeper
Copy link
Member

leeper commented Sep 26, 2017

Report from Vitalina Komashko:

system.time(copy_object(from_object = from_object[1], to_object = to_object[1],
from_bucket = from_bucket, to_bucket = to_bucket, verbose = TRUE))
Using Environment Variable 'AWS_ACCESS_KEY_ID' for AWS Access Key ID
Using Environment Variable 'AWS_SECRET_ACCESS_KEY' for AWS Secret Access Key

Checking bucket region using get_location('<REDACTED>')
Executing request using bucket region us-west-1
S3 Request URL: https://s3-us-west-1.amazonaws.com/<REDACTED>
Executing request with AWS credentials
Parsing AWS API response
Success: (200) OK
    user   system  elapsed
   0.836    0.674 2437.777

This runs about 10x slower than the AWS CLI. Issue may be with locate_credentials():

/usr/bin/time -l aws s3 cp s3://<REDACTED> --profile corporate
copy: s3://<REDACTED> to s3://<REDACTED>

      263.67 real         2.24 user         0.65 sys
@rosseji
Copy link

rosseji commented Jan 17, 2018

@leeper is this the reason that copy_bucket() still uses the deprecated copyobject() underneath still?

@leeper
Copy link
Member Author

leeper commented Jul 28, 2018

Migrate to: cloudyr/aws.signature#33

@leeper leeper closed this as completed Jul 28, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants