Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds basic AWS S3 client #1019

Merged
merged 11 commits into from
Oct 29, 2023
Merged

Adds basic AWS S3 client #1019

merged 11 commits into from
Oct 29, 2023

Conversation

chiefnoah
Copy link
Collaborator

Includes support for:

  • Bucket get/set/list/delete/exists?
  • Server-side object copying
  • Object get/set/list/delete
  • Bucket exists

Partially resolves #840

@netlify
Copy link

netlify bot commented Oct 18, 2023

Deploy Preview for elastic-ritchie-8f47f9 ready!

Name Link
🔨 Latest commit aacba25
🔍 Latest deploy log https://app.netlify.com/sites/elastic-ritchie-8f47f9/deploys/653ec9635f7d260008c1171b
😎 Deploy Preview https://deploy-preview-1019--elastic-ritchie-8f47f9.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@chiefnoah
Copy link
Collaborator Author

This is definitely incomplete. I have not even given it a once over myself, so expect there to be silly mistakes 🙂

@fare
Copy link
Collaborator

fare commented Oct 18, 2023

Looks generally good. I'll let vyzo decide whether we want that in gerbil itself, or in a separate pkg gerbil-aws or such. Since it's a proprietary service, I'd lean towards making it a separate package, myself.

@vyzo
Copy link
Collaborator

vyzo commented Oct 18, 2023

If not in stdlib, this desrves to be a mighty gerbils official package.

Let's get it reviewed here and we decide.

Copy link
Collaborator

@vyzo vyzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks pretty good, left some comments.

In general, if you have a contract check at the interface level (you do), you can elide it in the method implementation.

src/std/net/s3.ss Outdated Show resolved Hide resolved
src/std/net/s3/api.ss Outdated Show resolved Hide resolved
src/std/net/s3/api.ss Outdated Show resolved Hide resolved
@chiefnoah
Copy link
Collaborator Author

chiefnoah commented Oct 18, 2023

Looks generally good. I'll let vyzo decide whether we want that in gerbil itself, or in a separate pkg gerbil-aws or such. Since it's a proprietary service, I'd lean towards making it a separate package, myself.

This is already in a separate repo under chifnoah/gerbil-s3, I'd be happy to clean that up and transfer ownership to the might-gerbils organization. That being said, while S3 is a proprietary service, the S3 api is definitely a de-facto standard by this point with several open source and 3rd party proprietary implementations. I think I still lean towards having it be in a separate package though.

@chiefnoah chiefnoah requested a review from vyzo October 18, 2023 22:37
src/std/net/s3/api.ss Outdated Show resolved Hide resolved
src/std/net/s3/api.ss Outdated Show resolved Hide resolved
src/std/net/s3/api.ss Outdated Show resolved Hide resolved
src/std/net/s3/api.ss Outdated Show resolved Hide resolved
src/std/net/s3/api.ss Outdated Show resolved Hide resolved
src/std/net/s3/api.ss Outdated Show resolved Hide resolved
src/std/net/s3/api.ss Outdated Show resolved Hide resolved
src/std/net/s3/api.ss Outdated Show resolved Hide resolved
src/std/net/s3/api.ss Outdated Show resolved Hide resolved
src/std/net/s3/interface.ss Outdated Show resolved Hide resolved
src/std/net/s3/interface.ss Outdated Show resolved Hide resolved
@vyzo
Copy link
Collaborator

vyzo commented Oct 20, 2023

Don't worry about the gambit master CI failure, there has been a change in the locat constructor, I'll fix that.

@chiefnoah chiefnoah force-pushed the gerbil-s3 branch 2 times, most recently from 60e1c5c to b282178 Compare October 21, 2023 03:41
@chiefnoah
Copy link
Collaborator Author

Alright, I think it's good to go now 🙂

@chiefnoah chiefnoah marked this pull request as ready for review October 25, 2023 22:17
Includes support for:
- Bucket get/set/list/delete/exists?
- Server-side object copying
- Object get/set/list/delete
- Bucket exists
* Raise exceptions instead of return #f
* Don't export structs, create a constructor that wraps the return value
  in the interface
* Raise error when credentials are not supplied
* memv -> memq for speed
* more appropriate use of (using ...)
* Renames interfaces to be more clear
* Renames s3-client constructor from make-s3-client to S3Client to not
  conflict with the one generated by `defstruct`
* Calculate the empty-body sha256 hash on module import time
Copy link
Collaborator

@vyzo vyzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, can we add some docs?

@chiefnoah
Copy link
Collaborator Author

Yup! Will work on that

(get (name :~ string?))
(put! (name :~ string?)
(data :~ u8vector?)
content-type: (content-type := "octet-stream" :~ string?))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we care about this?

Note that dynamic dispatch with kw lambdas is expensive.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we pay that cost no matter what or only when we specify content-type:?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess it's not really necessary if we're enforcing (data :~ u8vector?)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it is a dynamic call through the interface, so the compiler cannot optimize dispatch.

@chiefnoah chiefnoah marked this pull request as draft October 28, 2023 21:17
@chiefnoah chiefnoah marked this pull request as ready for review October 29, 2023 20:51
Copy link
Collaborator

@vyzo vyzo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say it is ready to merge, one minor thing -- see comment.

doc/reference/std/net/README.md Show resolved Hide resolved
@chiefnoah chiefnoah enabled auto-merge (squash) October 29, 2023 21:07
@chiefnoah chiefnoah merged commit b31852b into master Oct 29, 2023
7 of 11 checks passed
@chiefnoah chiefnoah deleted the gerbil-s3 branch October 29, 2023 22:01
drewc pushed a commit to drewc/gerbil that referenced this pull request Nov 29, 2023
Includes support for:
- Bucket get/set/list/delete/exists?
- Server-side object copying
- Object get/set/list/delete
- Bucket exists

Partially resolves mighty-gerbils#840

---------

Co-authored-by: vyzo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

S3 Library for std
3 participants