Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adds basic AWS S3 client #1019

Merged
merged 11 commits into from
Oct 29, 2023
3 changes: 2 additions & 1 deletion doc/.vuepress/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,8 @@ module.exports = {
"net/uri",
"net/address",
"net/sasl",
"net/repl"
"net/repl",
"net/s3"
vyzo marked this conversation as resolved.
Show resolved Hide resolved
]
},

Expand Down
1 change: 1 addition & 0 deletions doc/reference/std/net/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,3 +11,4 @@ These are libraries related to network programming:
- [:std/net/address](net.md)
- [:std/net/sasl](sasl.md)
- [:std/net/repl](repl.md)
- [:std/net/s3](s3.md)
vyzo marked this conversation as resolved.
Show resolved Hide resolved
148 changes: 148 additions & 0 deletions doc/reference/std/net/s3.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,148 @@
# Amazon S3 Client

The `:std/net/s3` library provides basic support for interfacing with Amazon S3 and
compatible services.

::: warning
Only HTTPS is currently supported
:::

## Creating a Client

The primary way to interact with S3 via this library is with the `S3` interface.

To create an instance of an `S3` client, use the `S3Client` constructor:

```scheme
(S3Client
endpoint: (endpoint "s3.amazonaws.com")
access-key: (access-key (getenv "AWS_ACCESS_KEY_ID" #f))
secret-key: (secret-key (getenv "AWS_SECRET_KEY" #f))
region: (region (getenv "AWS_DEFAULT_REGION" "us-east-1")))
```

If `access-key` or `secret-key` aren't passed in and cannot be retrieved from the
environment (via `AWS_ACCESS_KEY_ID` and `AWS_SECRET_KEY` respectively) a `S3Error` is
raised.

## Client

The `S3` interface has the following signature:
```scheme
(interface S3
(get-bucket (name :~ string?))
(create-bucket! (name :~ string?))
(delete-bucket! (name :~ string?))
(bucket-exists? (name :~ string?))
(list-buckets))
```

## S3-get-bucket

```scheme
(S3-get-bucket client bucket-name) -> S3Bucket
```

`S3-get-bucket` retrieves a `S3Bucket` instance by name. If the bucket does not exist, a
`S3Error` is raised. Buckets are searched for in the service and region provided to
`S3Client`'s `endpoint` and `region` arguments.

## S3-create-bucket!

```scheme
(S3-create-bucket! client bucket-name) -> S3Bucket
```

`S3-create-bucket!` creates a new bucket in the service and region the client was
instantiated with. If a bucket with the name already exists, a `S3Error` is raised
indicating there was a conflict. If the bucket is successfully created, an instance of
`S3Bucket` corresponding to the newly created bucket is returned.

## S3-delete-bucket!

```scheme
(S3-delete-bucket! client bucket-name) -> void
```

`S3-delete-bucket!` attempts to delete a bucket with the given name. `delete-bucket!` is
idempotent and will **not** error if the bucket does not exist.

## S3-bucket-exists?

```scheme
(S3-bucket-exists? client bucket-name) -> bool
```

`S3-bucket-exists?` checks if a bucket with the provided name exists in the client's
configured region and endpoint. It returns `#t` if a bucket exists and `#f` otherwise.

## S3-list-buckets

```scheme
(S3-list-buckets client) -> list : S3Bucket
```

`S3-list-buckets` returns a list of all buckets available to the client as configured. If
none are available, an empty list is returned. All buckets are instances of `S3Bucket`.

## S3Bucket

The `S3Bucket` interface provides a consistent way to interact with buckets.

```scheme
(interface S3Bucket
(get (name :~ string?))
(put! (name :~ string?)
(data :~ u8vector?))
(delete! (name :~ string?))
(copy-to! (src :~ string?) (dest :~ string?))
(list-objects))
```
## S3Bucket-get

```scheme
(S3Bucket-get bucket object-name) -> u8vector
```

`S3Bucket-get` retrieves a object by name. If the object does not exist or the client
does not have permission to retrieve the object, an `S3Error` is raised.

## S3Bucket-put!

```scheme
(S3Bucket-put! bucket object-name) -> void
```

`S3Bucket-put!` stores `data` in a object with the provided `name`. Any failures result
in a `S3Error` being raised. All data is stored with `content-type: "octet-stream"` MIME
type.

## S3Bucket-delete!

```scheme
(S3Bucket-delete! bucket object-name) -> void
```

`S3Bucket-delete!` delete's an object with the provided `name`. If the object does not
exist, an `S3Error` is raised.

## S3Bucket-copy-to!

```scheme
(S3Bucket-copy-to! bucket src dest) -> void
```

`S3Bucket-copy-to!` performs server-side copy of object described by `src`. The `src`
format is `<bucket>/<objectname>`. The `dest` is the same as if performing a `put!` and
will store the object in the current `S3Bucket`. Copies must be within a region and
otherwise possible to perform using the `S3` client this bucket was created with.
Any failures result in a `S3Error` being raised.

## S3Bucket-list-objects

```scheme
(S3Bucket-list-objects bucket) -> list : string
```

`S3Bucket-list-objects` enumerates all objects in the bucket, returning their names as
strings.
4 changes: 4 additions & 0 deletions src/std/build-spec.ss
Original file line number Diff line number Diff line change
Expand Up @@ -217,6 +217,10 @@
"net/uri"
"net/request"
"net/json-rpc"
"net/s3"
"net/s3/interface"
"net/s3/api"
"net/s3/sigv4"
"net/websocket/interface"
"net/websocket/socket"
"net/websocket/client"
Expand Down
4 changes: 2 additions & 2 deletions src/std/interface.ss
Original file line number Diff line number Diff line change
Expand Up @@ -405,7 +405,7 @@
((kw (id default) . rest)
(stx-keyword? #'kw)
(lp #'rest (cons* #'(id absent-obj) #'kw args)))
((kw (id . contract) .rest)
((kw (id . contract) . rest)
(stx-keyword? #'kw)
(cond
((get-contract-default #'contract)
Expand Down Expand Up @@ -439,7 +439,7 @@
((kw (id default) . rest)
(stx-keyword? #'kw)
(lp #'rest (cons* #'(id default) #'kw args)))
((kw (id . contract) .rest)
((kw (id . contract) . rest)
(stx-keyword? #'kw)
(cond
((get-contract-default #'contract)
Expand Down
5 changes: 5 additions & 0 deletions src/std/net/s3.ss
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
;;; -*- Gerbil -*-
;;; © vyzo, ngp
;;; AWS S3 Client
(import ./s3/api ./s3/interface)
(export (import: ./s3/api ./s3/interface))
Loading
Loading