Skip to content

Commit

Permalink
Reference assets directly (#257)
Browse files Browse the repository at this point in the history
Previously we were generating a physical name for the staging resources
(S3 Bucket / ECR Repository) because it was mistakenly assumed that we
needed those to be known values for CDK to work. Turns out that we can
use `asString` on the Output values and CDK handles it correctly. So
this PR does a couple of things:

1. Removes the hard coded resource names and allows Pulumi to generate
an autoname.
2. Removes the dependency each stack had on the staging stack. Because
we are using Output values to reference the assets, each resource can
depend on the individual asset directly. This allows for more resources
to be provisioned in parallel. On the `eventbridge-sns` test this had a
20% decrease in deployment time.
  • Loading branch information
corymhall authored Nov 26, 2024
1 parent 596084d commit ace430d
Show file tree
Hide file tree
Showing 6 changed files with 89 additions and 94 deletions.
32 changes: 16 additions & 16 deletions api-docs/Namespace.synthesizer.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,15 +54,15 @@ resources and deploy the assets themselves.

###### Defined in

[synthesizer.ts:231](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L231)
[synthesizer.ts:217](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L217)

#### Properties

| Property | Modifier | Type | Default value | Description | Overrides | Defined in |
| ------ | ------ | ------ | ------ | ------ | ------ | ------ |
| `stagingBucket?` | `public` | `BucketV2` | `undefined` | The app-scoped, environment-keyed staging bucket. | - | [synthesizer.ts:158](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L158) |
| `stagingRepos` | `readonly` | `Record`\<`string`, `Repository`\> | `{}` | The app-scoped, environment-keyed ecr repositories associated with this app. | - | [synthesizer.ts:163](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L163) |
| `stagingStack` | `readonly` | `CdkConstruct` | `undefined` | The Pulumi ComponentResource wrapper which contains all of the staging resources. This can be added to the `dependsOn` of the main stack to ensure the staging assets are created first | [`PulumiSynthesizerBase`](Namespace.synthesizer.md#pulumisynthesizerbase).`stagingStack` | [synthesizer.ts:153](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L153) |
| `stagingBucket?` | `public` | `BucketV2` | `undefined` | The app-scoped, environment-keyed staging bucket. | - | [synthesizer.ts:159](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L159) |
| `stagingRepos` | `readonly` | `Record`\<`string`, `Repository`\> | `{}` | The app-scoped, environment-keyed ecr repositories associated with this app. | - | [synthesizer.ts:164](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L164) |
| `stagingStack` | `readonly` | `CdkConstruct` | `undefined` | The Pulumi ComponentResource wrapper which contains all of the staging resources. This can be added to the `dependsOn` of the main stack to ensure the staging assets are created first | [`PulumiSynthesizerBase`](Namespace.synthesizer.md#pulumisynthesizerbase).`stagingStack` | [synthesizer.ts:154](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L154) |

#### Methods

Expand All @@ -82,7 +82,7 @@ Returns the S3 key prefix that will be used for deploy time assets.

###### Defined in

[synthesizer.ts:114](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L114)
[synthesizer.ts:115](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L115)

##### getStagingBucket()

Expand All @@ -101,7 +101,7 @@ and custom resource responses.

###### Defined in

[synthesizer.ts:437](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L437)
[synthesizer.ts:416](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L416)

***

Expand Down Expand Up @@ -136,7 +136,7 @@ creates Pulumi resources then you should extend this class.

| Property | Modifier | Type | Description | Defined in |
| ------ | ------ | ------ | ------ | ------ |
| `stagingStack` | `abstract` | `CdkConstruct` | The Pulumi ComponentResource wrapper which contains all of the staging resources. This can be added to the `dependsOn` of the main stack to ensure the staging assets are created first | [synthesizer.ts:103](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L103) |
| `stagingStack` | `abstract` | `CdkConstruct` | The Pulumi ComponentResource wrapper which contains all of the staging resources. This can be added to the `dependsOn` of the main stack to ensure the staging assets are created first | [synthesizer.ts:104](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L104) |

#### Methods

Expand All @@ -152,7 +152,7 @@ Returns the S3 key prefix that will be used for deploy time assets.

###### Defined in

[synthesizer.ts:114](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L114)
[synthesizer.ts:115](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L115)

##### getStagingBucket()

Expand All @@ -167,7 +167,7 @@ and custom resource responses.

###### Defined in

[synthesizer.ts:109](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L109)
[synthesizer.ts:110](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L110)

## Interfaces

Expand All @@ -177,10 +177,10 @@ and custom resource responses.

| Property | Modifier | Type | Description | Defined in |
| ------ | ------ | ------ | ------ | ------ |
| `appId` | `readonly` | `string` | A unique identifier for the application that the staging stack belongs to. This identifier will be used in the name of staging resources created for this application, and should be unique across CDK apps. The identifier should include lowercase characters, numbers, periods (.) and dashes ('-') only and have a maximum of 17 characters. | [synthesizer.ts:28](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L28) |
| `autoDeleteStagingAssets?` | `readonly` | `boolean` | Auto deletes objects in the staging S3 bucket and images in the staging ECR repositories. This will also delete the S3 buckets and ECR repositories themselves when all objects / images are removed. **Default** `true` | [synthesizer.ts:73](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L73) |
| `deployTimeFileAssetLifetime?` | `readonly` | `Duration` | The lifetime for deploy time file assets. Assets that are only necessary at deployment time (for instance, CloudFormation templates and Lambda source code bundles) will be automatically deleted after this many days. Assets that may be read from the staging bucket during your application's run time will not be deleted. Set this to the length of time you wish to be able to roll back to previous versions of your application without having to do a new `cdk synth` and re-upload of assets. **Default** `- Duration.days(30)` | [synthesizer.ts:52](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L52) |
| `imageAssetVersionCount?` | `readonly` | `number` | The maximum number of image versions to store in a repository. Previous versions of an image can be stored for rollback purposes. Once a repository has more than 3 image versions stored, the oldest version will be discarded. This allows for sensible garbage collection while maintaining a few previous versions for rollback scenarios. **Default** `- up to 3 versions stored` | [synthesizer.ts:85](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L85) |
| `parent?` | `readonly` | `Resource` | The parent resource for any Pulumi resources created by the Synthesizer | [synthesizer.ts:90](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L90) |
| `stagingBucketName?` | `readonly` | `string` | Explicit name for the staging bucket **Default** `- a well-known name unique to this app/env.` | [synthesizer.ts:35](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L35) |
| `stagingStackNamePrefix?` | `readonly` | `string` | Specify a custom prefix to be used as the staging stack name and construct ID. The prefix will be appended before the appId, which is required to be part of the stack name and construct ID to ensure uniqueness. **Default** `'staging-stack'` | [synthesizer.ts:62](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L62) |
| `appId` | `readonly` | `string` | A unique identifier for the application that the staging stack belongs to. This identifier will be used in the name of staging resources created for this application, and should be unique across CDK apps. The identifier should include lowercase characters, numbers, periods (.) and dashes ('-') only and have a maximum of 17 characters. | [synthesizer.ts:29](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L29) |
| `autoDeleteStagingAssets?` | `readonly` | `boolean` | Auto deletes objects in the staging S3 bucket and images in the staging ECR repositories. This will also delete the S3 buckets and ECR repositories themselves when all objects / images are removed. **Default** `true` | [synthesizer.ts:74](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L74) |
| `deployTimeFileAssetLifetime?` | `readonly` | `Duration` | The lifetime for deploy time file assets. Assets that are only necessary at deployment time (for instance, CloudFormation templates and Lambda source code bundles) will be automatically deleted after this many days. Assets that may be read from the staging bucket during your application's run time will not be deleted. Set this to the length of time you wish to be able to roll back to previous versions of your application without having to do a new `cdk synth` and re-upload of assets. **Default** `- Duration.days(30)` | [synthesizer.ts:53](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L53) |
| `imageAssetVersionCount?` | `readonly` | `number` | The maximum number of image versions to store in a repository. Previous versions of an image can be stored for rollback purposes. Once a repository has more than 3 image versions stored, the oldest version will be discarded. This allows for sensible garbage collection while maintaining a few previous versions for rollback scenarios. **Default** `- up to 3 versions stored` | [synthesizer.ts:86](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L86) |
| `parent?` | `readonly` | `Resource` | The parent resource for any Pulumi resources created by the Synthesizer | [synthesizer.ts:91](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L91) |
| `stagingBucketName?` | `readonly` | `string` | Explicit name for the staging bucket **Default** `- a well-known name unique to this app/env.` | [synthesizer.ts:36](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L36) |
| `stagingStackNamePrefix?` | `readonly` | `string` | Specify a custom prefix to be used as the staging stack name and construct ID. The prefix will be appended before the appId, which is required to be part of the stack name and construct ID to ensure uniqueness. **Default** `'staging-stack'` | [synthesizer.ts:63](https://github.com/pulumi/pulumi-cdk/blob/main/src/synthesizer.ts#L63) |
2 changes: 1 addition & 1 deletion examples/s3-object-lambda/src/s3-object-lambda-stack.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ import * as s3 from 'aws-cdk-lib/aws-s3';
import * as s3ObjectLambda from 'aws-cdk-lib/aws-s3objectlambda';

// configurable variables
const S3_ACCESS_POINT_NAME = 'example-test-ap';
const S3_ACCESS_POINT_NAME = 'cdk-example-test-ap';
const OBJECT_LAMBDA_ACCESS_POINT_NAME = 's3-object-lambda-ap';

export class S3ObjectLambdaStack extends pulumicdk.Stack {
Expand Down
1 change: 0 additions & 1 deletion src/converters/app-converter.ts
Original file line number Diff line number Diff line change
Expand Up @@ -238,7 +238,6 @@ export class StackConverter extends ArtifactConverter implements intrinsics.Intr

private stackDependsOn(dependencies: Set<ArtifactConverter>): pulumi.Resource[] {
const dependsOn: pulumi.Resource[] = [];
dependsOn.push(...this.app.dependencies);
for (const d of dependencies) {
if (d instanceof StackConverter) {
dependsOn.push(d.stackResource);
Expand Down
Loading

0 comments on commit ace430d

Please sign in to comment.