diff --git a/aws-gov/README.md b/aws-gov/README.md index ed34ca2..62221d3 100644 --- a/aws-gov/README.md +++ b/aws-gov/README.md @@ -1,126 +1,118 @@ # Security Reference Architectures (SRA) - Terraform Templates +## Read Before Deploying -## Introduction +SRA is a purpose-built, simplified deployment pattern designed for highly secure and regulated customers. -Databricks has worked with thousands of customers to securely deploy the Databricks platform with appropriate security features to meet their architecture requirements. +This architecture includes specific functionalities that may affect certain use cases, as outlined below. -This Security Reference Architecture (SRA) repository implements common security features as a unified terraform templates that are typically deployed by our security conscious customers. +- **No outbound internet traffic**: There is no outbound internet from the classic compute plane, meaning there is no access to public package repositories, public APIs, and [Apache Derby](https://kb.databricks.com/metastore/set-up-embedded-metastore) configurations must be used on every classic compute cluster. + - To add packages to classic compute plane clusters, set up a private repository for scanned packages. + - Consider using a modern firewall solution to connect to public API endpoints. + - An example cluster is provided with the correct Apache Derby configurations. +- **Restrictive AWS Resource Policies**: Restrictive endpoint policies have been implemented for the workspace root storage bucket, S3 gateway endpoint, STS interface endpoint, and Kinesis endpoint. These restrictions are refined continuously as the product evolves. + - Policies can be adjusted to allow access to additional AWS resources, such as other S3 buckets. + - If you encounter unexpected product behavior due to a policy in this repository, please raise a Git issue. -## Component Breakdown and Description +- **Isolated Unity Catalog Securables**: Unity Catalog securables like catalogs, Storage Credentials, and External Locations are isolated to individual workspaces. + - To share securables between workspaces, update the resources using the [databricks_workspace_binding](https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/workspace_binding) resource. -In this section, we break down each of the components that we've included in this Security Reference Architecture. +## Customizations -In various `.tf` scripts, we have included direct links to the Databricks Terraform documentation. The [official documentation](https://registry.terraform.io/providers/databricks/databricks/latest/docs) can be found here. +Terraform customizations are available to support the baseline deployment of the Security Reference Architecture (SRA). These customizations are organized by provider: +- **Account**: Databricks account provider. +- **Workspace**: Databricks workspace provider. +- **Networking Configuration**: AWS provider. -## Operation Mode: +These extensions can be found in the top-level customization folder. -There are four separate operation modes you can choose for the underlying network configurations of your workspaces: **sandbox**, **firewall**, **isolated**, and **custom**. +## SRA Component Breakdown and Description -- **Sandbox**: Sandbox or open egress. Selecting 'sandbox' as the operation mode allows traffic to flow freely to the public internet. This mode is suitable for sandbox or development scenarios where data exfiltration protection is of minimal concern, and developers need to access public APIs, packages, and more. +In this section, we break down core components included in this Security Reference Architecture. -- **Firewall**: Firewall or limited egress. Choosing 'firewall' as the operation mode permits traffic flow only to a selected list of public addresses. This mode is applicable in situations where open internet access is necessary for certain tasks, but unfiltered traffic is not an option due to the sensitivity of the workloads or data. - - **WARNING**: Due to a limitation in AWS Network Firewall's support for fully qualified domain names (FQDNs) in non-HTTP/HTTPS traffic, an IP address is required to allow communication with the Hive Metastore. This dependency on a static IP introduces the potential for downtime if the Hive Metastore's IP changes. For sensitive production workloads, it is recommended to explore the isolated operation mode or consider alternative firewall solutions that provide better handling of dynamic IPs or FQDNs. +Various `.tf` scripts contain direct links to the Databricks Terraform documentation. You can find the [official documentation here](https://registry.terraform.io/providers/databricks/databricks/latest/docs). -- **Isolated**: Isolated or no egress. Opting for 'isolated' as the operation mode prevents any traffic to the public internet. Traffic is limited to AWS private endpoints, either to AWS services or the Databricks control plane. This mode should be used in cases where access to the public internet is completely unsupported. **NOTE**: Apache Derby Metastore will be required for clusters and non-serverless SQL Warehouses. For more information, please view this [knowledge article](https://kb.databricks.com/metastore/set-up-embedded-metastore). +### Network Configuration -- **Custom**: Custom or bring your own network. Selecting 'custom' allows you to input your own details for a VPC ID, subnet IDs, security group IDs, and PrivateLink endpoint IDs. This mode is recommended when networking assets are created in different pipelines or are pre-assigned to a team by a centralized infrastructure team. +Choose from two network configurations for your workspaces: **isolated** or **custom**. -See the below networking diagrams for more information. +- **Isolated (Default)**: Opting for 'isolated' prevents any traffic to the public internet, limiting traffic to AWS private endpoints for AWS services or the Databricks control plane. + - **NOTE**: Apache Derby Metastore will be required for clusters and non-serverless SQL Warehouses. For more information, view this [knowledge article](https://kb.databricks.com/metastore/set-up-embedded-metastore). +- **Custom**: Selecting 'custom' allows you to specify your own VPC ID, subnet IDs, security group IDs, and PrivateLink endpoint IDs. This mode is recommended when networking assets are created in different pipelines or pre-assigned by a centralized infrastructure team. -## Infrastructure Deployment +### Core AWS Components -- **Customer-managed VPC**: A [customer-managed VPC](https://docs.databricks.com/administration-guide/cloud-configurations/aws/customer-managed-vpc.html) allows Databricks customers to exercise more control over network configuration to comply with specific cloud security and governance standards that a customer's organization may require. +- **Customer-managed VPC**: A [customer-managed VPC](https://docs.databricks.com/administration-guide/cloud-configurations/aws/customer-managed-vpc.html) allows Databricks customers to exercise more control over network configurations to comply with specific cloud security and governance standards required by their organization. -- **AWS VPC Endpoints for S3, STS, and Kinesis**: Using AWS PrivateLink technology, a VPC endpoint is a service that connects a customer's VPC endpoint to AWS services without traversing public IP addresses. [S3, STS, and Kinesis endpoints](https://docs.databricks.com/administration-guide/cloud-configurations/aws/privatelink.html#step-5-add-vpc-endpoints-for-other-aws-services-recommended-but-optional) are best practices for standard enterprise Databricks deployments. Additional endpoints can be configured depending on use case (e.g. Amazon DynamoDB and AWS Glue). +- **S3 Buckets**: Two S3 buckets are created to support the following functionalities: + - [Workspace Root Bucket](https://docs.databricks.com/en/admin/account-settings-e2/storage.html) + - [Unity Catalog - Workspace Catalog](https://docs.databricks.com/en/catalogs/create-catalog.html) -- **Back-end AWS PrivateLink Connectivity**: AWS PrivateLink provides a private network route from one AWS environment to another. [Back-end PrivateLink](https://docs.databricks.com/administration-guide/cloud-configurations/aws/privatelink.html#overview) is configured so that communication between the customer's data plane and the Databricks control plane does not traverse public IP addresses. This is accomplished through Databricks specific interface VPC endpoints. Front-end PrivateLink is available as well for customers to ensure users traffic remains over the AWS backbone. However front-end PrivateLink is not included in this Terraform template. +- **IAM Roles**: Two IAM roles are created to support the following functionalities: + - [Classic Compute (EC2) Provisioning](https://docs.databricks.com/en/admin/account-settings-e2/credentials.html) + - [Data Access for Unity Catalog - Workspace Catalog](https://docs.databricks.com/en/connect/unity-catalog/cloud-storage/storage-credentials.html#step-1-create-an-iam-role) -- **Scoped-down IAM Policy for the Databricks cross-account role**: A [cross-account role](https://docs.databricks.com/administration-guide/account-api/iam-role.html) is needed for users, jobs, and other third-party tools to spin up Databricks clusters within the customer's data plane environment. This cross-account role can be scoped down to only function within the parameters of the data plane's VPC, subnets, and security group. +- **AWS VPC Endpoints for S3, STS, and Kinesis**: Using AWS PrivateLink, a VPC endpoint connects a customer's VPC endpoint to AWS services without traversing public IP addresses. [S3, STS, and Kinesis endpoints](https://docs.databricks.com/administration-guide/cloud-configurations/aws/privatelink.html#step-5-add-vpc-endpoints-for-other-aws-services-recommended-but-optional) are best practices for enterprise Databricks deployments. Additional endpoints can be configured based on your use case (e.g., Amazon DynamoDB and AWS Glue). -- **Restrictive Root Bucket**: Each workspace, prior to creation, registers a [dedicated S3 bucket](https://docs.databricks.com/administration-guide/account-api/aws-storage.html). This bucket is for workspace assets. On AWS, S3 bucket policies can be applied to limit access to the Databricks control plane and the customer data plane. +- **Back-end AWS PrivateLink Connectivity**: AWS PrivateLink provides a private network route from one AWS environment to another. [Back-end PrivateLink](https://docs.databricks.com/administration-guide/cloud-configurations/aws/privatelink.html#overview) is configured so communication between the customer's data plane and the Databricks control plane does not traverse public IP addresses. This is accomplished through Databricks-specific interface VPC endpoints. Front-end PrivateLink is also available for customers to keep user traffic over the AWS backbone, though front-end PrivateLink is not included in this Terraform template. -- **Unity Catalog**: [Unity Catalog](https://docs.databricks.com/data-governance/unity-catalog/index.html) is a unified governance solution for all data and AI assets including files, tables, and machine learning models. Unity Catalog provides a modern approach to granular access controls with centralized policy, auditing, and lineage tracking - all integrated into your Databricks workflow. **NOTE**: SRA creates a workspace specific catalog that is isolated to that individual workspace. To change these settings please update uc_catalog.tf under the workspace_security_modules. +- **Scoped-down IAM Policy for the Databricks cross-account role**: A [cross-account role](https://docs.databricks.com/administration-guide/account-api/iam-role.html) is needed for users, jobs, and other third-party tools to spin up Databricks clusters within the customer's data plane environment. This role can be scoped down to function only within the data plane's VPC, subnets, and security group. +- **AWS KMS Keys**: Three AWS KMS keys are created to support the following functionalities: + - [Workspace Storage](https://docs.databricks.com/en/security/keys/customer-managed-keys.html#customer-managed-keys-for-workspace-storage) + - [Managed Services](https://docs.databricks.com/en/security/keys/customer-managed-keys.html#customer-managed-keys-for-managed-services) + - [Unity Catalog - Workspace Catalog](https://docs.databricks.com/en/connect/unity-catalog/cloud-storage/manage-external-locations.html#configure-an-encryption-algorithm-on-an-external-location) -## Optional Deployment Configurations +### Core Databricks Components -- **Audit and Billable Usage Logs**: Databricks delivers logs to your S3 buckets. [Audit logs](https://docs.databricks.com/administration-guide/account-settings/audit-logs.html) contain two levels of events: workspace-level audit logs with workspace-level events and account-level audit logs with account-level events. In addition to these logs, you can generate additional events by enabling verbose audit logs. [Billable usage logs](https://docs.databricks.com/administration-guide/account-settings/billable-usage-delivery.html) are delivered daily to an AWS S3 storage bucket. There will be a separate CSV file for each workspace. This file contains historical data about the workspace's cluster usage in Databricks Units (DBUs). -- **System Tables Schemas**: System Tables provide visiblity into access, billing, compute, Lakeflow, and storage logs. These tables can be found within the system catalog in Unity Catalog. +- **Unity Catalog**: [Unity Catalog](https://docs.databricks.com/data-governance/unity-catalog/index.html) is a unified governance solution for data and AI assets, including files, tables, and machine learning models. It provides granular access controls with centralized policy, auditing, and lineage tracking—all integrated into the Databricks workflow. -- **Cluster Example**: An example of a cluster and a cluster policy has been included. **NOTE:** Please be aware this will create a cluster within your Databricks workspace including the underlying EC2 instance. +- **System Tables Schemas (COMING SOON TO AWS-GOV)**: [System Tables](https://docs.databricks.com/en/admin/system-tables/index.html) provide visibility into access, billing, compute, Lakeflow, query, serving, and storage logs. These tables can be found within the system catalog in Unity Catalog. -- **IP Access Lists**: IP Access can be enabled to only allow a subset of IPs to access the Databricks workspace console. **NOTE:** Please verify all of the IPs are correct prior to enabling this feature to prevent a lockout scenario. +- **Cluster Example**: An example cluster and cluster policy have been included with Derby Metastore configurations. **NOTE:** This will create a cluster within your Databricks workspace, including the underlying EC2 instance. -- **Read Only External Location**: This creates a read-only external location in Unity Catalog for a given bucket as well as the corresponding AWS IAM role. +--- -- **Restrictive Root Bucket**: A restrictive root bucket policy can be applied to the root bucket of the workspace. **NOTE:** Please be aware this bucket is updated frequently, however, may not contain prefixes for the latest product releases. +## Critical Next Steps -- **Restrictive Kinesis, STS, and S3 Endpoint Policies**: Restrictive policies for Kinesis, STS, and S3 endpoints can be added for Databricks specific assets. **NOTE:** Please be aware thse policies could be updated and may result in potentially breaking changes. If this is the case, we recommend removing the policy. +- **Implement a Front-End Mitigation Strategy**: + - [IP Access Lists](https://docs.databricks.com/en/security/network/front-end/ip-access-list.html): The Terraform code for enabling IP access lists can be found in the customization folder. + - [Front-End PrivateLink](https://docs.databricks.com/en/security/network/classic/privatelink.html#step-5-configure-internal-dns-to-redirect-user-requests-to-the-web-application-front-end). -- **System Tables**: System tables are a Databricks-hosted analytical store of your account’s operational data found in the system catalog. System tables can be used for historical observability across your account. This is currently in public preview, so is optional to enable or not. +- **Implement Single Sign-On, Multi-factor Authentication, SCIM Provisioning**: Most enterprise deployments enable [Single Sign-On (SSO)](https://docs.databricks.com/administration-guide/users-groups/single-sign-on/index.html) and multi-factor authentication (MFA). For user management, we recommend integrating [SCIM (System for Cross-domain Identity Management)](https://docs.databricks.com/dev-tools/api/latest/scim/index.html) with your account console. -- **Workspace Admin. Configurations**: Workspace administration configurations that can be enabled that align with security best practices. The Terraform resource is experimental, which is why it is optional. Documentation on each configuration is provided in the Terraform file. +--- +## Additional Security Recommendations -## Solution Accelerators +This section provides additional security recommendations to help maintain a strong security posture. These cannot always be configured into this Terraform script or may be specific to individual customers (e.g., SCIM, SSO, Front-End PrivateLink, etc.) -- **Security Analysis Tool (SAT)**: The Security Analysis Tool analyzes customer's Databricks account and workspace security configurations and provides recommendations that can help them follow Databricks' security best practices. This can be enabled into the workspace that is being created. **NOTE:** Please be aware this creates a cluster, a job, and a dashboard within your environment. - -- **Audit Log Alerting**: Audit Log Alerting, based on this [blog post](https://www.databricks.com/blog/improve-lakehouse-security-monitoring-using-system-tables-databricks-unity-catalog), creates 40+ SQL alerts to monitor for incidents based on a Zero Trust Architecture (ZTA) model. **NOTE:** Please be aware this creates a cluster, a job, and queries within your environment. - - -## Additional Security Recommendations and Opportunities - -In this section, we break down additional security recommendations and opportunities to maintain a strong security posture that either cannot be configured into this Terraform script or is very specific to individual customers (e.g. SCIM, SSO, Front-End PrivateLink, etc.) - -- **Segment Workspaces for Various Levels of Data Separation**: While Databricks has numerous capabilities for isolating different workloads, such as table ACLs and IAM passthrough for very sensitive workloads, the primary isolation method is to move sensitive workloads to a different workspace. This sometimes happens when a customer has very different teams (for example, a security team and a marketing team) who must both analyze different data in Databricks. - -- **Avoid Storing Production Datasets in Databricks File Store**: Because the DBFS root is accessible to all users in a workspace, all users can access any data stored here. It is important to instruct users to avoid using this location for storing sensitive data. The default location for managed tables in the Hive metastore on Databricks is the DBFS root; to prevent end users who create managed tables from writing to the DBFS root, declare a location on external storage when creating databases in the Hive metastore. - -- **Single Sign-On, Multi-factor Authentication, SCIM Provisioning**: Most production or enterprise deployments enable their workspaces to use [Single Sign-On (SSO)](https://docs.databricks.com/administration-guide/users-groups/single-sign-on/index.html) and multi-factor authentication (MFA). As users are added, changed, and deleted, we recommended customers integrate [SCIM (System for Cross-domain Identity Management)](https://docs.databricks.com/dev-tools/api/latest/scim/index.html)to their account console to sync these actions. - -- **Backup Assets from the Databricks Control Plane**: While Databricks does not offer disaster recovery services, many customers use Databricks capabilities, including the Account API, to create a cold (standby) workspace in another region. This can be done using various tools such as the Databricks [migration tool](https://github.com/databrickslabs/migrate), [Databricks sync](https://github.com/databrickslabs/databricks-sync), or the [Terraform exporter](https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/experimental-exporter) - -- **Regularly Restart Databricks Clusters**: When you restart a cluster, it gets the latest images for the compute resource containers and the VM hosts. It is particularly important to schedule regular restarts for long-running clusters such as those used for processing streaming data. If you enable the compliance security profile for your account or your workspace, long-running clusters are automatically restarted after 25 days. Databricks recommends that admins restart clusters manually during a scheduled maintenance window. This reduces the risk of an auto-restart disrupting a scheduled job. - -- **Evaluate Whether your Workflow requires using Git Repos or CI/CD**: Mature organizations often build production workloads by using CI/CD to integrate code scanning, better control permissions, perform linting, and more. When there is highly sensitive data analyzed, a CI/CD process can also allow scanning for known scenarios such as hard coded secrets. +- **Segment Workspaces for Data Separation**: This approach is particularly useful when teams such as security and marketing require distinct data access. +- **Avoid Storing Production Datasets in Databricks File Store**: The DBFS root is accessible to all users in a workspace. Specify a location on external storage when creating databases in the Hive metastore. +- **Backup Assets from the Databricks Control Plane**: Use tools such as the Databricks [migration tool](https://github.com/databrickslabs/migrate) or [Terraform exporter](https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/experimental-exporter). +- **Regularly Restart Databricks Clusters**: Restart clusters periodically to ensure the latest compute resource images are used. +- **Evaluate Your Workflow for Git Repos or CI/CD Needs**: Integrate CI/CD for code scanning, permission control, and sensitive data detection. +--- ## Getting Started -1. Clone this Repo -2. Install [Terraform](https://developer.hashicorp.com/terraform/downloads) -3. Decide which [operation](https://github.com/databricks/terraform-databricks-sra/tree/main/aws-gov/tf#operation-mode) mode you'd like to use. -4. Fill out `sra.tf` in place -5. Fill out `template.tfvars.example` remove the .example part of the file name -6. Configure the [AWS](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration) and [Databricks](https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication) provider authentication -7. CD into `tf` -8. Run `terraform init` -9. Run `terraform validate` -10. From `tf` directory, run `terraform plan -var-file ../example.tfvars` -11. Run `terraform apply -var-file ../example.tfvars` - - -## Network Diagram - Sandbox -![Architecture Diagram](https://github.com/databricks/terraform-databricks-sra/blob/main/aws-gov/img/Sandbox%20-%20Network%20Topology.png) - - -## Network Diagram - Firewall -![Architecture Diagram](https://github.com/databricks/terraform-databricks-sra/blob/main/aws-gov/img/Firewall%20-%20Network%20Topology.png) - - -## Network Diagram - Isolated -![Architecture Diagram](https://github.com/databricks/terraform-databricks-sra/blob/main/aws-gov/img/Isolated%20-%20Network%20Topology.png) - - -## FAQ +1. Clone this Repo. +2. Install [Terraform](https://developer.hashicorp.com/terraform/downloads). +3. Fill out `sra.tf`. +4. Fill out `template.tfvars.example` and rename the file to `template.tfvars` by removing `.example`. +5. Configure the [AWS](https://registry.terraform.io/providers/hashicorp/aws/latest/docs#authentication-and-configuration) and [Databricks](https://registry.terraform.io/providers/databricks/databricks/latest/docs#authentication) provider authentication. +6. Change directory into `tf`. +7. Run `terraform init`. +8. Run `terraform validate`. +9. From the `tf` directory, run `terraform plan -var-file ../example.tfvars`. +10. Run `terraform apply -var-file ../example.tfvars`. -- **I've cloned the GitHub repo, what's the recommended way to add Databricks additional resources to it?** +--- -If you'd like to add additional resources to the repository, the first step is to identify if this resource is using the **account** or **workspace** provider. +## Network Diagram -For example, if it uses the **account** provider, then we'd recommend creating a new module under the [modules/sra/databricks_account](https://github.com/databricks/terraform-databricks-sra/tree/main/aws-gov/tf/modules/sra/databricks_account) folder. Then, that module can be called in the top level [databricks_account.tf](https://github.com/databricks/terraform-databricks-sra/blob/main/aws-gov/tf/modules/sra/databricks_account.tf) file. This process is the same for the workspace provider by placing a new module in the [modules/sra/databricks_workspace folder](https://github.com/databricks/terraform-databricks-sra/tree/main/aws-gov/tf/modules/sra/databricks_workspace) and call it in the [databricks_workspace.tf](https://github.com/databricks/terraform-databricks-sra/blob/main/aws-gov/tf/modules/sra/databricks_workspace.tf) file. \ No newline at end of file +![Architecture Diagram](https://github.com/databricks/terraform-databricks-sra/blob/main/aws-gov/img/Isolated%20-%20Network%20Topology.png) \ No newline at end of file diff --git a/aws-gov/customizations/account/logging_configuration/logging_configuration.md b/aws-gov/customizations/account/logging_configuration/logging_configuration.md new file mode 100644 index 0000000..4ca3753 --- /dev/null +++ b/aws-gov/customizations/account/logging_configuration/logging_configuration.md @@ -0,0 +1,22 @@ +### Audit and Billable Usage Logs: +Databricks delivers logs to your S3 buckets. [Audit logs](https://docs.databricks.com/administration-guide/account-settings/audit-logs.html) contain two levels of events: workspace-level audit logs with workspace-level events, and account-level audit logs with account-level events. Additionally, you can generate more detailed events by enabling verbose audit logs. [Billable usage logs](https://docs.databricks.com/administration-guide/account-settings/billable-usage-delivery.html) are delivered daily to an AWS S3 storage bucket. A separate CSV file is created for each workspace, containing historical data about the workspace’s cluster usage in Databricks Units (DBUs). + +### How to add this resource to SRA: + +1. Copy the `logging_configuration` folder into `modules/sra/databricks_account/` +2. Add the following code block into `modules/sra/databricks_account.tf` +``` +module "log_delivery" { + source = "./databricks_account/logging_configuration" + providers = { + databricks = databricks.mws + } + + databricks_account_id = var.databricks_account_id + resource_prefix = var.resource_prefix +} +``` +3. Run `terraform init` +4. Run `terraform validate` +5. From `tf` directory, run `terraform plan -var-file ../example.tfvars` +6. Run `terraform apply -var-file ../example.tfvars` diff --git a/aws-gov/tf/modules/sra/databricks_account/logging_configuration/logging_configuration.tf b/aws-gov/customizations/account/logging_configuration/logging_configuration.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_account/logging_configuration/logging_configuration.tf rename to aws-gov/customizations/account/logging_configuration/logging_configuration.tf diff --git a/aws-gov/tf/modules/sra/databricks_account/logging_configuration/provider.tf b/aws-gov/customizations/account/logging_configuration/provider.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_account/logging_configuration/provider.tf rename to aws-gov/customizations/account/logging_configuration/provider.tf diff --git a/aws-gov/tf/modules/sra/databricks_account/logging_configuration/variables.tf b/aws-gov/customizations/account/logging_configuration/variables.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_account/logging_configuration/variables.tf rename to aws-gov/customizations/account/logging_configuration/variables.tf diff --git a/aws-gov/customizations/customizations.md b/aws-gov/customizations/customizations.md new file mode 100644 index 0000000..3765250 --- /dev/null +++ b/aws-gov/customizations/customizations.md @@ -0,0 +1,21 @@ +# Customizations + +Customizations are **Terraform code** available to support the baseline deployment of the **Security Reference Architecture (SRA) - Terraform Templates**. + +Customizations are sectioned by providers: +- **Account**: Databricks account provider. +- **Workspace**: Databricks workspace provider. +- **Networking Configuration**: AWS provider. + +The current customizations available are: + +| Provider | Customization | Summary | +|-----------------------------|-------------------------------|---------| +| **Account** | **Audit and Billable Usage Logs** | Databricks delivers logs to your S3 buckets. [Audit logs](https://docs.databricks.com/administration-guide/account-settings/audit-logs.html) contain workspace and account-level events. Additionally, [Billable usage logs](https://docs.databricks.com/administration-guide/account-settings/billable-usage-delivery.html) are delivered daily to an AWS S3 bucket, containing historical data about cluster usage in DBUs for each workspace. | +| **Workspace** | **Workspace Admin Configurations** | Workspace administration configurations can be enabled to align with security best practices. The Terraform resource is experimental and optional, with documentation on each configuration provided in the Terraform file. | +| **Workspace** | **IP Access Lists** | IP Access can be enabled to restrict console access to a subset of IPs. **NOTE:** Ensure IPs are accurate to prevent lockout scenarios. | +| **Workspace** | **Security Analysis Tool (SAT)** | The Security Analysis Tool evaluates a customer’s Databricks account and workspace security configurations, providing recommendations that align with Databricks’ best practices. This can be enabled within the workspace. | +| **Workspace** | **Audit Log Alerting** | Based on this [blog post](https://www.databricks.com/blog/improve-lakehouse-security-monitoring-using-system-tables-databricks-unity-catalog), Audit Log Alerting creates 40+ SQL alerts to monitor incidents following a Zero Trust Architecture (ZTA) model. **NOTE:** This configuration creates a cluster, a job, and queries within your environment. | +| **Workspace** | **Read-Only External Location** | Creates a read-only external location in Unity Catalog for a specified bucket, as well as the corresponding AWS IAM role. | +| **AWS** | **Firewall (Limited Egress)** | This firewall networking configuration restricts traffic to a specified list of public addresses, suitable for situations where open internet access is limited due to workload or data sensitivity. | +| **AWS** | **Sandbox (Open Egress)** | Open egress allows free traffic flow to the public internet, ideal for sandbox or development scenarios where data exfiltration protection is minimal, and developers need public API access. | \ No newline at end of file diff --git a/aws-gov/img/Firewall - Network Topology.png b/aws-gov/customizations/networking_configurations/firewall/Firewall - Network Topology.png similarity index 100% rename from aws-gov/img/Firewall - Network Topology.png rename to aws-gov/customizations/networking_configurations/firewall/Firewall - Network Topology.png diff --git a/aws-gov/img/Firewall - VPC Resource Map Example.png b/aws-gov/customizations/networking_configurations/firewall/Firewall - VPC Resource Map Example.png similarity index 100% rename from aws-gov/img/Firewall - VPC Resource Map Example.png rename to aws-gov/customizations/networking_configurations/firewall/Firewall - VPC Resource Map Example.png diff --git a/aws-gov/customizations/networking_configurations/firewall/firewall.md b/aws-gov/customizations/networking_configurations/firewall/firewall.md new file mode 100644 index 0000000..96d56c0 --- /dev/null +++ b/aws-gov/customizations/networking_configurations/firewall/firewall.md @@ -0,0 +1,66 @@ +### Firewall (Limited Egress) +Using a firewall networking configuration restricts traffic flow to a specified list of public addresses. This setup is applicable in situations where open internet access is necessary for certain tasks, but unfiltered traffic is not an option due to workload or data sensitivity. + +- **WARNING**: Due to a limitation in AWS Network Firewall's support for fully qualified domain names (FQDNs) in non-HTTP/HTTPS traffic, an IP address is required to allow communication with the Hive Metastore. This reliance on a static IP introduces the potential for downtime if the Hive Metastore's IP changes. For sensitive production workloads, it is recommended to explore the isolated operation mode or consider alternative firewall solutions that provide better handling of dynamic IPs or FQDNs. + +Please note that a public and firewall subnet CIDR ranges are required. + +### How to use this network configuration: +1. Replace the VPC module (lines 4-28) in `modules/sra/network.tf` with the following code block: +``` +module "vpc" { + source = "terraform-aws-modules/vpc/aws" + version = "5.1.1" + + count = 1 + + name = "${var.resource_prefix}-classic-compute-plane-vpc" + cidr = var.vpc_cidr_range + azs = var.availability_zones + + enable_dns_hostnames = true + enable_nat_gateway = false + single_nat_gateway = false + one_nat_gateway_per_az = false + create_igw = false + + public_subnet_names = [] + public_subnets = [] + + private_subnet_names = [for az in var.availability_zones : format("%s-private-%s", var.resource_prefix, az)] + private_subnets = var.private_subnets_cidr + + intra_subnet_names = [for az in var.availability_zones : format("%s-privatelink-%s", var.resource_prefix, az)] + intra_subnets = var.privatelink_subnets_cidr + + tags = { + Project = var.resource_prefix + } +} +``` +2. Create a new directory `modules/sra/firewall` +3. Copy the `firewall` directory from `networking_configurations/firewall` into the new directory. +3. Create a new file `modules/sra/firewall.tf` +4. Add the following code block into `modules/sra/firewall.tf` +``` +module "harden_firewall" { + source = "./firewall" + providers = { + aws = aws + } + + vpc_id = module.vpc[0].vpc_id + vpc_cidr_range = var.vpc_cidr_range + public_subnets_cidr = + private_subnets_cidr = module.vpc[0].private_subnets_cidr_blocks + private_subnet_rt = module.vpc[0].private_route_table_ids + firewall_subnets_cidr = + firewall_allow_list = + hive_metastore_fqdn = var.hive_metastore_fqdn[var.databricks_gov_shard] // + availability_zones = var.availability_zones + region = var.region + resource_prefix = var.resource_prefix + + depends_on = [module.databricks_mws_workspace] +} +``` diff --git a/aws-gov/tf/modules/sra/data_plane_hardening/firewall/firewall.tf b/aws-gov/customizations/networking_configurations/firewall/firewall.tf similarity index 98% rename from aws-gov/tf/modules/sra/data_plane_hardening/firewall/firewall.tf rename to aws-gov/customizations/networking_configurations/firewall/firewall.tf index 18536c0..0f087f3 100644 --- a/aws-gov/tf/modules/sra/data_plane_hardening/firewall/firewall.tf +++ b/aws-gov/customizations/networking_configurations/firewall/firewall.tf @@ -187,7 +187,7 @@ resource "aws_networkfirewall_rule_group" "databricks_fqdn_allowlist" { } data "dns_a_record_set" "metastore_dns" { - host = var.hive_metastore_fqdn + host = var.hive_metastore_fqdn[var.databricks_gov_shard] } // JDBC Firewall group IP allow list @@ -205,7 +205,7 @@ resource "aws_networkfirewall_rule_group" "databricks_metastore_allowlist" { content { action = "PASS" header { - destination = stateful_rule.value + destination = stateful_rule.value destination_port = 3306 direction = "FORWARD" protocol = "TCP" diff --git a/aws-gov/tf/modules/sra/data_plane_hardening/firewall/provider.tf b/aws-gov/customizations/networking_configurations/firewall/provider.tf similarity index 100% rename from aws-gov/tf/modules/sra/data_plane_hardening/firewall/provider.tf rename to aws-gov/customizations/networking_configurations/firewall/provider.tf diff --git a/aws-gov/tf/modules/sra/data_plane_hardening/firewall/variables.tf b/aws-gov/customizations/networking_configurations/firewall/variables.tf similarity index 66% rename from aws-gov/tf/modules/sra/data_plane_hardening/firewall/variables.tf rename to aws-gov/customizations/networking_configurations/firewall/variables.tf index 2d8b5b8..1b4205e 100644 --- a/aws-gov/tf/modules/sra/data_plane_hardening/firewall/variables.tf +++ b/aws-gov/customizations/networking_configurations/firewall/variables.tf @@ -11,7 +11,11 @@ variable "firewall_subnets_cidr" { } variable "hive_metastore_fqdn" { - type = string + type = map(string) + default = { + "civilian" = "discovery-search-rds-prod-dbdiscoverysearch-uus7j2cyyu1m.c40ji7ukhesx.us-gov-west-1.rds.amazonaws.com" + "dod" = "lineage-usgovwest1dod-prod.cpnejponioft.us-gov-west-1.rds.amazonaws.com" + } } variable "private_subnet_rt" { @@ -40,4 +44,8 @@ variable "vpc_cidr_range" { variable "vpc_id" { type = string +} + +variable "databricks_gov_shard" { + type = string } \ No newline at end of file diff --git a/aws-gov/img/Sandbox - Network Topology.png b/aws-gov/customizations/networking_configurations/sandbox/Sandbox - Network Topology.png similarity index 100% rename from aws-gov/img/Sandbox - Network Topology.png rename to aws-gov/customizations/networking_configurations/sandbox/Sandbox - Network Topology.png diff --git a/aws-gov/img/Sandbox - VPC Resource Map Example.png b/aws-gov/customizations/networking_configurations/sandbox/Sandbox - VPC Resource Map Example.png similarity index 100% rename from aws-gov/img/Sandbox - VPC Resource Map Example.png rename to aws-gov/customizations/networking_configurations/sandbox/Sandbox - VPC Resource Map Example.png diff --git a/aws-gov/customizations/networking_configurations/sandbox/sandbox.md b/aws-gov/customizations/networking_configurations/sandbox/sandbox.md new file mode 100644 index 0000000..c8c0309 --- /dev/null +++ b/aws-gov/customizations/networking_configurations/sandbox/sandbox.md @@ -0,0 +1,41 @@ +### Sandbox (Open Egress) +Using open egress allows traffic to flow freely to the public internet. This networking configuration is suitable for sandbox or development scenarios where data exfiltration protection is of minimal concern, and developers need access to public APIs, packages, and more. + +Please note that a public subnet CIDR range is required. + + +### How to use this network configuration: + +1. Replace the VPC module (lines 4-28) in `modules/sra/network.tf` with the following code block: +``` +module "vpc" { + source = "terraform-aws-modules/vpc/aws" + version = "5.1.1" + + count = 1 + + name = "${var.resource_prefix}-classic-compute-plane-vpc" + cidr = var.vpc_cidr_range + azs = var.availability_zones + + enable_dns_hostnames = true + enable_nat_gateway = true + single_nat_gateway = false + one_nat_gateway_per_az = true + create_igw = true + + public_subnet_names = [for az in var.availability_zones : format("%s-public-%s", var.resource_prefix, az)] + public_subnets = + + private_subnet_names = [for az in var.availability_zones : format("%s-private-%s", var.resource_prefix, az)] + private_subnets = var.private_subnets_cidr + + intra_subnet_names = [for az in var.availability_zones : format("%s-privatelink-%s", var.resource_prefix, az)] + intra_subnets = var.privatelink_subnets_cidr + + tags = { + Project = var.resource_prefix + } +} + +``` \ No newline at end of file diff --git a/aws-gov/customizations/workspace/admin_configuration/admin_configuration.md b/aws-gov/customizations/workspace/admin_configuration/admin_configuration.md new file mode 100644 index 0000000..0000122 --- /dev/null +++ b/aws-gov/customizations/workspace/admin_configuration/admin_configuration.md @@ -0,0 +1,19 @@ +### **Workspace Admin. Configurations**: +Workspace administration configurations can be enabled to align with security best practices. This Terraform resource is experimental, making it optional. Documentation for each configuration is provided in the Terraform file. + +### How to add this resource to SRA: + +1. Copy the `admin_configuration` folder into `modules/sra/databricks_workspace/` +2. Add the following code block into `modules/sra/databricks_workspace.tf` +``` +module "admin_configuration" { + source = "./databricks_workspace/admin_configuration" + providers = { + databricks = databricks.created_workspace + } +} +``` +3. Run `terraform init` +4. Run `terraform validate` +5. From `tf` directory, run `terraform plan -var-file ../example.tfvars` +6. Run `terraform apply -var-file ../example.tfvars` \ No newline at end of file diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/admin_configuration/admin_configuration.tf b/aws-gov/customizations/workspace/admin_configuration/admin_configuration.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/admin_configuration/admin_configuration.tf rename to aws-gov/customizations/workspace/admin_configuration/admin_configuration.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/admin_configuration/provider.tf b/aws-gov/customizations/workspace/admin_configuration/provider.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/admin_configuration/provider.tf rename to aws-gov/customizations/workspace/admin_configuration/provider.tf diff --git a/aws-gov/customizations/workspace/ip_access_list/ip_access_list.md b/aws-gov/customizations/workspace/ip_access_list/ip_access_list.md new file mode 100644 index 0000000..914e25d --- /dev/null +++ b/aws-gov/customizations/workspace/ip_access_list/ip_access_list.md @@ -0,0 +1,23 @@ +### IP Access Lists +IP Access can be enabled to restrict access to the Databricks workspace console to a specified subset of IPs. +- **NOTE:** Verify that all IPs are correct before enabling this feature to prevent a lockout scenario. + + +### How to add this resource to SRA: + +1. Copy the `ip_access_list` folder into `modules/sra/databricks_workspace/` +2. Add the following code block into `modules/sra/databricks_workspace.tf` +``` +module "ip_access_list" { + source = "./databricks_workspace/ip_access_list" + providers = { + databricks = databricks.created_workspace + } + + ip_addresses = +} +``` +3. Run `terraform init` +4. Run `terraform validate` +5. From `tf` directory, run `terraform plan -var-file ../example.tfvars` +6. Run `terraform apply -var-file ../example.tfvars` diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/ip_access_list/ip_access_list.tf b/aws-gov/customizations/workspace/ip_access_list/ip_access_list.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/ip_access_list/ip_access_list.tf rename to aws-gov/customizations/workspace/ip_access_list/ip_access_list.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/cluster_configuration/provider.tf b/aws-gov/customizations/workspace/ip_access_list/provider.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/cluster_configuration/provider.tf rename to aws-gov/customizations/workspace/ip_access_list/provider.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/ip_access_list/variables.tf b/aws-gov/customizations/workspace/ip_access_list/variables.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/ip_access_list/variables.tf rename to aws-gov/customizations/workspace/ip_access_list/variables.tf diff --git a/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/aws/TERRAFORM_AWS.md b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/aws/TERRAFORM_AWS.md new file mode 100644 index 0000000..e5d5d26 --- /dev/null +++ b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/aws/TERRAFORM_AWS.md @@ -0,0 +1,87 @@ +## Setting up Terraform + +> **SAT v0.2.0 or higher** brings full support for Unity Catalog. Now you can pick your catalog instead of hive_metastore. Plus, you get to choose your own schema name. + +Step 1: [Install Terraform](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli) + +Step 2: [Install Git](https://git-scm.com/book/en/v2/Getting-Started-Installing-Git) on local machine + +Step 3: Git Clone Repo + + ```sh + git clone https://github.com/databricks-industry-solutions/security-analysis-tool.git + ``` + +Step 4: Change Directories + + ```sh + cd security-analysis-tool/terraform// + ``` + +Step 5: Generate a `terraform.tfvars` file base on `template.tfvars` + +Using any editor set the values in the `terraform.tfvars` file. The descriptions of all the variables are located in the variables.tf file. Once the variables are set you are ready to run Terraform. + +Further Documentation for some of the variables: + +[workspace_id](https://docs.databricks.com/workspace/workspace-details.html#workspace-instance-names-urls-and-ids) + +[account_console_id](https://docs.databricks.com/administration-guide/account-settings/index.html#locate-your-account-id) + +> **Proxies are now supported as part of SAT. You can add your HTTP and HTTPS links to use your proxies.** +```json +{ + "http": "http://example.com", + "https": "https://example.com" +} +``` + +## Run Terraform + +Step 6: Terraform [Init](https://developer.hashicorp.com/terraform/cli/commands/init) + +The terraform init command initializes a working directory containing configuration files and installs plugins for required providers. + + ```sh + terraform init + ``` + +Step 7: Terraform [Plan](https://developer.hashicorp.com/terraform/cli/commands/plan) + +The terraform plan command creates an execution plan, which lets you preview the changes that Terraform plans to make to your infrastructure. By default, when Terraform creates a plan it: + +* Reads the current state of any already-existing remote objects to make sure that the Terraform state is up-to-date. +* Compares the current configuration to the prior state and noting any differences. +* Proposes a set of change actions that should, if applied, make the remote objects match the configuration. + + ```sh + terraform plan + ``` + +Step 8: Terraform [Apply](https://developer.hashicorp.com/terraform/cli/commands/apply) + +The terraform apply command executes the actions proposed in a Terraform plan. + + ``` + terraform apply + ``` + +Step 9: Run Jobs + +You now have two jobs (SAT Initializer Notebook & SAT Driver Notebook). Run SAT Initializer Notebook and when it completes run SAT Driver Notebook; SAT Initializer Notebook should only be run once (although you can run it multiple times, it only needs to be run successfully one time), and SAT Driver Notebook can be run periodically (its scheduled to run once every Monday, Wednesday, and Friday). + +Step 10: SAT Dashboard + +Go to the SQL persona, select the Dashboard icon in the left menu and then select the SAT Dashboard. Once the dashboard loads pick the Workspace from the dropdown and refresh the dashboard + +Supplemental Documentation: + +[Databricks Documentation Terraform](https://docs.databricks.com/dev-tools/terraform/index.html) + +[Databricks Terraform Provider Docs](https://registry.terraform.io/providers/databricks/databricks/latest/docs) + +Additional Considerations: + +Your jobs may fail if there was a pre-existing secret scope named sat_scope when you run terraform apply. To remedy this, you will need to change the name of your secret scope in secrets.tf, re-run terraform apply, and then navigate to Workspace -> Applications -> SAT-TF ->/notebooks/Utils/initialize and change the secret scope name in 6 places (3 times in CMD 4 and 3 times in CMD 5). You then can re-run your failed jobs. + +Congratulations!!! [Please review the setup documentation for the instructions on usage, FAQs and general understanding of SAT setup](https://github.com/databricks-industry-solutions/security-analysis-tool/blob/main/docs/setup.md) diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/aws/provider.tf b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/aws/provider.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/aws/provider.tf rename to aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/aws/provider.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/aws/secrets.tf b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/aws/secrets.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/aws/secrets.tf rename to aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/aws/secrets.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/aws/variables.tf b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/aws/variables.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/aws/variables.tf rename to aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/aws/variables.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/data.tf b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/data.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/data.tf rename to aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/data.tf diff --git a/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/jobs.tf b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/jobs.tf new file mode 100644 index 0000000..dcebec4 --- /dev/null +++ b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/jobs.tf @@ -0,0 +1,40 @@ +resource "databricks_job" "initializer" { + name = "SAT Initializer Notebook (one-time)" + environment { + spec { + dependencies = ["dbl-sat-sdk"] + client = "1" + } + environment_key = "Default" + } + task { + task_key = "Initializer" + notebook_task { + notebook_path = "${databricks_repo.security_analysis_tool.workspace_path}/notebooks/security_analysis_initializer" + } + } +} + +resource "databricks_job" "driver" { + name = "SAT Driver Notebook" + environment { + spec { + dependencies = ["dbl-sat-sdk"] + client = "1" + } + environment_key = "Default" + } + task { + task_key = "Driver" + notebook_task { + notebook_path = "${databricks_repo.security_analysis_tool.workspace_path}/notebooks/security_analysis_driver" + } + } + + schedule { + #E.G. At 08:00:00am, on every Monday, Wednesday and Friday, every month; For more: http://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html + quartz_cron_expression = "0 0 8 ? * Mon,Wed,Fri" + # The system default is UTC; For more: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones + timezone_id = "America/New_York" + } +} diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/outputs.tf b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/outputs.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/outputs.tf rename to aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/outputs.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/provider.tf b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/provider.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/provider.tf rename to aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/provider.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/repo.tf b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/repo.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/repo.tf rename to aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/repo.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/secrets.tf b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/secrets.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/secrets.tf rename to aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/secrets.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/sql_warehouse.tf b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/sql_warehouse.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/sql_warehouse.tf rename to aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/sql_warehouse.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/variables.tf b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/variables.tf similarity index 86% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/variables.tf rename to aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/variables.tf index 150ac5a..e98b576 100644 --- a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/variables.tf +++ b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/common/variables.tf @@ -29,12 +29,6 @@ variable "notification_email" { default = "" } -variable "gcp_impersonate_service_account" { - type = string - description = "GCP Service Account to impersonate (e.g. xyz-sa-2@project.iam.gserviceaccount.com)" - default = "" -} - variable "analysis_schema_name" { type = string description = "Name of the schema to be used for analysis" diff --git a/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/security_analysis_tool.md b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/security_analysis_tool.md new file mode 100644 index 0000000..e1590bd --- /dev/null +++ b/aws-gov/customizations/workspace/solution_accelerators/security_analysis_tool/security_analysis_tool.md @@ -0,0 +1,34 @@ +### Security Analysis Tool (SAT): +The Security Analysis Tool analyzes a customer’s Databricks account and workspace security configurations and provides recommendations to help them follow Databricks’ security best practices. This tool can be enabled in the workspace being created. + +- **NOTE:** Enabling this tool will create a cluster, a job, and a dashboard within your environment. + +### How to add this resource to SRA: + +1. Copy the `security_analysis_tool` folder into `modules/sra/databricks_workspace/` +2. Add the following code block into `modules/sra/databricks_workspace.tf` +``` +module "security_analysis_tool" { + source = "./databricks_workspace/security_analysis_tool/aws" + providers = { + databricks = databricks.created_workspace + } + + databricks_url = module.databricks_mws_workspace.workspace_url + workspace_id = module.databricks_mws_workspace.workspace_id + account_console_id = var.databricks_account_id + client_id = var.client_id + client_secret = var.client_secret + use_sp_auth = true + proxies = {} + analysis_schema_name = "SAT" + + depends_on = [ + module.databricks_mws_workspace + ] +} +``` +3. Run `terraform init` +4. Run `terraform validate` +5. From `tf` directory, run `terraform plan -var-file ../example.tfvars` +6. Run `terraform apply -var-file ../example.tfvars` \ No newline at end of file diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/system_tables_audit_log/job.tf b/aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/job.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/system_tables_audit_log/job.tf rename to aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/job.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/system_tables_audit_log/main.tf b/aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/main.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/system_tables_audit_log/main.tf rename to aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/main.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/system_tables_audit_log/provider.tf b/aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/provider.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/system_tables_audit_log/provider.tf rename to aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/provider.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/system_tables_audit_log/queries_and_alerts.json b/aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/queries_and_alerts.json similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/system_tables_audit_log/queries_and_alerts.json rename to aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/queries_and_alerts.json diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/system_tables_audit_log/sql.tf b/aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/sql.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/system_tables_audit_log/sql.tf rename to aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/sql.tf diff --git a/aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/system_table_audit_log.md b/aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/system_table_audit_log.md new file mode 100644 index 0000000..736407e --- /dev/null +++ b/aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/system_table_audit_log.md @@ -0,0 +1,27 @@ +#### Audit Log Alerting: +Audit Log Alerting, based on this [blog post](https://www.databricks.com/blog/improve-lakehouse-security-monitoring-using-system-tables-databricks-unity-catalog), creates 40+ SQL alerts to monitor incidents using a Zero Trust Architecture (ZTA) model. + +- **NOTE:** Enabling this feature will create a cluster, a job, and queries within your environment. + +### How to add this resource to SRA: + +1. Copy the `system_tables_audit_log` folder into `modules/sra/databricks_workspace/` +2. Add the following code block into `modules/sra/databricks_workspace.tf` +``` +module "system_tables_audit_log" { + source = "./databricks_workspace/system_tables_audit_log/" + providers = { + databricks = databricks.created_workspace + } + + alert_emails = [var.admin_user] + + depends_on = [ + module.databricks_mws_workspace, module.uc_assignment + ] +} +``` +3. Run `terraform init` +4. Run `terraform validate` +5. From `tf` directory, run `terraform plan -var-file ../example.tfvars` +6. Run `terraform apply -var-file ../example.tfvars` \ No newline at end of file diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/system_tables_audit_log/variables.tf b/aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/variables.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/system_tables_audit_log/variables.tf rename to aws-gov/customizations/workspace/solution_accelerators/system_tables_audit_log/variables.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/system_schema/provider.tf b/aws-gov/customizations/workspace/uc_external_location_read_only/provider.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/system_schema/provider.tf rename to aws-gov/customizations/workspace/uc_external_location_read_only/provider.tf diff --git a/aws-gov/customizations/workspace/uc_external_location_read_only/uc_external_location_read_only.md b/aws-gov/customizations/workspace/uc_external_location_read_only/uc_external_location_read_only.md new file mode 100644 index 0000000..9686464 --- /dev/null +++ b/aws-gov/customizations/workspace/uc_external_location_read_only/uc_external_location_read_only.md @@ -0,0 +1,26 @@ +### Read-Only External Location +This creates a read-only external location in Unity Catalog for a specified bucket, along with the corresponding AWS IAM role. + + +### How to add this resource to SRA: + +1. Copy the `uc_external_location_read_only` folder into `modules/sra/databricks_workspace/` +2. Add the following code block into `modules/sra/databricks_workspace.tf` +``` +module "uc_external_location_read_only" { + source = "./databricks_workspace/uc_external_location_read_only" + providers = { + databricks = databricks.created_workspace + } + + databricks_account_id = var.databricks_account_id + aws_account_id = var.aws_account_id + resource_prefix = var.resource_prefix + read_only_data_bucket = + read_only_external_location_admin = +} +``` +3. Run `terraform init` +4. Run `terraform validate` +5. From `tf` directory, run `terraform plan -var-file ../example.tfvars` +6. Run `terraform apply -var-file ../example.tfvars` \ No newline at end of file diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/uc_external_location/uc_external_location.tf b/aws-gov/customizations/workspace/uc_external_location_read_only/uc_external_location_read_only.tf similarity index 95% rename from aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/uc_external_location/uc_external_location.tf rename to aws-gov/customizations/workspace/uc_external_location_read_only/uc_external_location_read_only.tf index 3404dab..80f9fd5 100644 --- a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/uc_external_location/uc_external_location.tf +++ b/aws-gov/customizations/workspace/uc_external_location_read_only/uc_external_location_read_only.tf @@ -1,9 +1,18 @@ resource "null_resource" "previous" {} -resource "time_sleep" "wait_30_seconds" { +resource "time_sleep" "wait_60_seconds" { depends_on = [null_resource.previous] - create_duration = "30s" + create_duration = "60s" +} + +// Storage Credential +resource "databricks_storage_credential" "external" { + name = aws_iam_role.storage_credential_role.name + aws_iam_role { + role_arn = aws_iam_role.storage_credential_role.arn + } + isolation_mode = "ISOLATION_MODE_ISOLATED" } // Storage Credential Trust Policy @@ -86,16 +95,6 @@ resource "aws_iam_role_policy" "storage_credential_policy" { ) } -// Storage Credential -resource "databricks_storage_credential" "external" { - name = aws_iam_role.storage_credential_role.name - aws_iam_role { - role_arn = aws_iam_role.storage_credential_role.arn - } - isolation_mode = "ISOLATION_MODE_ISOLATED" - depends_on = [aws_iam_role.storage_credential_role, time_sleep.wait_30_seconds] -} - // External Location resource "databricks_external_location" "data_example" { name = "external-location-example" @@ -104,6 +103,7 @@ resource "databricks_external_location" "data_example" { read_only = true comment = "Read only external location for ${var.read_only_data_bucket}" isolation_mode = "ISOLATION_MODE_ISOLATED" + depends_on = [time_sleep.wait_60_seconds] } // External Location Grant diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/uc_external_location/variables.tf b/aws-gov/customizations/workspace/uc_external_location_read_only/variables.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/uc_external_location/variables.tf rename to aws-gov/customizations/workspace/uc_external_location_read_only/variables.tf diff --git a/aws-gov/tf/modules/sra/credential.tf b/aws-gov/tf/modules/sra/credential.tf index 476a0be..ee3469f 100644 --- a/aws-gov/tf/modules/sra/credential.tf +++ b/aws-gov/tf/modules/sra/credential.tf @@ -63,6 +63,31 @@ resource "aws_iam_role_policy" "cross_account" { "*" ] }, + { + "Sid" : "FleetPermissions", + "Effect" : "Allow", + "Action" : [ + "ec2:DescribeFleetHistory", + "ec2:ModifyFleet", + "ec2:DeleteFleets", + "ec2:DescribeFleetInstances", + "ec2:DescribeFleets", + "ec2:CreateFleet", + "ec2:DeleteLaunchTemplate", + "ec2:GetLaunchTemplateData", + "ec2:CreateLaunchTemplate", + "ec2:DescribeLaunchTemplates", + "ec2:DescribeLaunchTemplateVersions", + "ec2:ModifyLaunchTemplate", + "ec2:DeleteLaunchTemplateVersions", + "ec2:CreateLaunchTemplateVersion", + "ec2:AssignPrivateIpAddresses", + "ec2:GetSpotPlacementScores" + ], + "Resource" : [ + "*" + ] + }, { "Sid" : "InstancePoolsSupport", "Effect" : "Allow", diff --git a/aws-gov/tf/modules/sra/data_plane_hardening.tf b/aws-gov/tf/modules/sra/data_plane_hardening.tf deleted file mode 100644 index cbb2cda..0000000 --- a/aws-gov/tf/modules/sra/data_plane_hardening.tf +++ /dev/null @@ -1,42 +0,0 @@ -// EXPLANATION: Optional modules that harden the AWS data plane - -// Implement an AWS Firewall -module "harden_firewall" { - count = var.operation_mode == "firewall" ? 1 : 0 - source = "./data_plane_hardening/firewall" - providers = { - aws = aws - } - - vpc_id = module.vpc[0].vpc_id - vpc_cidr_range = var.vpc_cidr_range - public_subnets_cidr = var.public_subnets_cidr - private_subnets_cidr = module.vpc[0].private_subnets_cidr_blocks - private_subnet_rt = module.vpc[0].private_route_table_ids - firewall_subnets_cidr = var.firewall_subnets_cidr - firewall_allow_list = var.firewall_allow_list - hive_metastore_fqdn = var.hms_fqdn[var.databricks_gov_shard] - availability_zones = var.availability_zones - region = var.region - resource_prefix = var.resource_prefix - - depends_on = [module.databricks_mws_workspace] -} - - -// Restrictive DBFS bucket policy -module "restrictive_root_bucket" { - count = var.enable_restrictive_root_bucket_boolean ? 1 : 0 - source = "./data_plane_hardening/restrictive_root_bucket" - providers = { - aws = aws - } - - workspace_id = module.databricks_mws_workspace.workspace_id - region_name = var.region_name - root_s3_bucket = "${var.resource_prefix}-workspace-root-storage" - databricks_gov_shard = var.databricks_gov_shard - databricks_prod_aws_account_id = var.databricks_prod_aws_account_id - - depends_on = [module.databricks_mws_workspace] -} \ No newline at end of file diff --git a/aws-gov/tf/modules/sra/databricks_account.tf b/aws-gov/tf/modules/sra/databricks_account.tf index bc03e38..97d8291 100644 --- a/aws-gov/tf/modules/sra/databricks_account.tf +++ b/aws-gov/tf/modules/sra/databricks_account.tf @@ -1,20 +1,5 @@ // EXPLANATION: All modules that reside at the account level -// Billable Usage and Audit Logs -module "log_delivery" { - source = "./databricks_account/logging_configuration" - count = var.enable_logging_boolean ? 1 : 0 - providers = { - databricks = databricks.mws - } - - databricks_account_id = var.databricks_account_id - resource_prefix = var.resource_prefix - databricks_gov_shard = var.databricks_gov_shard - databricks_prod_aws_account_id = var.databricks_prod_aws_account_id - log_delivery_role_name = var.log_delivery_role_name -} - // Create Unity Catalog Metastore - No Root Storage module "uc_init" { @@ -75,6 +60,6 @@ module "user_assignment" { } created_workspace_id = module.databricks_mws_workspace.workspace_id - workspace_access = var.user_workspace_admin + workspace_access = var.admin_user depends_on = [module.uc_assignment, module.databricks_mws_workspace] } \ No newline at end of file diff --git a/aws-gov/tf/modules/sra/databricks_workspace.tf b/aws-gov/tf/modules/sra/databricks_workspace.tf index 606685c..b65dc64 100644 --- a/aws-gov/tf/modules/sra/databricks_workspace.tf +++ b/aws-gov/tf/modules/sra/databricks_workspace.tf @@ -2,7 +2,7 @@ // Creates a Workspace Isolated Catalog module "uc_catalog" { - source = "./databricks_workspace/workspace_security_modules/uc_catalog" + source = "./databricks_workspace/uc_catalog" providers = { databricks = databricks.created_workspace } @@ -13,7 +13,7 @@ module "uc_catalog" { uc_catalog_name = "${var.resource_prefix}-catalog-${module.databricks_mws_workspace.workspace_id}" cmk_admin_arn = var.cmk_admin_arn == null ? "arn:aws-us-gov:iam::${var.aws_account_id}:root" : var.cmk_admin_arn workspace_id = module.databricks_mws_workspace.workspace_id - user_workspace_catalog_admin = var.user_workspace_catalog_admin + user_workspace_catalog_admin = var.admin_user databricks_gov_shard = var.databricks_gov_shard databricks_prod_aws_account_id = var.databricks_prod_aws_account_id uc_master_role_id = var.uc_master_role_id @@ -21,100 +21,42 @@ module "uc_catalog" { depends_on = [module.databricks_mws_workspace, module.uc_assignment] } -// Create Read-Only Storage Location for Data Bucket & External Location -module "uc_external_location" { - count = var.enable_read_only_external_location_boolean ? 1 : 0 - source = "./databricks_workspace/workspace_security_modules/uc_external_location" - providers = { - databricks = databricks.created_workspace - } - - databricks_account_id = var.databricks_account_id - aws_account_id = var.aws_account_id - resource_prefix = var.resource_prefix - read_only_data_bucket = var.read_only_data_bucket - read_only_external_location_admin = var.read_only_external_location_admin - databricks_gov_shard = var.databricks_gov_shard - databricks_prod_aws_account_id = var.databricks_prod_aws_account_id - uc_master_role_id = var.uc_master_role_id -} - -// Workspace Admin Configuration -module "admin_configuration" { - count = var.enable_admin_configs_boolean ? 1 : 0 - source = "./databricks_workspace/workspace_security_modules/admin_configuration" - providers = { - databricks = databricks.created_workspace - } -} - -// IP Access Lists - Optional -module "ip_access_list" { - source = "./databricks_workspace/workspace_security_modules/ip_access_list" - count = var.enable_ip_boolean ? 1 : 0 - providers = { - databricks = databricks.created_workspace - } - - ip_addresses = var.ip_addresses -} - -// Create Create Cluster - Optional -module "cluster_configuration" { - source = "./databricks_workspace/workspace_security_modules/cluster_configuration" - count = var.enable_cluster_boolean ? 1 : 0 - providers = { - databricks = databricks.created_workspace - } - - compliance_security_profile_egress_ports = var.compliance_security_profile_egress_ports - resource_prefix = var.resource_prefix - operation_mode = var.operation_mode -} - -// System Table Schemas Enablement - Optional +// System Table Schemas Enablement - Coming Soon to AWS-Gov +/* module "system_table" { - source = "./databricks_workspace/workspace_security_modules/system_schema/" - count = var.enable_system_tables_schema_boolean ? 1 : 0 + source = "./databricks_workspace/system_schema" providers = { databricks = databricks.created_workspace } depends_on = [ module.uc_assignment ] } +*/ -// SAT Implementation - Optional -module "security_analysis_tool" { - source = "./databricks_workspace/solution_accelerators/security_analysis_tool/aws" - count = var.enable_sat_boolean ? 1 : 0 +// Create Create Cluster +module "cluster_configuration" { + source = "./databricks_workspace/classic_cluster" providers = { databricks = databricks.created_workspace } - databricks_url = module.databricks_mws_workspace.workspace_url - workspace_id = module.databricks_mws_workspace.workspace_id - account_console_id = var.databricks_account_id - client_id = var.client_id - client_secret = var.client_secret - use_sp_auth = true - proxies = {} - analysis_schema_name = "SAT" + resource_prefix = var.resource_prefix depends_on = [ - module.databricks_mws_workspace + module.databricks_mws_workspace, module.vpc_endpoints ] } -// System Tables Schemas - Optional -module "audit_log_alerting" { - source = "./databricks_workspace/solution_accelerators/system_tables_audit_log/" - count = var.enable_audit_log_alerting ? 1 : 0 +// Restrictive DBFS bucket policy +module "restrictive_root_bucket" { + source = "./databricks_workspace/restrictive_root_bucket" providers = { - databricks = databricks.created_workspace + aws = aws } - alert_emails = [var.user_workspace_admin] + databricks_account_id = var.databricks_account_id + workspace_id = module.databricks_mws_workspace.workspace_id + region_name = var.region_name + root_s3_bucket = "${var.resource_prefix}-workspace-root-storage" - depends_on = [ - module.databricks_mws_workspace - ] -} \ No newline at end of file + depends_on = [module.databricks_mws_workspace] +} diff --git a/aws-gov/tf/modules/sra/databricks_workspace/classic_cluster/cluster_configuration.tf b/aws-gov/tf/modules/sra/databricks_workspace/classic_cluster/cluster_configuration.tf new file mode 100644 index 0000000..80b41e3 --- /dev/null +++ b/aws-gov/tf/modules/sra/databricks_workspace/classic_cluster/cluster_configuration.tf @@ -0,0 +1,37 @@ +// Terraform Documentation: https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/cluster + +// Cluster Version +data "databricks_spark_version" "latest_lts" { + long_term_support = true +} + +// Cluster Creation +resource "databricks_cluster" "example" { + cluster_name = "Shared Classic Compute Plane Cluster" + data_security_mode = "USER_ISOLATION" + spark_version = data.databricks_spark_version.latest_lts.id + node_type_id = "i3en.xlarge" + autotermination_minutes = 10 + + autoscale { + min_workers = 1 + max_workers = 2 + } + + // Derby Metastore configs + spark_conf = { + "spark.hadoop.datanucleus.autoCreateTables" : "true", + "spark.hadoop.datanucleus.autoCreateSchema" : "true", + "spark.hadoop.javax.jdo.option.ConnectionDriverName" : "org.apache.derby.jdbc.EmbeddedDriver", + "spark.hadoop.javax.jdo.option.ConnectionPassword" : "hivepass", + "spark.hadoop.javax.jdo.option.ConnectionURL" : "jdbc:derby:memory:myInMemDB;create=true", + "spark.sql.catalogImplementation" : "hive", + "spark.hadoop.javax.jdo.option.ConnectionUserName" : "hiveuser", + "spark.hadoop.datanucleus.fixedDatastore" : "false" + } + + // Custom Tags + custom_tags = { + "Project" = var.resource_prefix + } +} \ No newline at end of file diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/ip_access_list/provider.tf b/aws-gov/tf/modules/sra/databricks_workspace/classic_cluster/provider.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/ip_access_list/provider.tf rename to aws-gov/tf/modules/sra/databricks_workspace/classic_cluster/provider.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/classic_cluster/variables.tf b/aws-gov/tf/modules/sra/databricks_workspace/classic_cluster/variables.tf new file mode 100644 index 0000000..f5b0239 --- /dev/null +++ b/aws-gov/tf/modules/sra/databricks_workspace/classic_cluster/variables.tf @@ -0,0 +1,3 @@ +variable "resource_prefix" { + type = string +} \ No newline at end of file diff --git a/aws-gov/tf/modules/sra/data_plane_hardening/restrictive_root_bucket/provider.tf b/aws-gov/tf/modules/sra/databricks_workspace/restrictive_root_bucket/provider.tf similarity index 100% rename from aws-gov/tf/modules/sra/data_plane_hardening/restrictive_root_bucket/provider.tf rename to aws-gov/tf/modules/sra/databricks_workspace/restrictive_root_bucket/provider.tf diff --git a/aws-gov/tf/modules/sra/data_plane_hardening/restrictive_root_bucket/restrictive_root_bucket.tf b/aws-gov/tf/modules/sra/databricks_workspace/restrictive_root_bucket/restrictive_root_bucket.tf similarity index 89% rename from aws-gov/tf/modules/sra/data_plane_hardening/restrictive_root_bucket/restrictive_root_bucket.tf rename to aws-gov/tf/modules/sra/databricks_workspace/restrictive_root_bucket/restrictive_root_bucket.tf index 4e3fafc..5ec72d5 100644 --- a/aws-gov/tf/modules/sra/data_plane_hardening/restrictive_root_bucket/restrictive_root_bucket.tf +++ b/aws-gov/tf/modules/sra/databricks_workspace/restrictive_root_bucket/restrictive_root_bucket.tf @@ -21,7 +21,12 @@ resource "aws_s3_bucket_policy" "databricks_bucket_restrictive_policy" { Resource = [ "arn:aws-us-gov:s3:::${var.root_s3_bucket}/*", "arn:aws-us-gov:s3:::${var.root_s3_bucket}" - ] + ], + Condition = { + StringEquals = { + "aws:PrincipalTag/DatabricksAccountId" = var.databricks_prod_aws_account_id[var.databricks_gov_shard] + } + } }, { Sid = "Grant Databricks Write Access", @@ -49,7 +54,12 @@ resource "aws_s3_bucket_policy" "databricks_bucket_restrictive_policy" { "arn:aws-us-gov:s3:::${var.root_s3_bucket}/${var.region_name}-prod/${var.workspace_id}/pipelines/*", "arn:aws-us-gov:s3:::${var.root_s3_bucket}/${var.region_name}-prod/${var.workspace_id}/local_disk0/tmp/*", "arn:aws-us-gov:s3:::${var.root_s3_bucket}/${var.region_name}-prod/${var.workspace_id}/tmp/*" - ] + ], + Condition = { + StringEquals = { + "aws:PrincipalTag/DatabricksAccountId" = var.databricks_prod_aws_account_id[var.databricks_gov_shard] + } + } }, { Sid = "AllowSSLRequestsOnly", diff --git a/aws-gov/tf/modules/sra/data_plane_hardening/restrictive_root_bucket/variables.tf b/aws-gov/tf/modules/sra/databricks_workspace/restrictive_root_bucket/variables.tf similarity index 99% rename from aws-gov/tf/modules/sra/data_plane_hardening/restrictive_root_bucket/variables.tf rename to aws-gov/tf/modules/sra/databricks_workspace/restrictive_root_bucket/variables.tf index a48b064..c42a410 100644 --- a/aws-gov/tf/modules/sra/data_plane_hardening/restrictive_root_bucket/variables.tf +++ b/aws-gov/tf/modules/sra/databricks_workspace/restrictive_root_bucket/variables.tf @@ -13,6 +13,7 @@ variable "workspace_id" { variable "databricks_gov_shard" { type = string } + variable "databricks_prod_aws_account_id" { type = map(string) } \ No newline at end of file diff --git a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/jobs.tf b/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/jobs.tf deleted file mode 100644 index 4fdfe04..0000000 --- a/aws-gov/tf/modules/sra/databricks_workspace/solution_accelerators/security_analysis_tool/common/jobs.tf +++ /dev/null @@ -1,71 +0,0 @@ -resource "databricks_job" "initializer" { - name = "SAT Initializer Notebook (one-time)" - job_cluster { - job_cluster_key = "job_cluster" - new_cluster { - num_workers = 5 - spark_version = data.databricks_spark_version.latest_lts.id - node_type_id = data.databricks_node_type.smallest.id - runtime_engine = "PHOTON" - dynamic "gcp_attributes" { - for_each = var.gcp_impersonate_service_account == "" ? [] : [var.gcp_impersonate_service_account] - content { - google_service_account = var.gcp_impersonate_service_account - } - } - } - } - - task { - task_key = "Initializer" - job_cluster_key = "job_cluster" - library { - pypi { - package = "dbl-sat-sdk" - } - } - notebook_task { - notebook_path = "${databricks_repo.security_analysis_tool.workspace_path}/notebooks/security_analysis_initializer" - } - } -} - -resource "databricks_job" "driver" { - name = "SAT Driver Notebook" - job_cluster { - job_cluster_key = "job_cluster" - new_cluster { - num_workers = 5 - spark_version = data.databricks_spark_version.latest_lts.id - node_type_id = data.databricks_node_type.smallest.id - runtime_engine = "PHOTON" - dynamic "gcp_attributes" { - for_each = var.gcp_impersonate_service_account == "" ? [] : [var.gcp_impersonate_service_account] - content { - google_service_account = var.gcp_impersonate_service_account - } - } - } - } - - - task { - task_key = "Driver" - job_cluster_key = "job_cluster" - library { - pypi { - package = "dbl-sat-sdk" - } - } - notebook_task { - notebook_path = "${databricks_repo.security_analysis_tool.workspace_path}/notebooks/security_analysis_driver" - } - } - - schedule { - #E.G. At 08:00:00am, on every Monday, Wednesday and Friday, every month; For more: http://www.quartz-scheduler.org/documentation/quartz-2.3.0/tutorials/crontrigger.html - quartz_cron_expression = "0 0 8 ? * Mon,Wed,Fri" - # The system default is UTC; For more: https://en.wikipedia.org/wiki/List_of_tz_database_time_zones - timezone_id = "America/New_York" - } -} diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/uc_catalog/provider.tf b/aws-gov/tf/modules/sra/databricks_workspace/system_schema/provider.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/uc_catalog/provider.tf rename to aws-gov/tf/modules/sra/databricks_workspace/system_schema/provider.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/system_schema/system_schema.tf b/aws-gov/tf/modules/sra/databricks_workspace/system_schema/system_schema.tf similarity index 80% rename from aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/system_schema/system_schema.tf rename to aws-gov/tf/modules/sra/databricks_workspace/system_schema/system_schema.tf index 180bf9f..2b38bde 100644 --- a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/system_schema/system_schema.tf +++ b/aws-gov/tf/modules/sra/databricks_workspace/system_schema/system_schema.tf @@ -23,3 +23,11 @@ resource "databricks_system_schema" "marketplace" { resource "databricks_system_schema" "storage" { schema = "storage" } + +resource "databricks_system_schema" "serving" { + schema = "serving" +} + +resource "databricks_system_schema" "query" { + schema = "query" +} \ No newline at end of file diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/uc_external_location/provider.tf b/aws-gov/tf/modules/sra/databricks_workspace/uc_catalog/provider.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/uc_external_location/provider.tf rename to aws-gov/tf/modules/sra/databricks_workspace/uc_catalog/provider.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/uc_catalog/uc_catalog.tf b/aws-gov/tf/modules/sra/databricks_workspace/uc_catalog/uc_catalog.tf similarity index 82% rename from aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/uc_catalog/uc_catalog.tf rename to aws-gov/tf/modules/sra/databricks_workspace/uc_catalog/uc_catalog.tf index b2491d6..60ac534 100644 --- a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/uc_catalog/uc_catalog.tf +++ b/aws-gov/tf/modules/sra/databricks_workspace/uc_catalog/uc_catalog.tf @@ -1,9 +1,65 @@ resource "null_resource" "previous" {} -resource "time_sleep" "wait_30_seconds" { - depends_on = [null_resource.previous] +// Wait to prevent race condition between IAM role and external location validation +resource "time_sleep" "wait_60_seconds" { + depends_on = [null_resource.previous] + create_duration = "60s" +} - create_duration = "30s" +locals { + uc_iam_role = "${var.resource_prefix}-catalog-${var.workspace_id}" + uc_catalog_name_us = replace(var.uc_catalog_name, "-", "_") +} + +// Unity Catalog KMS +resource "aws_kms_key" "catalog_storage" { + description = "KMS key for Databricks catalog storage ${var.workspace_id}" + policy = jsonencode({ + Version : "2012-10-17", + "Id" : "key-policy-catalog-storage-${var.workspace_id}", + Statement : [ + { + "Sid" : "Enable IAM User Permissions", + "Effect" : "Allow", + "Principal" : { + "AWS" : [var.cmk_admin_arn] + }, + "Action" : "kms:*", + "Resource" : "*" + }, + { + "Sid" : "Allow IAM Role to use the key", + "Effect" : "Allow", + "Principal" : { + "AWS" : "arn:aws-us-gov:iam::${var.aws_account_id}:role/${local.uc_iam_role}" + }, + "Action" : [ + "kms:Decrypt", + "kms:Encrypt", + "kms:GenerateDataKey*" + ], + "Resource" : "*" + } + ] + }) + tags = { + Name = "${var.resource_prefix}-catalog-storage-${var.workspace_id}-key" + Project = var.resource_prefix + } +} + +resource "aws_kms_alias" "catalog_storage_key_alias" { + name = "alias/${var.resource_prefix}-catalog-storage-${var.workspace_id}-key" + target_key_id = aws_kms_key.catalog_storage.id +} + +// Storage Credential (created before role): https://registry.terraform.io/providers/databricks/databricks/latest/docs/guides/unity-catalog#configure-external-locations-and-credentials +resource "databricks_storage_credential" "workspace_catalog_storage_credential" { + name = "${var.uc_catalog_name}-storage-credential" + aws_iam_role { + role_arn = "arn:aws-us-gov:iam::${var.aws_account_id}:role/${local.uc_iam_role}" + } + isolation_mode = "ISOLATION_MODE_ISOLATED" } // Unity Catalog Trust Policy - Data Source @@ -32,7 +88,7 @@ data "aws_iam_policy_document" "passrole_for_unity_catalog_catalog" { condition { test = "ArnLike" variable = "aws:PrincipalArn" - values = ["arn:aws-us-gov:iam::${var.aws_account_id}:role/${var.resource_prefix}-unity-catalog-${var.workspace_id}"] + values = ["arn:aws-us-gov:iam::${var.aws_account_id}:role/${local.uc_iam_role}"] } condition { test = "StringEquals" @@ -43,11 +99,11 @@ data "aws_iam_policy_document" "passrole_for_unity_catalog_catalog" { } // Unity Catalog Role -resource "aws_iam_role" "unity_catalog_role" { - name = "${var.resource_prefix}-catalog-${var.workspace_id}" +resource "aws_iam_role" "unity_catalog" { + name = local.uc_iam_role assume_role_policy = data.aws_iam_policy_document.passrole_for_unity_catalog_catalog.json tags = { - Name = "${var.resource_prefix}-catalog-${var.workspace_id}" + Name = local.uc_iam_role Project = var.resource_prefix } } @@ -75,7 +131,7 @@ data "aws_iam_policy_document" "unity_catalog_iam_policy" { statement { actions = ["sts:AssumeRole"] - resources = ["arn:aws-us-gov:iam::${var.aws_account_id}:role/${var.resource_prefix}-unity-catalog-${var.workspace_id}"] + resources = ["arn:aws-us-gov:iam::${var.aws_account_id}:role/${local.uc_iam_role}"] effect = "Allow" } } @@ -83,52 +139,10 @@ data "aws_iam_policy_document" "unity_catalog_iam_policy" { // Unity Catalog Policy resource "aws_iam_role_policy" "unity_catalog" { name = "${var.resource_prefix}-catalog-policy-${var.workspace_id}" - role = aws_iam_role.unity_catalog_role.id + role = aws_iam_role.unity_catalog.id policy = data.aws_iam_policy_document.unity_catalog_iam_policy.json } -// Unity Catalog KMS -resource "aws_kms_key" "catalog_storage" { - description = "KMS key for Databricks catalog storage ${var.workspace_id}" - policy = jsonencode({ - Version : "2012-10-17", - "Id" : "key-policy-catalog-storage-${var.workspace_id}", - Statement : [ - { - "Sid" : "Enable IAM User Permissions", - "Effect" : "Allow", - "Principal" : { - "AWS" : [var.cmk_admin_arn] - }, - "Action" : "kms:*", - "Resource" : "*" - }, - { - "Sid" : "Allow IAM Role to use the key", - "Effect" : "Allow", - "Principal" : { - "AWS" : "arn:aws-us-gov:iam::${var.aws_account_id}:role/${var.resource_prefix}-catalog-${var.workspace_id}" - }, - "Action" : [ - "kms:Decrypt", - "kms:Encrypt", - "kms:GenerateDataKey*" - ], - "Resource" : "*" - } - ] - }) - tags = { - Name = "${var.resource_prefix}-catalog-storage-${var.workspace_id}-key" - Project = var.resource_prefix - } -} - -resource "aws_kms_alias" "catalog_storage_key_alias" { - name = "alias/${var.resource_prefix}-catalog-storage-${var.workspace_id}-key" - target_key_id = aws_kms_key.catalog_storage.id -} - // Unity Catalog S3 resource "aws_s3_bucket" "unity_catalog_bucket" { bucket = var.uc_catalog_name @@ -148,7 +162,6 @@ resource "aws_s3_bucket_versioning" "unity_catalog_versioning" { resource "aws_s3_bucket_server_side_encryption_configuration" "unity_catalog" { bucket = aws_s3_bucket.unity_catalog_bucket.bucket - rule { bucket_key_enabled = true apply_server_side_encryption_by_default { @@ -168,33 +181,22 @@ resource "aws_s3_bucket_public_access_block" "unity_catalog" { depends_on = [aws_s3_bucket.unity_catalog_bucket] } -// Storage Credential -resource "databricks_storage_credential" "workspace_catalog_storage_credential" { - name = aws_iam_role.unity_catalog_role.name - aws_iam_role { - role_arn = aws_iam_role.unity_catalog_role.arn - } - depends_on = [aws_iam_role.unity_catalog_role, time_sleep.wait_30_seconds] - isolation_mode = "ISOLATION_MODE_ISOLATED" -} - // External Location resource "databricks_external_location" "workspace_catalog_external_location" { - name = var.uc_catalog_name - url = "s3://${var.uc_catalog_name}/catalog/" + name = "${var.uc_catalog_name}-external-location" + url = "s3://${var.uc_catalog_name}/" credential_name = databricks_storage_credential.workspace_catalog_storage_credential.id - skip_validation = true - read_only = false comment = "External location for catalog ${var.uc_catalog_name}" isolation_mode = "ISOLATION_MODE_ISOLATED" + depends_on = [aws_iam_role_policy.unity_catalog, time_sleep.wait_60_seconds] } // Workspace Catalog resource "databricks_catalog" "workspace_catalog" { - name = replace(var.uc_catalog_name, "-", "_") + name = local.uc_catalog_name_us comment = "This catalog is for workspace - ${var.workspace_id}" isolation_mode = "ISOLATED" - storage_root = "s3://${var.uc_catalog_name}/catalog/" + storage_root = "s3://${var.uc_catalog_name}/" properties = { purpose = "Catalog for workspace - ${var.workspace_id}" } @@ -204,7 +206,7 @@ resource "databricks_catalog" "workspace_catalog" { // Set Workspace Catalog as Default resource "databricks_default_namespace_setting" "this" { namespace { - value = replace(var.uc_catalog_name, "-", "_") + value = local.uc_catalog_name_us } } @@ -214,4 +216,4 @@ resource "databricks_grant" "workspace_catalog" { principal = var.user_workspace_catalog_admin privileges = ["ALL_PRIVILEGES"] -} +} \ No newline at end of file diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/uc_catalog/variables.tf b/aws-gov/tf/modules/sra/databricks_workspace/uc_catalog/variables.tf similarity index 100% rename from aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/uc_catalog/variables.tf rename to aws-gov/tf/modules/sra/databricks_workspace/uc_catalog/variables.tf diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/cluster_configuration/cluster_configuration.tf b/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/cluster_configuration/cluster_configuration.tf deleted file mode 100644 index ed3d319..0000000 --- a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/cluster_configuration/cluster_configuration.tf +++ /dev/null @@ -1,79 +0,0 @@ -// Terraform Documentation: https://registry.terraform.io/providers/databricks/databricks/latest/docs/resources/cluster - -// Cluster Version -data "databricks_spark_version" "latest_lts" { - long_term_support = true -} - -// Example Cluster Policy -locals { - default_policy = { - "dbus_per_hour" : { - "type" : "range", - "maxValue" : 10 - }, - "autotermination_minutes" : { - "type" : "fixed", - "value" : 10, - "hidden" : true - }, - "custom_tags.Project" : { - "type" : "fixed", - "value" : var.resource_prefix - }, - "spark_conf.spark.hadoop.javax.jdo.option.ConnectionURL" : null, - "spark_conf.spark.hadoop.javax.jdo.option.ConnectionDriverName" : null, - "spark_conf.spark.hadoop.javax.jdo.option.ConnectionUserName" : null, - "spark_conf.spark.hadoop.javax.jdo.option.ConnectionPassword" : null - } - - isolated_policy = merge( - local.default_policy, - { - "spark_conf.spark.hadoop.javax.jdo.option.ConnectionURL" : { - "type" : "fixed", - "value" : "jdbc:derby:memory:myInMemDB;create=true" - }, - "spark_conf.spark.hadoop.javax.jdo.option.ConnectionDriverName" : { - "type" : "fixed", - "value" : "org.apache.derby.jdbc.EmbeddedDriver" - }, - "spark_conf.spark.hadoop.javax.jdo.option.ConnectionUserName" : { - "type" : "fixed", - "value" : "" - }, - "spark_conf.spark.hadoop.javax.jdo.option.ConnectionPassword" : { - "type" : "fixed", - "value" : "" - } - } - ) - - selected_policy = var.operation_mode == "isolated" ? local.isolated_policy : local.default_policy - - final_policy = { for k, v in local.selected_policy : k => v if v != null } -} - -resource "databricks_cluster_policy" "example" { - name = "Example Cluster Policy" - definition = jsonencode(local.final_policy) -} - -// Cluster Creation -resource "databricks_cluster" "example" { - cluster_name = "Shared Cluster" - data_security_mode = "USER_ISOLATION" - spark_version = data.databricks_spark_version.latest_lts.id - node_type_id = var.compliance_security_profile_egress_ports ? "i3en.xlarge" : "i3.xlarge" - policy_id = databricks_cluster_policy.example.id - autotermination_minutes = 10 - - autoscale { - min_workers = 1 - max_workers = 2 - } - - depends_on = [ - databricks_cluster_policy.example - ] -} \ No newline at end of file diff --git a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/cluster_configuration/variables.tf b/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/cluster_configuration/variables.tf deleted file mode 100644 index 29344f3..0000000 --- a/aws-gov/tf/modules/sra/databricks_workspace/workspace_security_modules/cluster_configuration/variables.tf +++ /dev/null @@ -1,12 +0,0 @@ -variable "compliance_security_profile_egress_ports" { - type = bool - nullable = false -} - -variable "operation_mode" { - type = string -} - -variable "resource_prefix" { - type = string -} \ No newline at end of file diff --git a/aws-gov/tf/modules/sra/network.tf b/aws-gov/tf/modules/sra/network.tf index b4333ca..4557156 100644 --- a/aws-gov/tf/modules/sra/network.tf +++ b/aws-gov/tf/modules/sra/network.tf @@ -4,21 +4,17 @@ module "vpc" { source = "terraform-aws-modules/vpc/aws" version = "5.1.1" - - count = var.operation_mode != "custom" ? 1 : 0 + count = var.network_configuration != "custom" ? 1 : 0 name = "${var.resource_prefix}-classic-compute-plane-vpc" cidr = var.vpc_cidr_range azs = var.availability_zones enable_dns_hostnames = true - enable_nat_gateway = var.operation_mode == "firewall" || var.operation_mode == "isolated" ? false : true + enable_nat_gateway = false single_nat_gateway = false - one_nat_gateway_per_az = var.operation_mode == "firewall" || var.operation_mode == "isolated" ? false : true - create_igw = var.operation_mode == "firewall" || var.operation_mode == "isolated" ? false : true - - public_subnet_names = var.operation_mode == "firewall" || var.operation_mode == "isolated" ? [] : [for az in var.availability_zones : format("%s-public-%s", var.resource_prefix, az)] - public_subnets = var.operation_mode == "firewall" || var.operation_mode == "isolated" ? [] : var.public_subnets_cidr + one_nat_gateway_per_az = false + create_igw = false private_subnet_names = [for az in var.availability_zones : format("%s-private-%s", var.resource_prefix, az)] private_subnets = var.private_subnets_cidr @@ -31,9 +27,10 @@ module "vpc" { } } + // Security group - skipped in custom mode resource "aws_security_group" "sg" { - count = var.operation_mode != "custom" ? 1 : 0 + count = var.network_configuration != "custom" ? 1 : 0 vpc_id = module.vpc[0].vpc_id depends_on = [module.vpc] @@ -71,17 +68,6 @@ resource "aws_security_group" "sg" { } } - dynamic "egress" { - for_each = var.compliance_security_profile_egress_ports ? [2443] : [] - - content { - description = "Databricks - Workspace Security Group - FIPS encryption" - from_port = 2443 - to_port = 2443 - protocol = "tcp" - cidr_blocks = ["0.0.0.0/0"] - } - } tags = { Name = "${var.resource_prefix}-workspace-sg" Project = var.resource_prefix diff --git a/aws-gov/tf/modules/sra/privatelink.tf b/aws-gov/tf/modules/sra/privatelink.tf index d85dff3..63d0f44 100644 --- a/aws-gov/tf/modules/sra/privatelink.tf +++ b/aws-gov/tf/modules/sra/privatelink.tf @@ -1,6 +1,6 @@ // Security group for privatelink - skipped in custom operation mode resource "aws_security_group" "privatelink" { - count = var.operation_mode != "custom" ? 1 : 0 + count = var.network_configuration != "custom" ? 1 : 0 vpc_id = module.vpc[0].vpc_id @@ -13,9 +13,9 @@ resource "aws_security_group" "privatelink" { } ingress { - description = "Databricks - PrivateLink Endpoint SG - Secure Cluster Connectivity" - from_port = 6666 - to_port = 6666 + description = "Databricks - PrivateLink Endpoint SG - Secure Cluster Connectivity - CSP" + from_port = 2443 + to_port = 2443 protocol = "tcp" security_groups = [aws_security_group.sg[0].id] } @@ -28,18 +28,6 @@ resource "aws_security_group" "privatelink" { security_groups = [aws_security_group.sg[0].id] } - dynamic "ingress" { - for_each = var.compliance_security_profile_egress_ports ? [2443] : [] - - content { - description = "Databricks - PrivateLink Endpoint SG - FIPS encryption" - from_port = 2443 - to_port = 2443 - protocol = "tcp" - security_groups = [aws_security_group.sg[0].id] - } - } - tags = { Name = "${var.resource_prefix}-private-link-sg", Project = var.resource_prefix @@ -48,10 +36,9 @@ resource "aws_security_group" "privatelink" { // EXPLANATION: VPC Gateway Endpoint for S3, Interface Endpoint for Kinesis, and Interface Endpoint for STS - -// Restrictive S3 endpoint policy - only used if restrictive S3 endpoint policy is enabled +// Restrictive S3 endpoint policy: data "aws_iam_policy_document" "s3_vpc_endpoint_policy" { - count = var.enable_restrictive_s3_endpoint_boolean ? 1 : 0 + count = var.network_configuration != "custom" ? 1 : 0 statement { sid = "Grant access to Databricks Root Bucket" @@ -80,14 +67,6 @@ data "aws_iam_policy_document" "s3_vpc_endpoint_policy" { variable = "aws:PrincipalAccount" values = ["${var.databricks_prod_aws_account_id[var.databricks_gov_shard]}"] } - - condition { - test = "StringEqualsIfExists" - variable = "aws:SourceVpc" - values = [ - module.vpc[0].vpc_id - ] - } } statement { @@ -111,15 +90,26 @@ data "aws_iam_policy_document" "s3_vpc_endpoint_policy" { "arn:aws-us-gov:s3:::${var.resource_prefix}-catalog-${module.databricks_mws_workspace.workspace_id}/*", "arn:aws-us-gov:s3:::${var.resource_prefix}-catalog-${module.databricks_mws_workspace.workspace_id}" ] + + condition { + test = "StringEquals" + variable = "aws:PrincipalAccount" + values = [var.aws_account_id] + } + condition { + test = "StringEquals" + variable = "s3:ResourceAccount" + values = [var.aws_account_id] + } } statement { - sid = "Grant read-only access to Data Bucket" - effect = "Allow" + sid = "Grant access to Artifact Buckets" + effect = "Allow" actions = [ - "s3:GetObject", - "s3:GetObjectVersion", "s3:ListBucket", + "s3:GetObjectVersion", + "s3:GetObject", "s3:GetBucketLocation" ] @@ -129,13 +119,19 @@ data "aws_iam_policy_document" "s3_vpc_endpoint_policy" { } resources = [ - "arn:aws-us-gov:s3:::${var.read_only_data_bucket}/*", - "arn:aws-us-gov:s3:::${var.read_only_data_bucket}" + "arn:aws-us-gov:s3:::databricks-prod-artifacts-${var.region}/*", + "arn:aws-us-gov:s3:::databricks-prod-artifacts-${var.region}" ] + + condition { + test = "StringEquals" + variable = "aws:ResourceAccount" + values = ["${var.databricks_prod_aws_account_id[var.databricks_gov_shard]}"] + } } statement { - sid = "Grant Databricks Read Access to Artifact and Data Buckets" + sid = "Grant access to Databricks System Tables Bucket" effect = "Allow" actions = [ "s3:ListBucket", @@ -150,16 +146,47 @@ data "aws_iam_policy_document" "s3_vpc_endpoint_policy" { } resources = [ - "arn:aws-us-gov:s3:::databricks-prod-artifacts-${var.region}/*", - "arn:aws-us-gov:s3:::databricks-prod-artifacts-${var.region}", - "arn:aws-us-gov:s3:::databricks-datasets-${var.region_name}/*", - "arn:aws-us-gov:s3:::databricks-datasets-${var.region_name}" + "arn:aws-us-gov:s3:::system-tables-prod-${var.region}-uc-metastore-bucket/*", + "arn:aws-us-gov:s3:::system-tables-prod-${var.region}-uc-metastore-bucket", ] + + condition { + test = "StringEquals" + variable = "aws:PrincipalAccount" + values = ["${var.databricks_prod_aws_account_id[var.databricks_gov_shard]}"] + } } statement { - sid = "Grant access to Databricks Log Bucket" + sid = "Grant access to Databricks Sample Data Bucket" effect = "Allow" + actions = [ + "s3:ListBucket", + "s3:GetObjectVersion", + "s3:GetObject", + "s3:GetBucketLocation" + ] + + principals { + type = "AWS" + identifiers = ["*"] + } + + resources = [ + "arn:aws-us-gov:s3:::databricks-datasets-${var.region_bucket_name}/*", + "arn:aws-us-gov:s3:::databricks-datasets-${var.region_bucket_name}" + ] + + condition { + test = "StringEquals" + variable = "aws:PrincipalAccount" + values = ["${var.databricks_prod_aws_account_id[var.databricks_gov_shard]}"] + } + } + + statement { + sid = "Grant access to Databricks Log Bucket" + effect = "Allow" actions = [ "s3:PutObject", "s3:ListBucket", @@ -172,8 +199,8 @@ data "aws_iam_policy_document" "s3_vpc_endpoint_policy" { } resources = [ - "arn:aws-us-gov:s3:::databricks-prod-storage-${var.region_name}/*", - "arn:aws-us-gov:s3:::databricks-prod-storage-${var.region_name}" + "arn:aws-us-gov:s3:::databricks-prod-storage-${var.region_bucket_name}/*", + "arn:aws-us-gov:s3:::databricks-prod-storage-${var.region_bucket_name}" ] condition { @@ -182,13 +209,11 @@ data "aws_iam_policy_document" "s3_vpc_endpoint_policy" { values = ["${var.databricks_prod_aws_account_id[var.databricks_gov_shard]}"] } } - depends_on = [module.databricks_mws_workspace] } -// Restrictive STS endpoint policy - only used if restrictive STS endpoint policy is enabled +// Restrictive STS endpoint policy: data "aws_iam_policy_document" "sts_vpc_endpoint_policy" { - count = var.enable_restrictive_sts_endpoint_boolean ? 1 : 0 - + count = var.network_configuration != "custom" ? 1 : 0 statement { actions = [ "sts:AssumeRole", @@ -225,10 +250,9 @@ data "aws_iam_policy_document" "sts_vpc_endpoint_policy" { } } -// Restrictive Kinesis endpoint policy - only used if restrictive Kinesis endpoint policy is enabled +// Restrictive Kinesis endpoint policy: data "aws_iam_policy_document" "kinesis_vpc_endpoint_policy" { - count = var.enable_restrictive_kinesis_endpoint_boolean ? 1 : 0 - + count = var.network_configuration != "custom" ? 1 : 0 statement { actions = [ "kinesis:PutRecord", @@ -247,7 +271,7 @@ data "aws_iam_policy_document" "kinesis_vpc_endpoint_policy" { // VPC endpoint creation - Skipped in custom operation mode module "vpc_endpoints" { - count = var.operation_mode != "custom" ? 1 : 0 + count = var.network_configuration != "custom" ? 1 : 0 source = "terraform-aws-modules/vpc/aws//modules/vpc-endpoints" version = "3.11.0" @@ -260,7 +284,7 @@ module "vpc_endpoints" { service = "s3" service_type = "Gateway" route_table_ids = module.vpc[0].private_route_table_ids - policy = var.enable_restrictive_s3_endpoint_boolean ? data.aws_iam_policy_document.s3_vpc_endpoint_policy[0].json : null + policy = data.aws_iam_policy_document.s3_vpc_endpoint_policy[0].json tags = { Name = "${var.resource_prefix}-s3-vpc-endpoint" Project = var.resource_prefix @@ -270,7 +294,7 @@ module "vpc_endpoints" { service = "sts" private_dns_enabled = true subnet_ids = module.vpc[0].intra_subnets - policy = var.enable_restrictive_sts_endpoint_boolean ? data.aws_iam_policy_document.sts_vpc_endpoint_policy[0].json : null + policy = data.aws_iam_policy_document.sts_vpc_endpoint_policy[0].json tags = { Name = "${var.resource_prefix}-sts-vpc-endpoint" Project = var.resource_prefix @@ -280,7 +304,7 @@ module "vpc_endpoints" { service = "kinesis-streams" private_dns_enabled = true subnet_ids = module.vpc[0].intra_subnets - policy = var.enable_restrictive_kinesis_endpoint_boolean ? data.aws_iam_policy_document.kinesis_vpc_endpoint_policy[0].json : null + policy = data.aws_iam_policy_document.kinesis_vpc_endpoint_policy[0].json tags = { Name = "${var.resource_prefix}-kinesis-vpc-endpoint" Project = var.resource_prefix @@ -291,7 +315,7 @@ module "vpc_endpoints" { // Databricks REST endpoint - skipped in custom operation mode resource "aws_vpc_endpoint" "backend_rest" { - count = var.operation_mode != "custom" ? 1 : 0 + count = var.network_configuration != "custom" ? 1 : 0 vpc_id = module.vpc[0].vpc_id service_name = var.workspace[var.databricks_gov_shard] @@ -299,7 +323,6 @@ resource "aws_vpc_endpoint" "backend_rest" { security_group_ids = [aws_security_group.privatelink[0].id] subnet_ids = module.vpc[0].intra_subnets private_dns_enabled = true - depends_on = [module.vpc.vpc_id] tags = { Name = "${var.resource_prefix}-databricks-backend-rest" Project = var.resource_prefix @@ -308,7 +331,7 @@ resource "aws_vpc_endpoint" "backend_rest" { // Databricks SCC endpoint - skipped in custom operation mode resource "aws_vpc_endpoint" "backend_relay" { - count = var.operation_mode != "custom" ? 1 : 0 + count = var.network_configuration != "custom" ? 1 : 0 vpc_id = module.vpc[0].vpc_id service_name = var.scc_relay[var.databricks_gov_shard] @@ -316,7 +339,6 @@ resource "aws_vpc_endpoint" "backend_relay" { security_group_ids = [aws_security_group.privatelink[0].id] subnet_ids = module.vpc[0].intra_subnets private_dns_enabled = true - depends_on = [module.vpc.vpc_id] tags = { Name = "${var.resource_prefix}-databricks-backend-relay" Project = var.resource_prefix diff --git a/aws-gov/tf/modules/sra/root_s3_bucket.tf b/aws-gov/tf/modules/sra/root_s3_bucket.tf index 6c1d617..9d55524 100644 --- a/aws-gov/tf/modules/sra/root_s3_bucket.tf +++ b/aws-gov/tf/modules/sra/root_s3_bucket.tf @@ -18,7 +18,6 @@ resource "aws_s3_bucket_versioning" "root_bucket_versioning" { resource "aws_s3_bucket_server_side_encryption_configuration" "root_storage_bucket" { bucket = aws_s3_bucket.root_storage_bucket.bucket - rule { bucket_key_enabled = true apply_server_side_encryption_by_default { @@ -66,19 +65,7 @@ data "aws_iam_policy_document" "this" { } } -# Bucket policy to use if the restrictive root bucket is set to false resource "aws_s3_bucket_policy" "root_bucket_policy" { - count = var.enable_restrictive_root_bucket_boolean ? 0 : 1 - - bucket = aws_s3_bucket.root_storage_bucket.id - policy = data.aws_iam_policy_document.this.json - depends_on = [aws_s3_bucket_public_access_block.root_storage_bucket] -} - -# Bucket policy to use if the restrictive root bucket is set to true -resource "aws_s3_bucket_policy" "root_bucket_policy_ignore" { - count = var.enable_restrictive_root_bucket_boolean ? 1 : 0 - bucket = aws_s3_bucket.root_storage_bucket.id policy = data.aws_iam_policy_document.this.json depends_on = [aws_s3_bucket_public_access_block.root_storage_bucket] diff --git a/aws-gov/tf/modules/sra/variables.tf b/aws-gov/tf/modules/sra/variables.tf index 6fb531d..8ac24ca 100644 --- a/aws-gov/tf/modules/sra/variables.tf +++ b/aws-gov/tf/modules/sra/variables.tf @@ -1,3 +1,8 @@ +variable "admin_user" { + description = "Email of the admin user for the workspace and workspace catalog." + type = string +} + variable "availability_zones" { description = "List of AWS availability zones." type = list(string) @@ -24,37 +29,37 @@ variable "client_secret" { variable "cmk_admin_arn" { description = "Amazon Resource Name (ARN) of the CMK admin." type = string -} - -variable "compliance_security_profile_egress_ports" { - type = bool - description = "Add 2443 to security group configuration or nitro instance" - nullable = false + default = null } variable "custom_private_subnet_ids" { type = list(string) description = "List of custom private subnet IDs" + default = null } variable "custom_relay_vpce_id" { type = string description = "Custom Relay VPC Endpoint ID" + default = null } variable "custom_sg_id" { type = string description = "Custom security group ID" + default = null } variable "custom_vpc_id" { type = string description = "Custom VPC ID" + default = null } variable "custom_workspace_vpce_id" { type = string description = "Custom Workspace VPC Endpoint ID" + default = null } variable "databricks_account_id" { @@ -63,148 +68,39 @@ variable "databricks_account_id" { sensitive = true } -variable "enable_admin_configs_boolean" { - type = bool - description = "Enable workspace configs" - nullable = false -} - -variable "enable_audit_log_alerting" { - description = "Flag to audit log alerting." - type = bool - sensitive = true - default = false -} - -variable "enable_cluster_boolean" { - description = "Flag to enable cluster." - type = bool - sensitive = true - default = false -} - -variable "enable_ip_boolean" { - description = "Flag to enable IP-related configurations." - type = bool - sensitive = true - default = false -} - -variable "enable_logging_boolean" { - description = "Flag to enable logging." - type = bool - sensitive = true - default = false -} - -variable "enable_read_only_external_location_boolean" { - description = "Flag to enable read only external location" - type = bool - sensitive = true - default = false -} - -variable "enable_restrictive_kinesis_endpoint_boolean" { - type = bool - description = "Enable restrictive Kinesis endpoint boolean flag" - default = false -} - -variable "enable_restrictive_root_bucket_boolean" { - description = "Flag to enable restrictive root bucket settings." - type = bool - sensitive = true - default = false -} - -variable "enable_restrictive_s3_endpoint_boolean" { - type = bool - description = "Enable restrictive S3 endpoint boolean flag" - default = false -} - -variable "enable_restrictive_sts_endpoint_boolean" { - type = bool - description = "Enable restrictive STS endpoint boolean flag" - default = false -} - variable "enable_sat_boolean" { - description = "Flag for a specific SAT (Service Access Token) configuration." + description = "Flag to enable the security analysis tool." type = bool sensitive = true default = false } -variable "enable_system_tables_schema_boolean" { - description = "Flag for enabling public preview system schema access" - type = bool - sensitive = true - default = false -} - -variable "firewall_allow_list" { - description = "List of allowed firewall rules." - type = list(string) -} - -variable "firewall_subnets_cidr" { - description = "CIDR blocks for firewall subnets." - type = list(string) -} - -variable "hms_fqdn" { - type = map(string) - default = { - "civilian" = "discovery-search-rds-prod-dbdiscoverysearch-uus7j2cyyu1m.c40ji7ukhesx.us-gov-west-1.rds.amazonaws.com" - "dod" = "lineage-usgovwest1dod-prod.cpnejponioft.us-gov-west-1.rds.amazonaws.com" - } -} - -variable "ip_addresses" { - description = "List of IP addresses to allow list." - type = list(string) -} - variable "metastore_exists" { description = "If a metastore exists" type = bool } -variable "operation_mode" { +variable "network_configuration" { type = string - description = "The type of Operation Mode for the workspace network configuration." + description = "The type of network set-up for the workspace network configuration." nullable = false validation { - condition = contains(["sandbox", "firewall", "custom", "isolated"], var.operation_mode) - error_message = "Invalid operation mode. Allowed values are: sandbox, firewall, custom, isolated." + condition = contains(["custom", "isolated"], var.network_configuration) + error_message = "Invalid network configuration. Allowed values are: custom, isolated." } } variable "private_subnets_cidr" { description = "CIDR blocks for private subnets." type = list(string) + nullable = true } variable "privatelink_subnets_cidr" { description = "CIDR blocks for private link subnets." type = list(string) -} - -variable "public_subnets_cidr" { - description = "CIDR blocks for public subnets." - type = list(string) -} - -variable "read_only_data_bucket" { - description = "S3 bucket for data storage." - type = string -} - -variable "read_only_external_location_admin" { - description = "User to grant external location admin." - type = string + nullable = true } variable "region" { @@ -217,6 +113,11 @@ variable "region_name" { type = string } +variable "region_bucket_name" { + description = "Name of the AWS region for buckets." + type = string +} + variable "resource_prefix" { description = "Prefix for the resource names." type = string @@ -235,17 +136,6 @@ variable "sg_egress_ports" { type = list(string) } -variable "user_workspace_admin" { - description = "User to grant admin workspace access." - type = string - nullable = false -} - -variable "user_workspace_catalog_admin" { - description = "Admin for the workspace catalog" - type = string -} - variable "vpc_cidr_range" { description = "CIDR range for the VPC." type = string @@ -265,7 +155,6 @@ variable "databricks_gov_shard" { type = string } - variable "databricks_prod_aws_account_id" { description = "Databricks Prod AWS Account Id" type = map(string) @@ -291,4 +180,4 @@ variable "uc_master_role_id" { "civilian" = "1QRFA8SGY15OJ" "dod" = "1DI6DL6ZP26AS" } -} +} \ No newline at end of file diff --git a/aws-gov/tf/provider.tf b/aws-gov/tf/provider.tf index 66469b3..5b101f9 100644 --- a/aws-gov/tf/provider.tf +++ b/aws-gov/tf/provider.tf @@ -2,7 +2,7 @@ terraform { required_providers { databricks = { source = "databricks/databricks" - version = " 1.50.0" + version = " 1.54.0" } aws = { source = "hashicorp/aws" diff --git a/aws-gov/tf/sra.tf b/aws-gov/tf/sra.tf index 90af84e..a2500ae 100644 --- a/aws-gov/tf/sra.tf +++ b/aws-gov/tf/sra.tf @@ -5,67 +5,35 @@ module "SRA" { aws = aws } - // REQUIRED - Authentication: databricks_account_id = var.databricks_account_id client_id = var.client_id client_secret = var.client_secret aws_account_id = var.aws_account_id region = var.region - databricks_gov_shard = var.databricks_gov_shard region_name = var.region_name[var.databricks_gov_shard] + region_bucket_name = var.region_bucket_name[var.databricks_gov_shard] + databricks_gov_shard = var.databricks_gov_shard + admin_user = var.admin_user + resource_prefix = var.resource_prefix - // REQUIRED - Naming and Tagging: - resource_prefix = var.resource_prefix - - // REQUIRED - Workspace and Unity Catalog: - user_workspace_admin = null // Workspace admin user email. - user_workspace_catalog_admin = null // Workspace catalog admin email. - operation_mode = "isolated" // Operation mode (sandbox, custom, firewall, isolated), see README.md for more information. - metastore_exists = false // If a regional metastore exists set to true. If there are multiple regional metastores, you can comment out "uc_init" and add the metastore ID directly in to the module call for "uc_assignment". + // REQUIRED: + network_configuration = "isolated" // Network (custom or isolated), see README.md for more information. + metastore_exists = false // If a regional metastore exists set to true. - // REQUIRED - AWS Infrastructure: - cmk_admin_arn = null // CMK admin ARN, defaults to the AWS account root user. + // REQUIRED IF USING ISOLATED NETWORK: vpc_cidr_range = "10.0.0.0/18" // Please re-define the subsequent subnet ranges if the VPC CIDR range is updated. private_subnets_cidr = ["10.0.0.0/22", "10.0.4.0/22", "10.0.8.0/22"] privatelink_subnets_cidr = ["10.0.28.0/26", "10.0.28.64/26", "10.0.28.128/26"] availability_zones = [data.aws_availability_zones.available.names[0], data.aws_availability_zones.available.names[1], data.aws_availability_zones.available.names[2]] - sg_egress_ports = [443, 3306, 8443, 8444, 8445, 8446, 8447, 8448, 8449, 8450, 8451] - compliance_security_profile_egress_ports = true // Set to true to enable compliance security profile related egress ports (2443) - - // Operation Mode Specific: - // Sandbox and Firewall Operation Mode: - public_subnets_cidr = ["10.0.29.0/26", "10.0.29.64/26", "10.0.29.128/26"] - - // Firewall Operation Mode: - firewall_subnets_cidr = ["10.0.33.0/26", "10.0.33.64/26", "10.0.33.128/26"] - firewall_allow_list = [".pypi.org", ".cran.r-project.org", ".pythonhosted.org", ".spark-packages.org", ".maven.org", "maven.apache.org", ".storage-download.googleapis.com"] - - // Custom Operation Mode: - custom_vpc_id = null - custom_private_subnet_ids = null // List of custom private subnet IDs required. - custom_sg_id = null - custom_relay_vpce_id = null - custom_workspace_vpce_id = null - - // OPTIONAL - Examples, Workspace Hardening, and Solution Accelerators: - enable_read_only_external_location_boolean = false // Set to true to enable a read-only external location. - read_only_data_bucket = null // S3 bucket name for read-only data. - read_only_external_location_admin = null // Admin for the external location. - - enable_cluster_boolean = false // Set to true to create a default Databricks clusters. - enable_admin_configs_boolean = false // Set to true to enable optional admin configurations. - enable_logging_boolean = false // Set to true to enable log delivery and creation of related assets (e.g. S3 bucket and IAM role) - - enable_restrictive_root_bucket_boolean = false // Set to true to enable a restrictive root bucket policy, this is subject to change and may cause unexpected issues in the event of a change. - enable_restrictive_s3_endpoint_boolean = false // Set to true to enable a restrictive S3 endpoint policy, this is subject to change and may cause unexpected issues in the event of a change. - enable_restrictive_sts_endpoint_boolean = false // Set to true to enable a restrictive STS endpoint policy, this is subject to change and may cause unexpected issues in the event of a change. - enable_restrictive_kinesis_endpoint_boolean = false // Set to true to enable a restrictive Kinesis endpoint policy, this is subject to change and may cause unexpected issues in the event of a change. - - enable_ip_boolean = false // Set to true to enable IP access list. - ip_addresses = ["X.X.X.X", "X.X.X.X/XX", "X.X.X.X/XX"] // Specify IP addresses for access. + sg_egress_ports = [443, 2443, 3306, 8443, 8444, 8445, 8446, 8447, 8448, 8449, 8450, 8451] - enable_system_tables_schema_boolean = false // Set to true to enable system table schemas + // REQUIRED IF USING NON-ROOT ACCOUNT CMK ADMIN: + # cmk_admin_arn = "arn:aws-us-gov:iam::123456789012:user/CMKAdmin" // Example CMK ARN - enable_sat_boolean = false // Set to true to enable Security Analysis Tool. https://github.com/databricks-industry-solutions/security-analysis-tool - enable_audit_log_alerting = false // Set to true to create 40+ queries for audit log alerting based on user activity. https://github.com/andyweaves/system-tables-audit-logs + // REQUIRED IF USING CUSTOM NETWORK: + # custom_vpc_id = "vpc-0abc123456def7890" // Example VPC ID + # custom_private_subnet_ids = ["subnet-0123456789abcdef0", "subnet-0abcdef1234567890"] // Example private subnet IDs + # custom_sg_id = "sg-0123456789abcdef0" // Example security group ID + # custom_relay_vpce_id = "vpce-0abc123456def7890" // Example PrivateLink endpoint ID for Databricks relay + # custom_workspace_vpce_id = "vpce-0abcdef1234567890" // Example PrivateLink endpoint ID for Databricks workspace } \ No newline at end of file diff --git a/aws-gov/tf/template.tfvars.example b/aws-gov/tf/template.tfvars.example index 88e026d..41d15c6 100644 --- a/aws-gov/tf/template.tfvars.example +++ b/aws-gov/tf/template.tfvars.example @@ -1,8 +1,13 @@ # Configuration Variables for AWS and Databricks +# Remove comments from file before using -aws_account_id = "" // AWS account ID where resources will be deployed. +// Databricks Variables +admin_user = "" // Admin user email. client_id = "" // Service principal ID for Databricks with admin permissions. client_secret = "" // Secret for the corresponding service principal. databricks_account_id = "" // Databricks account ID. databricks_gov_shard = "" // (civilian or dod) resource_prefix = "" // Prefix used for naming and tagging resources (e.g., S3 buckets, IAM roles). + +// AWS Variables +aws_account_id = "" // AWS account ID where resources will be deployed. \ No newline at end of file diff --git a/aws-gov/tf/variables.tf b/aws-gov/tf/variables.tf index 638e1ab..5e452e5 100644 --- a/aws-gov/tf/variables.tf +++ b/aws-gov/tf/variables.tf @@ -2,6 +2,11 @@ data "aws_availability_zones" "available" { state = "available" } +variable "admin_user" { + description = "Email of the admin user for the workspace and workspace catalog." + type = string +} + variable "aws_account_id" { description = "ID of the AWS account." type = string @@ -35,11 +40,20 @@ variable "region" { } variable "region_name" { - description = "Name of the AWS region. (e.g. pendleton)" - type = map(string) + description = "Name of the AWS region. (e.g. pendleton)" + type = map(string) + default = { + "civilian" = "pendleton" + "dod" = "pendleton-dod" + } +} + +variable "region_bucket_name" { + description = "Name of the AWS region. (e.g. pendleton)" + type = map(string) default = { "civilian" = "pendleton" - "dod" = "pendleton-dod" + "dod" = "pendleton-dod" } } diff --git a/aws/tf/modules/sra/variables.tf b/aws/tf/modules/sra/variables.tf index 9aa4f90..f454436 100644 --- a/aws/tf/modules/sra/variables.tf +++ b/aws/tf/modules/sra/variables.tf @@ -87,7 +87,7 @@ variable "network_configuration" { validation { condition = contains(["custom", "isolated"], var.network_configuration) - error_message = "Invalid network configuration. Allowed values are: sandbox, firewall, custom, isolated." + error_message = "Invalid network configuration. Allowed values are: custom, isolated." } }