Fix confusing error message gcp

docs/_docs/02_features/04-state-backend.md

@@ -9,7 +9,7 @@ order: 204
 nav_title: Documentation
 nav_title_link: /docs/
 redirect_from:
-    - /docs/features/keep-your-remote-state-configuration-dry/
+  - /docs/features/keep-your-remote-state-configuration-dry/
 slug: state-backend
 ---
 
@@ -24,7 +24,7 @@ slug: state-backend
 
 OpenTofu/Terraform supports [remote state storage](https://www.terraform.io/docs/state/remote.html) via a variety of [backends](https://www.terraform.io/docs/backends) that you normally configure in your `.tf` files as follows:
 
-``` hcl
+```hcl
 terraform {
   backend "s3" {
     bucket         = "my-tofu-state"
@@ -103,7 +103,7 @@ interpolated values.
 To inherit this configuration in each unit, such as `mysql/terragrunt.hcl`, you can
 tell Terragrunt to automatically include all the settings from the root `root.hcl` file as follows:
 
-``` hcl
+```hcl
 include "root" {
   path = find_in_parent_folders("root.hcl")
 }
@@ -154,27 +154,27 @@ When you run `terragrunt` with a `remote_state` configuration, it will automatic
 
 - **S3 bucket**: If you are using the [S3 backend](https://opentofu.org/docs/language/settings/backends/s3) for remote state storage and the `bucket` you specify in `remote_state.config` doesn’t already exist, Terragrunt will create it automatically, with [versioning](https://docs.aws.amazon.com/AmazonS3/latest/dev/Versioning.html), [server-side encryption](https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingServerSideEncryption.html), and [access logging](https://docs.aws.amazon.com/AmazonS3/latest/dev/ServerLogs.html) enabled.
 
-    In addition, you can let terragrunt tag the bucket with custom tags that you specify in `remote_state.config.s3_bucket_tags`.
+  In addition, you can let terragrunt tag the bucket with custom tags that you specify in `remote_state.config.s3_bucket_tags`.
 
 - **DynamoDB table**: If you are using the [S3 backend](https://opentofu.org/docs/language/settings/backends/s3) for remote state storage and you specify a `dynamodb_table` (a [DynamoDB table used for locking](https://opentofu.org/docs/language/settings/backends/s3/#dynamodb-state-locking)) in `remote_state.config`, if that table doesn’t already exist, Terragrunt will create it automatically, with [server-side encryption](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/EncryptionAtRest.html) enabled, including a primary key called `LockID`.
 
-    You may configure custom endpoint for the AWS DynamoDB API using `remote_state.config.dynamodb_endpoint`.
+  You may configure custom endpoint for the AWS DynamoDB API using `remote_state.config.dynamodb_endpoint`.
 
-    In addition, you can let terragrunt tag the DynamoDB table with custom tags that you specify in `remote_state.config.dynamodb_table_tags`.
+  In addition, you can let terragrunt tag the DynamoDB table with custom tags that you specify in `remote_state.config.dynamodb_table_tags`.
 
 - **GCS bucket**: If you are using the [GCS backend](https://opentofu.org/docs/language/settings/backends/gcs) for remote state storage and the `bucket` you specify in `remote_state.config` doesn’t already exist, Terragrunt will create it automatically, with [versioning](https://cloud.google.com/storage/docs/object-versioning) enabled. For this to work correctly you must also specify `project` and `location` keys in `remote_state.config`, so Terragrunt knows where to create the bucket. You will also need to supply valid credentials using either `remote_state.config.credentials` or by setting the `GOOGLE_APPLICATION_CREDENTIALS` environment variable. If you want to skip creating the bucket entirely, simply set `skip_bucket_creation` to `true` and Terragrunt will assume the bucket has already been created. If you don’t specify `bucket` in `remote_state` then terragrunt will assume that you will pass `bucket` through `-backend-config` in `extra_arguments`.
 
-    We also strongly recommend you enable [Cloud Audit Logs](https://cloud.google.com/storage/docs/access-logs) to audit and track API operations performed against the state bucket.
+  We also strongly recommend you enable [Cloud Audit Logs](https://cloud.google.com/storage/docs/access-logs) to audit and track API operations performed against the state bucket.
 
-    In addition, you can let Terragrunt label the bucket with custom labels that you specify in `remote_state.config.gcs_bucket_labels`.
+  In addition, you can let Terragrunt label the bucket with custom labels that you specify in `remote_state.config.gcs_bucket_labels`.
 
 **Note**: If you specify a `profile` key in `remote_state.config`, Terragrunt will automatically use this AWS profile when creating the S3 bucket or DynamoDB table.
 
 **Note**: You can disable automatic remote state initialization by setting `remote_state.disable_init`, this will skip the automatic creation of remote state resources and will execute `terraform init` passing the `backend=false` option. This can be handy when running commands such as `validate-all` as part of a CI process where you do not want to initialize remote state.
 
 The following example demonstrates using an environment variable to configure this option:
 
-``` hcl
+```hcl
 remote_state {
   # ...
 
@@ -232,7 +232,7 @@ For the `s3` backend, the following config options can be used for S3-compatible
 
 **Note**: The `skip_bucket_accesslogging` is now DEPRECATED. It is replaced by `accesslogging_bucket_name`. Please read below for more details on when to use the new config option.
 
-``` hcl
+```hcl
 remote_state {
   # ...
 
@@ -261,7 +261,7 @@ Further, the `config` options `s3_bucket_tags`, `dynamodb_table_tags`, `accesslo
 
 For the `gcs` backend, the following config options can be used for GCS-compatible object stores, as necessary:
 
-``` hcl
+```hcl
 remote_state {
  # ...
 

remote/remote_state_gcs.go

@@ -177,12 +177,12 @@ func (initializer GCSInitializer) buildInitializerCacheKey(gcsConfig *RemoteStat
 // Initialize the remote state GCS bucket specified in the given config. This function will validate the config
 // parameters, create the GCS bucket if it doesn't already exist, and check that versioning is enabled.
 func (initializer GCSInitializer) Initialize(ctx context.Context, remoteState *RemoteState, terragruntOptions *options.TerragruntOptions) error {
-	gcsConfigExtended, err := parseExtendedGCSConfig(remoteState.Config)
+	gcsConfigExtended, err := ParseExtendedGCSConfig(remoteState.Config)
 	if err != nil {
 		return err
 	}
 
-	if err := validateGCSConfig(gcsConfigExtended); err != nil {
+	if err := ValidateGCSConfig(gcsConfigExtended); err != nil {
 		return err
 	}
 
@@ -254,8 +254,8 @@ func parseGCSConfig(config map[string]interface{}) (*RemoteStateConfigGCS, error
 	return &gcsConfig, nil
 }
 
-// Parse the given map into a GCS config
-func parseExtendedGCSConfig(config map[string]interface{}) (*ExtendedRemoteStateConfigGCS, error) {
+// ParseExtendedGCSConfig parses the given map into a GCS config.
+func ParseExtendedGCSConfig(config map[string]interface{}) (*ExtendedRemoteStateConfigGCS, error) {
 	var (
 		gcsConfig      RemoteStateConfigGCS
 		extendedConfig ExtendedRemoteStateConfigGCS
@@ -274,14 +274,53 @@ func parseExtendedGCSConfig(config map[string]interface{}) (*ExtendedRemoteState
 	return &extendedConfig, nil
 }
 
-// Validate all the parameters of the given GCS remote state configuration
-func validateGCSConfig(extendedConfig *ExtendedRemoteStateConfigGCS) error {
-	var config = extendedConfig.remoteStateConfigGCS
+// ValidateGCSConfig validates the configuration for GCS remote state.
+func ValidateGCSConfig(extendedConfig *ExtendedRemoteStateConfigGCS) error {
+	config := extendedConfig.remoteStateConfigGCS
 
+	// If skip_bucket_creation is true, bypass all validation
+	// This allows using existing buckets without restrictions
+	if extendedConfig.SkipBucketCreation {
+		return nil
+	}
+
+	// Bucket is always a required configuration parameter
 	if config.Bucket == "" {
 		return errors.New(MissingRequiredGCSRemoteStateConfig("bucket"))
 	}
 
+	// If both project and location are provided, the configuration is valid
+	if extendedConfig.Project != "" && extendedConfig.Location != "" {
+		return nil
+	}
+
+	// Create a GCS client to check bucket existence
+	gcsClient, err := CreateGCSClient(config)
+	if err != nil {
+		return fmt.Errorf("error creating GCS client: %w", err)
+	}
+
+	defer func() {
+		if closeErr := gcsClient.Close(); closeErr != nil {
+			log.Warnf("Error closing GCS client: %v", closeErr)
+		}
+	}()
+
+	// Check if the bucket exists
+	bucketExists := DoesGCSBucketExist(gcsClient, &config)
+	if bucketExists {
+		return nil
+	}
+
+	// At this point, the bucket doesn't exist and we need both project and location
+	if extendedConfig.Project == "" {
+		return errors.New(MissingRequiredGCSRemoteStateConfig("project"))
+	}
+
+	if extendedConfig.Location == "" {
+		return errors.New(MissingRequiredGCSRemoteStateConfig("location"))
+	}
+
 	return nil
 }
 
@@ -424,7 +463,7 @@ func CreateGCSBucket(gcsClient *storage.Client, config *ExtendedRemoteStateConfi
 	}
 
 	if err := bucket.Create(ctx, projectID, bucketAttrs); err != nil {
-		return errors.Errorf("error creating GCS bucket %s: %w", config.remoteStateConfigGCS.Bucket, err)
+		return fmt.Errorf("error creating GCS bucket %s: %w", config.remoteStateConfigGCS.Bucket, err)
 	}
 
 	return nil
@@ -452,7 +491,7 @@ func WaitUntilGCSBucketExists(gcsClient *storage.Client, config *RemoteStateConf
 
 // DoesGCSBucketExist returns true if the GCS bucket specified in the given config exists and the current user has the
 // ability to access it.
-func DoesGCSBucketExist(gcsClient *storage.Client, config *RemoteStateConfigGCS) bool {
+var DoesGCSBucketExist = func(gcsClient *storage.Client, config *RemoteStateConfigGCS) bool {
 	ctx := context.Background()
 
 	// Creates a Bucket instance.
@@ -476,7 +515,7 @@ func DoesGCSBucketExist(gcsClient *storage.Client, config *RemoteStateConfigGCS)
 }
 
 // CreateGCSClient creates an authenticated client for GCS
-func CreateGCSClient(gcsConfigRemote RemoteStateConfigGCS) (*storage.Client, error) {
+var CreateGCSClient = func(gcsConfigRemote RemoteStateConfigGCS) (*storage.Client, error) {
 	ctx := context.Background()
 
 	var opts []option.ClientOption

remote/remote_state_gcs_test.go

@@ -5,6 +5,7 @@ package remote_test
 import (
 	"testing"
 
+	"cloud.google.com/go/storage"
 	"github.com/gruntwork-io/terragrunt/options"
 	"github.com/gruntwork-io/terragrunt/remote"
 	"github.com/stretchr/testify/assert"
@@ -130,3 +131,108 @@ func TestGcpConfigValuesEqual(t *testing.T) {
 		})
 	}
 }
+
+func TestValidateGCSConfig(t *testing.T) {
+	testCases := map[string]struct {
+		config           map[string]interface{}
+		expectedError    bool
+		mockBucketExists bool // Simulate whether the bucket exists
+	}{
+		"Valid_config_with_project_and_location": {
+			config: map[string]interface{}{
+				"bucket":   "test-bucket",
+				"project":  "test-project",
+				"location": "us-central1",
+			},
+			expectedError:    false,
+			mockBucketExists: false,
+		},
+		"Valid_config_with_skip_bucket_creation": {
+			config: map[string]interface{}{
+				"bucket":               "test-bucket",
+				"skip_bucket_creation": true,
+			},
+			expectedError:    false,
+			mockBucketExists: false,
+		},
+		"Missing_bucket": {
+			config: map[string]interface{}{
+				"project":  "test-project",
+				"location": "us-central1",
+			},
+			expectedError:    true,
+			mockBucketExists: false,
+		},
+		"Missing_project_when_bucket_does_not_exist": {
+			config: map[string]interface{}{
+				"bucket":   "test-bucket",
+				"location": "us-central1",
+			},
+			expectedError:    true,
+			mockBucketExists: false,
+		},
+		"Missing_location_when_bucket_does_not_exist": {
+			config: map[string]interface{}{
+				"bucket":  "test-bucket",
+				"project": "test-project",
+			},
+			expectedError:    true,
+			mockBucketExists: false,
+		},
+		"Existing_bucket_without_project_and_location_when_skip_bucket_creation_is_true": {
+			config: map[string]interface{}{
+				"bucket":               "test-bucket",
+				"skip_bucket_creation": true,
+			},
+			expectedError:    false,
+			mockBucketExists: true,
+		},
+		"Existing_bucket_without_project_and_location_when_bucket_exists": {
+			config: map[string]interface{}{
+				"bucket": "test-bucket",
+			},
+			expectedError:    false,
+			mockBucketExists: true,
+		},
+		"Missing_project_when_bucket_exists": {
+			config: map[string]interface{}{
+				"bucket":   "test-bucket",
+				"location": "us-central1",
+			},
+			expectedError:    false,
+			mockBucketExists: true,
+		},
+		"Missing_location_when_bucket_exists": {
+			config: map[string]interface{}{
+				"bucket":  "test-bucket",
+				"project": "test-project",
+			},
+			expectedError:    false,
+			mockBucketExists: true,
+		},
+	}
+
+	// Mock the DoesGCSBucketExist function to simulate bucket existence
+	originalDoesGCSBucketExist := remote.DoesGCSBucketExist
+	defer func() { remote.DoesGCSBucketExist = originalDoesGCSBucketExist }()
+
+	for name, tc := range testCases {
+		t.Run(name, func(t *testing.T) {
+			// Set up the mock bucket existence check
+			remote.DoesGCSBucketExist = func(gcsClient *storage.Client, config *remote.RemoteStateConfigGCS) bool {
+				return tc.mockBucketExists
+			}
+
+			extendedConfig, err := remote.ParseExtendedGCSConfig(tc.config)
+			require.NoError(t, err)
+
+			err = remote.ValidateGCSConfig(extendedConfig)
+
+			if tc.expectedError {
+				require.Error(t, err)
+			} else {
+				require.NoError(t, err)
+			}
+		})
+	}
+}