Inaccessible ResultMetadata in bedrockruntime service response

[pzierahn]

Describe the bug

While working with the AWS SDK Go v2 - bedrockruntime.InvokeModel() function, I encountered an issue where the ResultMetadata in the returned results is inaccessible. The map keys always appear as {} and there is no capacity to iterate over the underlying values map. Therefore, the Get, Set, and Has functions provided by the middleware.Metadata are rendered unusable. This is causing issues in handling and processing the result metadata properly.

Expected Behavior

The middleware.Metadata contained within the ResultMetadata map should be easily accessible and retrievable, allowing appropriate operations on the map like Get, Set, and Check for presence (Has) of specific metadata.

Current Behavior

Upon executing the bedrockruntime.InvokeModel() function, the following result shows that the ResultMetadata is returning a map with inaccessible keys, and I am unable to retrieve or operate on the stored metadata in a meaningful way.

Log Output of ResultMetadata:

2024/02/12 22:12:20 main.go:89: result.ResultMetadata: {map[{}:-215859000 {}:0x14000192250 {}:d87b6b1b-3b50-4487-b12e-9ae1b0d544b0 {}:{13937163434012294968 1580072667 0x1053c5160} {}:{0 63843369140 <nil>} {}:{[{<nil> false false {map[{}:-215859000 {}:0x14000192250 {}:d87b6b1b-3b50-4487-b12e-9ae1b0d544b0 {}:{13937163434012294968 1580072667 0x1053c5160} {}:{0 63843369140 <nil>}]}}]}]}

Reproduction Steps

1. Execute the bedrockruntime.InvokeModel() function with required input parameters.
2. Print the result.ResultMetadata to observe the inaccessible keys in the map.

Source Code Snippet:

package main

import (
    "context"
    "encoding/json"
    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/config"
    "github.com/aws/aws-sdk-go-v2/service/bedrockruntime"
    "log"
)

const region = "us-east-1"

type ClaudeRequest struct {
    Prompt            string   `json:"prompt"`
    MaxTokensToSample int      `json:"max_tokens_to_sample"`
    Temperature       float64  `json:"temperature,omitempty"`
    TopP              float64  `json:"top_p,omitempty"`
    TopK              int      `json:"top_k,omitempty"`
    StopSequences     []string `json:"stop_sequences,omitempty"`
}

type ClaudeResponse struct {
    Completion string `json:"completion"`
}

func main() {
    log.SetFlags(log.LstdFlags | log.Lshortfile)

    log.Printf("Using AWS region: %s\n", region)

    sdkConfig, err := config.LoadDefaultConfig(context.Background(), config.WithRegion(region))
    if err != nil {
        log.Println("Couldn't load default configuration. Have you set up your AWS account?")
        log.Println(err)
        return
    }

    client := bedrockruntime.NewFromConfig(sdkConfig)

    prompt := "Hello, how are you today?"
    wrappedPrompt := "Human: " + prompt + "\n\nAssistant:"
    request := ClaudeRequest{
        Prompt:            wrappedPrompt,
        MaxTokensToSample: 200,
        Temperature:       0.5,
    }

    body, err := json.Marshal(request)
    if err != nil {
        log.Fatal("failed to marshal request", err)
    }

    result, err := client.InvokeModel(context.Background(), &bedrockruntime.InvokeModelInput{
        ModelId:     aws.String("anthropic.claude-v2"),
        ContentType: aws.String("application/json"),
        Body:        body,
    })
    if err != nil {
        log.Fatalf("failed to invoke model: %v", err)
    }

    // The metadata are not accessible in the results.
    // The map keys are always {} and there is no way to iterate over the underlying values map.
    log.Printf("result.ResultMetadata: %v\n", result.ResultMetadata)
}

Possible Solution

No response

Additional Information/Context

No response

AWS Go SDK V2 Module Versions Used

github.com/aws/aws-sdk-go-v2 v1.24.1
github.com/aws/aws-sdk-go-v2/config v1.26.6
github.com/aws/aws-sdk-go-v2/service/bedrockruntime v1.5.6
github.com/aws/aws-sdk-go-v2/aws/protocol/eventstream v1.5.4 // indirect
github.com/aws/aws-sdk-go-v2/credentials v1.16.16 // indirect
github.com/aws/aws-sdk-go-v2/feature/ec2/imds v1.14.11 // indirect
github.com/aws/aws-sdk-go-v2/internal/configsources v1.2.10 // indirect
github.com/aws/aws-sdk-go-v2/internal/endpoints/v2 v2.5.10 // indirect
github.com/aws/aws-sdk-go-v2/internal/ini v1.7.3 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/accept-encoding v1.10.4 // indirect
github.com/aws/aws-sdk-go-v2/service/internal/presigned-url v1.10.10 // indirect
github.com/aws/aws-sdk-go-v2/service/sso v1.18.7 // indirect
github.com/aws/aws-sdk-go-v2/service/ssooidc v1.21.7 // indirect
github.com/aws/aws-sdk-go-v2/service/sts v1.26.7 // indirect
github.com/aws/smithy-go v1.19.0 // indirect

Compiler and Version used

go version go1.21.7 darwin/arm64

Operating System and version

macOS Sonoma Version 14.3.1

[RanVaknin]

Hi @pzierahn ,

This might be due a deserialization issue. We easily rule that out by inspecting the raw response data via enabling the response logs:

	cfg, err := config.LoadDefaultConfig(context.TODO(), config.WithRegion("us-east-1"), config.WithClientLogMode(aws.LogResponseWithBody))
	if err != nil {
		log.Fatalf("unable to load SDK config, %v", err)
	}

Can you share the output from the logger?

Thanks,
Ran~

[pzierahn]

Hi @RanVaknin,

thanks for the quick response! Here is my log output:

2024/02/12 23:17:01 main.go:30: Using AWS region: us-east-1
SDK 2024/02/12 23:17:03 DEBUG Response
HTTP/2.0 200 OK
Content-Length: 102
Content-Type: application/json
Date: Mon, 12 Feb 2024 22:17:03 GMT
X-Amzn-Bedrock-Input-Token-Count: 16
X-Amzn-Bedrock-Invocation-Latency: 1717
X-Amzn-Bedrock-Output-Token-Count: 13
X-Amzn-Requestid: 240529a6-3c4c-4c75-8a1b-9e22445f5248

{"completion":" I'm doing well, thanks for asking!","stop_reason":"stop_sequence","stop":"\n\nHuman:"}
2024/02/12 23:17:03 main.go:68: result.ResultMetadata: {map[{}:-707132000 {}:0x1400011a238 {}:240529a6-3c4c-4c75-8a1b-9e22445f5248 {}:{13937167603843070560 2130869501 0x104b59160} {}:{0 63843373023 <nil>} {}:{[{<nil> false false {map[{}:-707132000 {}:0x1400011a238 {}:240529a6-3c4c-4c75-8a1b-9e22445f5248 {}:{13937167603843070560 2130869501 0x104b59160} {}:{0 63843373023 <nil>}]}}]}]}

[lucix-aws]

The value keys within Metadata are unexported by design (much like how we don't export Context value keys). In practice if a metadata value is meant to be caller-facing, there is an API provided for accessing it.

See:

• https://aws.github.io/aws-sdk-go-v2/docs/middleware/#metadata-provided-by-the-sdk
• https://aws.github.io/aws-sdk-go-v2/docs/making-requests/#response-metadata

[RanVaknin]

Thanks @lucix-aws for the additional context.

@pzierahn ,

if you really need something from the metadata that is not exported, you can use a middleware:

	deserializeMiddleware := middleware.DeserializeMiddlewareFunc("myDeserializeMiddleware",
		func(ctx context.Context, input middleware.DeserializeInput, next middleware.DeserializeHandler) (middleware.DeserializeOutput, middleware.Metadata, error) {
			output, metadata, err := next.HandleDeserialize(ctx, input)
			if resp, ok := output.RawResponse.(*smithyhttp.Response); ok {
				fmt.Println(resp.Header) // print or do something else with the headers.
			}
			return output, metadata, err
		})
		
	client := bedrockruntime.NewFromConfig(sdkConfig, func(options *bedrockruntime.Options) {
		options.APIOptions = append(options.APIOptions, func(stack *middleware.Stack) error {
			return stack.Deserialize.Insert(deserializeMiddleware, "OperationDeserializer", middleware.After)
		})
	})

Let us know if you need anything else.
Ran~

[pzierahn]

Thank you so much! I believe I now have what I need to get started.

I'm aiming to keep track of the input/output token count for each request. Do you happen to know if there are any plans to develop tools or features that might simplify this process?

[lucix-aws]

if you really need something from the metadata that is not exported

To clarify, a field indexed via an unexported map key (as in, the key's value or type is not exported within its go package) that did not expose a getter API anywhere would be well and truly unretrievable, since that key could not functionally be used by a consumer in a Get or Has call. I'm not aware of any values we put into metadata that way, though (SDK-internal values like that basically always exist on the operation Context).

The example above is somewhat unrelated to that distinction, it shows how to inspect the raw transport response from the service via a handwritten middleware, which on a related note isn't actually necessary - there's a GetRawResponse metadata API available in the core module.