Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding typed_json log format #5270

Merged
merged 4 commits into from
Jan 2, 2024
Merged

Conversation

tenshinhigashi
Copy link
Contributor

@tenshinhigashi tenshinhigashi commented Aug 28, 2023

Description

The purpose of this PR is to add typed_json log format as an option for envoy logging.

Related Issues

https://github.com/datawire/apro/issues/3408

Testing

All testing was done locally. I compared logs between standard json and the new typed_json.

This was the module I applied:

apiVersion: getambassador.io/v3alpha1
kind:  Module
metadata:
  name: ambassador
spec:
  config:
    envoy_log_type: typed_json
    envoy_log_format:
      {
        "protocol": "%PROTOCOL%",
        "duration": "%DURATION%"
      }

When looking at the logs after some simple curls, this is what I see:
Screen Shot 2023-08-30 at 10 57 07 AM

Note the duration values are ints rather than strings.

When looking at the logs with the same Module but json log type:
Screen Shot 2023-08-30 at 10 57 49 AM

The duration values are strings.

If no envoy_log_format is provided, the default format is the same as json's. Here is a full log line without any envoy_log_format:

{"requested_server_name":null,"user_agent":"curl/7.79.1","method":"GET","response_flags":"-","upstream_local_address":"10.42.0.12:55614","duration":0,"upstream_cluster":"quote_default","start_time":"2023-08-30T15:22:44.539Z","response_code":200,"downstream_local_address":"10.42.0.12:8443","upstream_service_time":"0","downstream_remote_address":"10.42.0.1:40191","x_forwarded_for":"10.42.0.1","upstream_transport_failure_reason":null,"path":"/backend/","authority":"34.133.232.144","istio_policy_status":null,"protocol":"HTTP/1.1","bytes_sent":220,"upstream_host":"10.43.1.125:80","request_id":"ae2e5f5f-d14d-4714-b138-37b3c4aefa70","bytes_received":0}

Note that the values are still formatted as expected.

  • null instead of "-"
  • int values where expected such as duration, response_code, bytes_sent, bytes_received, etc

Checklist

  • Does my change need to be backported to a previous release?

    • What backport versions were discussed with the Maintainers in the Issue?
  • I made sure to update CHANGELOG.md.

    Remember, the CHANGELOG needs to mention:

    • Any new features
    • Any changes to our included version of Envoy
    • Any non-backward-compatible changes
    • Any deprecations
  • This is unlikely to impact how Ambassador performs at scale.

    Remember, things that might have an impact at scale include:

    • Any significant changes in memory use that might require adjusting the memory limits
    • Any significant changes in CPU use that might require adjusting the CPU limits
    • Anything that might change how many replicas users should use
    • Changes that impact data-plane latency/scalability
  • My change is adequately tested.

    Remember when considering testing:

    • Your change needs to be specifically covered by tests.
      • Tests need to cover all the states where your change is relevant: for example, if you add a behavior that can be enabled or disabled, you'll need tests that cover the enabled case and tests that cover the disabled case. It's not sufficient just to test with the behavior enabled.
    • You also need to make sure that the entire area being changed has adequate test coverage.
      • If existing tests don't actually cover the entire area being changed, add tests.
      • This applies even for aspects of the area that you're not changing – check the test coverage, and improve it if needed!
    • We should lean on the bulk of code being covered by unit tests, but...
    • ... an end-to-end test should cover the integration points
  • I updated DEVELOPING.md with any any special dev tricks I had to use to work on this code efficiently.

  • The changes in this PR have been reviewed for security concerns and adherence to security best practices.

@tenshinhigashi tenshinhigashi changed the title [DRAFT] Adding typeed_json log format [DRAFT] Adding typed_json log format Aug 29, 2023
@tenshinhigashi tenshinhigashi changed the title [DRAFT] Adding typed_json log format Adding typed_json log format Aug 30, 2023
@tenshinhigashi tenshinhigashi force-pushed the tenshinhigashi/typed_json branch from 9d76f8a to 55838ae Compare August 31, 2023 16:32
@tenshinhigashi tenshinhigashi marked this pull request as ready for review August 31, 2023 18:23
LanceEa
LanceEa previously approved these changes Sep 7, 2023
Copy link
Contributor

@LanceEa LanceEa left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall it LGTM, just one suggestion to reduce duplication.

python/ambassador/envoy/v3/v3listener.py Show resolved Hide resolved
Alice-Lilith
Alice-Lilith previously approved these changes Sep 7, 2023
@tenshinhigashi tenshinhigashi dismissed stale reviews from Alice-Lilith and LanceEa via f01c71a September 11, 2023 15:49
@tenshinhigashi tenshinhigashi force-pushed the tenshinhigashi/typed_json branch from f01c71a to e872f94 Compare September 11, 2023 16:30
Signed-off-by: Tenshin Higashi <[email protected]>
Signed-off-by: Tenshin Higashi <[email protected]>
Signed-off-by: Tenshin Higashi <[email protected]>
@tenshinhigashi tenshinhigashi force-pushed the tenshinhigashi/typed_json branch 7 times, most recently from bddce04 to 484c7e6 Compare December 28, 2023 20:32
Signed-off-by: Tenshin Higashi <[email protected]>
@tenshinhigashi tenshinhigashi merged commit 19a1554 into master Jan 2, 2024
37 checks passed
@tenshinhigashi tenshinhigashi deleted the tenshinhigashi/typed_json branch January 2, 2024 18:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants