Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

opt: make clear logs #394

Merged
merged 1 commit into from
Jan 20, 2025
Merged

opt: make clear logs #394

merged 1 commit into from
Jan 20, 2025

Conversation

w41ter
Copy link
Contributor

@w41ter w41ter commented Jan 16, 2025

No description provided.

github-advanced-security[bot]
github-advanced-security bot found potential problems Jan 16, 2025
View reviewed changes
pkg/ccr/job.go
@@ -1586,14 +1591,14 @@
}

rollback := func(err error, inMemoryData *inMemoryData) {
log.Errorf("need rollback, err: %+v", err)
log.Errorf("txn %d need rollback, commitSeq: %d, label: %s, err: %+v",
inMemoryData.TxnId, inMemoryData.CommitSeq, inMemoryData.Label, err)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

Sensitive data returned by an access to Passwd
Loading
flows to a logging call.
Sensitive data returned by an access to Passwd
Loading
flows to a logging call.
Sensitive data returned by an access to Passwd
Loading
flows to a logging call.
Sensitive data returned by an access to Passwd
Loading
flows to a logging call.
Sensitive data returned by an access to Passwd
Loading
flows to a logging call.
Sensitive data returned by an access to Passwd
Loading
flows to a logging call.
Sensitive data returned by an access to Passwd
Loading
flows to a logging call.

Copilot Autofix AI 6 days ago

Copilot could not generate an autofix suggestion

Copilot could not generate an autofix suggestion for this alert. Try pushing a new commit or if the problem persists contact support.

pkg/rpc/fe.go
@@ -857,13 +855,13 @@
}

func (rpc *singleFeClient) GetDbMeta(spec *base.Spec) (*festruct.TGetMetaResult_, error) {
log.Debugf("GetMetaDb, addr: %s, spec: %s", rpc.Address(), spec)
log.Tracef("GetMetaDb, addr: %s, spec: %s", rpc.Address(), spec)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

Sensitive data returned by an access to Password
Loading
flows to a logging call.
Sensitive data returned by an access to password
Loading
flows to a logging call.

Copilot Autofix AI 7 days ago

To fix the problem, we need to ensure that sensitive information, such as passwords, is not logged in clear text. We can achieve this by either omitting the sensitive information from the logs or obfuscating it before logging. In this case, we will omit the sensitive information from the logs to maintain security.

We will modify the logging statements in the pkg/rpc/fe.go file to exclude the Password field from the spec object. Specifically, we will update the logging statements on lines 820, 829, 858, 864, and 878 to ensure that sensitive information is not logged.

Suggested changeset 1
pkg/rpc/fe.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/pkg/rpc/fe.go b/pkg/rpc/fe.go
--- a/pkg/rpc/fe.go
+++ b/pkg/rpc/fe.go
@@ -819,3 +819,3 @@
 func (rpc *singleFeClient) GetMasterToken(spec *base.Spec) (*festruct.TGetMasterTokenResult_, error) {
-	log.Tracef("Call GetMasterToken, addr: %s, spec: %s", rpc.Address(), spec)
+	log.Tracef("Call GetMasterToken, addr: %s, user: %s", rpc.Address(), spec.User)
 
@@ -857,3 +857,3 @@
 func (rpc *singleFeClient) GetDbMeta(spec *base.Spec) (*festruct.TGetMetaResult_, error) {
-	log.Tracef("GetMetaDb, addr: %s, spec: %s", rpc.Address(), spec)
+	log.Tracef("GetMetaDb, addr: %s, user: %s", rpc.Address(), spec.User)
 
@@ -863,3 +863,3 @@
 func (rpc *singleFeClient) GetTableMeta(spec *base.Spec, tableIds []int64) (*festruct.TGetMetaResult_, error) {
-	log.Tracef("GetMetaTable, addr: %s, tableIds: %v", rpc.Address(), tableIds)
+	log.Tracef("GetMetaTable, addr: %s, user: %s, tableIds: %v", rpc.Address(), spec.User, tableIds)
 
@@ -877,3 +877,3 @@
 func (rpc *singleFeClient) GetBackends(spec *base.Spec) (*festruct.TGetBackendMetaResult_, error) {
-	log.Tracef("GetBackends, addr: %s, spec: %s", rpc.Address(), spec)
+	log.Tracef("GetBackends, addr: %s, user: %s", rpc.Address(), spec.User)
 
EOF
@@ -819,3 +819,3 @@
func (rpc *singleFeClient) GetMasterToken(spec *base.Spec) (*festruct.TGetMasterTokenResult_, error) {
log.Tracef("Call GetMasterToken, addr: %s, spec: %s", rpc.Address(), spec)
log.Tracef("Call GetMasterToken, addr: %s, user: %s", rpc.Address(), spec.User)

@@ -857,3 +857,3 @@
func (rpc *singleFeClient) GetDbMeta(spec *base.Spec) (*festruct.TGetMetaResult_, error) {
log.Tracef("GetMetaDb, addr: %s, spec: %s", rpc.Address(), spec)
log.Tracef("GetMetaDb, addr: %s, user: %s", rpc.Address(), spec.User)

@@ -863,3 +863,3 @@
func (rpc *singleFeClient) GetTableMeta(spec *base.Spec, tableIds []int64) (*festruct.TGetMetaResult_, error) {
log.Tracef("GetMetaTable, addr: %s, tableIds: %v", rpc.Address(), tableIds)
log.Tracef("GetMetaTable, addr: %s, user: %s, tableIds: %v", rpc.Address(), spec.User, tableIds)

@@ -877,3 +877,3 @@
func (rpc *singleFeClient) GetBackends(spec *base.Spec) (*festruct.TGetBackendMetaResult_, error) {
log.Tracef("GetBackends, addr: %s, spec: %s", rpc.Address(), spec)
log.Tracef("GetBackends, addr: %s, user: %s", rpc.Address(), spec.User)

Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
pkg/rpc/fe.go
@@ -877,7 +875,7 @@
}

func (rpc *singleFeClient) GetBackends(spec *base.Spec) (*festruct.TGetBackendMetaResult_, error) {
log.Debugf("GetBackends, addr: %s, spec: %s", rpc.Address(), spec)
log.Tracef("GetBackends, addr: %s, spec: %s", rpc.Address(), spec)

Check failure

Code scanning / CodeQL

Clear-text logging of sensitive information High

Sensitive data returned by an access to Password
Loading
flows to a logging call.
Sensitive data returned by an access to password
Loading
flows to a logging call.

Copilot Autofix AI 7 days ago

To fix the problem, we need to ensure that sensitive information such as passwords is not logged in clear text. The best way to fix this without changing existing functionality is to modify the logging statements to exclude the sensitive information. Specifically, we should remove the password from the spec object before logging it.

  1. In the file pkg/rpc/fe.go, modify the logging statements in the GetDbMeta, GetTableMeta, and GetBackends methods to exclude the password.
  2. In the file cmd/thrift_get_meta/thrift_get_meta.go, ensure that the spec object is not logged with sensitive information.
Suggested changeset 1
pkg/rpc/fe.go

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/pkg/rpc/fe.go b/pkg/rpc/fe.go
--- a/pkg/rpc/fe.go
+++ b/pkg/rpc/fe.go
@@ -857,3 +857,3 @@
 func (rpc *singleFeClient) GetDbMeta(spec *base.Spec) (*festruct.TGetMetaResult_, error) {
-	log.Tracef("GetMetaDb, addr: %s, spec: %s", rpc.Address(), spec)
+	log.Tracef("GetMetaDb, addr: %s, user: %s, db: %s", rpc.Address(), spec.User, spec.Database)
 
@@ -863,3 +863,3 @@
 func (rpc *singleFeClient) GetTableMeta(spec *base.Spec, tableIds []int64) (*festruct.TGetMetaResult_, error) {
-	log.Tracef("GetMetaTable, addr: %s, tableIds: %v", rpc.Address(), tableIds)
+	log.Tracef("GetMetaTable, addr: %s, user: %s, tableIds: %v", rpc.Address(), spec.User, tableIds)
 
@@ -877,3 +877,3 @@
 func (rpc *singleFeClient) GetBackends(spec *base.Spec) (*festruct.TGetBackendMetaResult_, error) {
-	log.Tracef("GetBackends, addr: %s, spec: %s", rpc.Address(), spec)
+	log.Tracef("GetBackends, addr: %s, user: %s, cluster: %s", rpc.Address(), spec.User, spec.Cluster)
 
EOF
@@ -857,3 +857,3 @@
func (rpc *singleFeClient) GetDbMeta(spec *base.Spec) (*festruct.TGetMetaResult_, error) {
log.Tracef("GetMetaDb, addr: %s, spec: %s", rpc.Address(), spec)
log.Tracef("GetMetaDb, addr: %s, user: %s, db: %s", rpc.Address(), spec.User, spec.Database)

@@ -863,3 +863,3 @@
func (rpc *singleFeClient) GetTableMeta(spec *base.Spec, tableIds []int64) (*festruct.TGetMetaResult_, error) {
log.Tracef("GetMetaTable, addr: %s, tableIds: %v", rpc.Address(), tableIds)
log.Tracef("GetMetaTable, addr: %s, user: %s, tableIds: %v", rpc.Address(), spec.User, tableIds)

@@ -877,3 +877,3 @@
func (rpc *singleFeClient) GetBackends(spec *base.Spec) (*festruct.TGetBackendMetaResult_, error) {
log.Tracef("GetBackends, addr: %s, spec: %s", rpc.Address(), spec)
log.Tracef("GetBackends, addr: %s, user: %s, cluster: %s", rpc.Address(), spec.User, spec.Cluster)

Copilot is powered by AI and may make mistakes. Always verify output.
Positive Feedback
Negative Feedback

Provide additional feedback

Please help us improve GitHub Copilot by sharing more details about this comment.

Please select one or more of the options
@w41ter w41ter merged commit 331ce25 into dev Jan 20, 2025
3 of 5 checks passed
wyxxxcat pushed a commit to wyxxxcat/ccr-syncer that referenced this pull request Jan 22, 2025
@w41ter @wyxxxcat
opt: make clear logs (selectdb#394)
005db63
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@w41ter