v2.13.3 reports OutOfSync object with empty diff

[sherif-fanous]

Describe the bug

Upgrading the Argo CD Helm chart from v7.7.12 to v7.7.13 upgrades Argo CD from v2.13.2 to v2.13.3. Following this upgrade a single object is constantly reported as OutOfSync however when inspecting the object the diff reported by Argo CD is empty.

Reverting back to Helm chart v7.7.12 and Argo CD v.2.13.2 the object is then back to being Synced

To Reproduce

The only object this is happening with is the kubernetes-dashboard-csrf Secret from the Kubernetes Dashboard application.

Expected behavior

An object with an empty diff should not be OutOfSync

Screenshots

Version

{
    "Version": "v2.13.3+a25c8a0",
    "BuildDate": "2025-01-03T18:44:26Z",
    "GitCommit": "a25c8a0eef7830be0c2c9074c92dbea8ff23a962",
    "GitTreeState": "clean",
    "GoVersion": "go1.23.1",
    "Compiler": "gc",
    "Platform": "linux/amd64",
    "KustomizeVersion": "v5.4.3 2024-07-19T16:40:33Z",
    "HelmVersion": "v3.15.4+gfa9efb0",
    "KubectlVersion": "v0.31.0",
    "JsonnetVersion": "v0.20.0"
}

[agaudreault]

@sherif-fanous do you have Server-side diff enabled? If so, do you have the same issue when disabling it?

[sherif-fanous]

@sherif-fanous do you have Server-side diff enabled? If so, do you have the same issue when disabling it?

Yes, I'm using server-side diff. I'll try to find an env where I can disable it.

[crenshaw-dev]

@sherif-fanous you can disable it for a single application: https://argo-cd.readthedocs.io/en/stable/user-guide/diff-strategies/#enabling-it

[sherif-fanous]

@agaudreault Upgraded back to v2.13.3 and disabled server side diff for the application and it is no longer out of sync.

[crenshaw-dev]

Strangely, none of the changes in 2.13.3 looks remotely relevant: v2.13.2...v2.13.3

[sherif-fanous]

At the risk of looking like a fool since I know nothing about the codebase but there is a change in server/application/application.go introduced by 030a7be#diff-18d6bf6e8588f888e23c671094d44b6bba958c624dccb0f734f40cfe91b2f1f0R372 that seems to be making a change to how specs are determined to be equal.

[crenshaw-dev]

I'd be very surprised if that were it, but I don't have a better theory. Do you have time to revert that commit on v2.13.3 and run a custom image?

[sherif-fanous]

I'd be very surprised if that were it, but I don't have a better theory. Do you have time to revert that commit on v2.13.3 and run a custom image?

Sure. I have an env I can run this on