Make PD and TiKV wait until local IP address matches the one published to external DNS #5381
Conversation
… one published to DNS
This reverts commit 6a9990a.
|
Welcome @smineyev81! It looks like this is your first PR to pingcap/tidb-operator 🎉 |
…orDnsNameIpMatchOnStartup
…orDnsNameIpMatchOnStartup
addressed with moving new feature under feature flag which is exposed via new field: TidbCluster.Spec.WaitForDnsNameIpMatchOnStartup |
Codecov Report
Additional details and impacted files@@ Coverage Diff @@
## master #5381 +/- ##
==========================================
+ Coverage 61.56% 67.63% +6.07%
==========================================
Files 228 239 +11
Lines 28866 32644 +3778
==========================================
+ Hits 17772 22080 +4308
+ Misses 9349 8761 -588
- Partials 1745 1803 +58
|
regenerated and pushed |
|
/run-pull-e2e-kind |
|
/run-pull-e2e-kind-across-kubernetes |
|
/run-pull-e2e-kind-serial |
pkg/apis/pingcap/v1alpha1/types.go
Outdated
|
|
||
| // WaitForDnsNameIpMatchOnStartup indicates whether PD and TiKV has to wait | ||
| // until local IP address matches the one published to external DNS | ||
| WaitForDnsNameIpMatchOnStartup bool `json:"waitForDnsNameIpMatchOnStartup,omitempty"` |
There was a problem hiding this comment.
How about adding a comment to indicate it is only supported in start script v2 now?
There was a problem hiding this comment.
I didn't see any new commit after this comment.
There was a problem hiding this comment.
I didn't see any new commit after this comment.
sorry, push did not go through first time.
now changes are there
There was a problem hiding this comment.
you need to re-generate OpenAPI spec.
There was a problem hiding this comment.
what do you think if I change WaitForDnsNameIpMatchOnStartup bool with more generic field StartupScriptFeatureFlags []string json:"startupScriptFeatureFlags, omitempty"`` ? This will allow us to enable features without extra fields to CRD in future. See last commit.
PS: I want to change this as we foresee another feature that needs to be added to startup scripts very soon
…only with v2 scripts
…r into 1.5.1-airbnb-dev
|
/run-all-tests |
do not be surprised if some E2E tests failed as the CI environment is not stable now |
|
/run-pull-e2e-kind |
|
/run-pull-e2e-kind-across-kubernetes |
|
/run-pull-e2e-kind-br |
|
/run-pull-e2e-kind-across-kubernetes |
|
/run-pull-e2e-kind-br |
|
/run-pull-e2e-kind-tngm |
1 similar comment
|
/run-pull-e2e-kind-tngm |
|
/run-pull-e2e-kind-across-kubernetes |
|
[APPROVALNOTIFIER] This PR is APPROVED This pull-request has been approved by: csuzhangxc The full list of commands accepted by this bot can be found here. The pull request process is described here
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
[LGTM Timeline notifier]Timeline:
|
|
/cherry-pick release-1.5 |
|
@csuzhangxc: new pull request created to branch In response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
…d to external DNS (pingcap#5381)
…d to external DNS (#5381) (#5399) Co-authored-by: Sergey <[email protected]> Co-authored-by: csuzhangxc <[email protected]>
What problem does this PR solve?
Currently PD and TiKV pods await for dns name assigned to them to be resolvable by external DNS but they do not check if that IP matches IP assigned to them which causes problems on pod restart when DNS has old IP which might not match new IP. Which leads to false-positive: PD/TiKv pods are ready from k8s point of view but not accessible by dns name assigned to them.
What is changed and how does it work?
This change adds extra check to DP and TiKv startup scripts that makes sure that IP address received from external DNS matches the IP on a host (essentially
nslookupresult should match one of the IPs returned byhostname -I)Code changes
Tests
Side effects
Related changes
Release Notes
NONE