Cluster not accessible after keeping it down for prolonged period

[vaibhavdhingra]

Recently, we observed an unusual behaviour with Okd 4.15 clusters: if the cluster is kept down for a prolonged period (e.g., more than 30 days), causing problems when nodes try to rejoin the cluster.
Later on we have identified that CSRs are pending to approve.
Any solution available to auto approved the pending CSRs when we start the cluster?

[SupremeMortal]

It's expected behaviour for you to verify and approve the certificate signing requests as mentioned in the docs during an extended period of downtime.

The next best thing you can do is script a command to run when your cluster is online e.g.

oc get csr -o go-template='{{range .items}}{{if not .status}}{{.metadata.name}}{{"\n"}}{{end}}{{end}}' | xargs --no-run-if-empty oc adm certificate approve

[vaibhavdhingra]

@SupremeMortal , Thank you for your response! Indeed, this was the approach we followed previously to restore the cluster to a healthy state—by SSHing directly into one of the master nodes, referencing the kubeConfig file generated during installation, and manually approving the pending CSRs.

However, we are now looking for an automated solution to eliminate the need for manual intervention as cluster administrators.

Currently, we’ve adopted the following approach, although we are unsure if it will fully resolve the issue:

We bring up the master nodes daily for three hours, allowing the cluster sufficient time to renew its control plane certificates. During this time, the Machine Config Auto Approver service, which operates exclusively on the master nodes, will automatically approve pending CSRs when worker nodes rejoin the cluster.

Unlike Cert-Manager, which provides certificate renewal timelines, OKD's certificate renewal service lacks such visibility. As a result, we have adopted the practice of starting the cluster's control plane nodes daily for 3 hours.