-
Notifications
You must be signed in to change notification settings - Fork 42
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: added hook to job to delete/recreate during upgrade/rollback #70
Conversation
I've noticed something awkward when deleting a job before the deployment can init is causing the k8s-wait-for initcontainer to not start if you're using v1.7 or later. Gonna see what I can figure out. |
So with groundnuty/k8s-wait-for v1.6, it didn't cause things to get jammed up with starting the deployment if the job completed and got deleted. v1.7 and later, it DID cause the deployment to fail to init due to the job not being found. Thus, deleting the job at the end of an upgrade/rollback could cause issues with starting the deployment. Setting the job to be deleted at the beginning of the install/upgrade/rollback process will allow the job to linger as a way to ensure that the deployment initContainer will be able to successfully validate job completion. groundnuty/k8s-wait-for can remain at v1.6 for flexibility (if the job doesn't exist, the deployment can still init), but using v1.7 and later can help ensure that the new OpenFGA release doesn't actually start until after the db migration job has completed. Also, groundnuty/k8s-wait-for v1.7 and later uses |
Found an issue that by doing this, the job doesn't use the latest image, but the previously deployed one. |
This solution still isn't my favorite. If you deploy with helm using the |
I've gotten this style to work fairly consistently with a separate helm chart that I manage. Able to consistently deploy and update without issue. |
@jliedy sorry this one has fallen through the cracks. We've been heads down working on some big features in the core OpenFGA server. Would you mind rebasing on 🙇 |
I've had my head down in some stuff myself, but we've been running in openshift what I've suggested with the change here. I'll try to get back to this this week with an update. |
@jliedy just following up? Any updates/progress since our last chat? |
Sorry, no, I stink. Had some personal things that needed to be taken care of. Lemme go ahead and get cracking on this now. |
Signed-off-by: Jonathan Liedy <[email protected]>
the beginning of the helm upgrade/rollback process. Update the role rules to allow both list and get to support using the latest release of groundnuty/k8s-wait-for Update groundnuty/k8s-wait-for to use v2.0 Signed-off-by: Jonathan Liedy <[email protected]>
to job config in order for the job to be deleted when "helm uninstall" is run. Signed-off-by: Jonathan Liedy <[email protected]>
so it will be able to actually pull updated configmaps/secrets/image references Signed-off-by: Jonathan Liedy <[email protected]>
k8s-wait-for v2.0. This does require that the successfully completed job stay in the kubernetes cluster as the check will fail if the job does not exist. Signed-off-by: Jonathan Liedy <[email protected]>
Signed-off-by: Jonathan Liedy <[email protected]>
This wasn't supposed to be closed. |
|
Hi @jliedy - thanks! I converted it to a draft PR. Once you believe it's ready for review, feel free to mark it as such |
Signed-off-by: Jonathan Liedy <[email protected]>
I think it's ready to be reviewed. Let me know if you'd prefer me to add the job migration annotations for the helm hooks to the values.yaml file instead of the job.yaml file. |
Thanks! Will refer to the team for review |
@jliedy everything looks good, but I think this would be a good idea as it is more extensible. If you are using the chart and you wish not to have these hooks, then that should be overridable with the values. Mind making that change real quick and then we can merge. |
Signed-off-by: Jonathan Liedy <[email protected]>
Signed-off-by: Jonathan Liedy <[email protected]>
Yeah, I could make an argument for it either way so I figured I'd put the option out there. Done. |
Added annotations to the job in order for the migration job to be deleted after a successful run.
Description
In reference to Issue #69 I've added hook annotations to the migration job in order for the job to be deleted and then recreated during a helm upgrade/rollback.
References
For views on how to add hooks to jobs, one can reference the Helm documentation at: https://helm.sh/docs/topics/charts_hooks/
Review Checklist
main