Skip to content

Commit

Permalink
Not marking pods as done if the ReportDone api call fails (#397)
Browse files Browse the repository at this point in the history
Currently the API call can fail, but the pods are still marked and done and deleted.

This causes the lease of these jobs to expire (as they weren't properly closed off) and the jobs to be resubmitted.
  • Loading branch information
JamesMurkin authored Jun 26, 2020
1 parent fa8b8c8 commit 0074e64
Showing 1 changed file with 3 additions and 1 deletion.
4 changes: 3 additions & 1 deletion internal/executor/service/job_lease.go
Original file line number Diff line number Diff line change
Expand Up @@ -142,8 +142,10 @@ func (jobLeaseService *JobLeaseService) ReportDone(pods []*v1.Pod) error {
defer cancel()
log.Infof("Reporting done for jobs %s", strings.Join(jobIds, ","))
_, err := jobLeaseService.queueClient.ReportDone(ctx, &api.IdList{Ids: jobIds})
if err == nil {
jobLeaseService.markAsDone(pods)
}

jobLeaseService.markAsDone(pods)
return err
}

Expand Down

0 comments on commit 0074e64

Please sign in to comment.