Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

move data extract jobs to use k8s #36137

Open
wants to merge 10 commits into
base: master
Choose a base branch
from

Conversation

GEvans-HMCTS
Copy link
Contributor

@GEvans-HMCTS GEvans-HMCTS commented Jan 15, 2025

Jira link

Resolves: https://tools.hmcts.net/jira/browse/DTSPB-2859

Change description

Moving Probate data extract jobs to use k8s cron

🤖AEP PR SUMMARY🤖

I'm a bot that generates AI summaries of pull requests, see AEP for more details

  • Updated kustomization.yaml:

    • Added new cron jobs for iron mountain extract, exela extract, grant delayed extract, and grant awaiting documentation extract.
  • Updated demo-image-policy.yaml:

    • Updated the pattern for filterTags.
  • Added new files for:

    • probate-cron-exela-extract/demo/demo.yaml
    • probate-cron-exela-extract/image-policy.yaml
    • probate-cron-exela-extract/image-repo.yaml
    • probate-cron-exela-extract/probate-cron-exela-extract.yaml
    • probate-cron-grant-awaiting-documentation-extract/demo/demo.yaml
    • probate-cron-grant-awaiting-documentation-extract/image-policy.yaml
    • probate-cron-grant-awaiting-documentation-extract/image-repo.yaml
    • probate-cron-grant-awaiting-documentation-extract/probate-cron-grant-awaiting-documentation-extract.yaml
    • probate-cron-grant-delayed-extract/demo/demo.yaml
    • probate-cron-grant-delayed-extract/image-policy.yaml
    • probate-cron-grant-delayed-extract/image-repo.yaml
    • probate-cron-grant-delayed-extract/probate-cron-grant-delayed-extract.yaml
    • probate-cron-iron-mountain-extract/demo/demo.yaml
    • probate-cron-iron-mountain-extract/image-policy.yaml
    • probate-cron-iron-mountain-extract/image-repo.yaml
    • probate-cron-iron-mountain-extract/probate-cron-iron-mountain-extract.yaml
    • probate-cron-iron-mountain-extract/image-policy.yaml
    • probate-cron-iron-mountain-extract/image-repo.yaml
    • probate-cron-iron-mountain-extract/probate-cron-iron-mountain-extract.yaml
    • probate-cron-iron-mountain-extract/image-policy.yaml
    • probate-cron-iron-mountain-extract/image-repo.yaml
    • probate-cron-iron-mountain-extract/probate-cron-iron-mountain-extract.yaml

job:
environment:
TASK_NAME: IronMountainExtractTask
VAR_FV2: trigger-2
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

does this var actually do anything for the cron trigger jobs?

IDAM_SERVICE_HOST: https://idam-api.platform.hmcts.net
PRINTSERVICE_HOST: https://return-case-doc.ccd.platform.hmcts.net
TASK_NAME: IronMountainExtractTask
schedule: "0 5 2 * *"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is 0500 every 2nd of the month? is that what we intend

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this has been updated to be 5 mins past 2am every day
the others are set to match their previous times from the orch service. This issue here came about as the old cron has 6 values not 5

Copy link
Contributor

@tom-saunders-cts tom-saunders-cts Jan 23, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://kubernetes.io/docs/concepts/workloads/controllers/cron-jobs/#schedule-syntax is this the cron schedule type being used? if so this is 0500 every 2nd of the month was looking at the old version, ignore

VAR_FV2: trigger-2
useInterpodAntiAffinity: true
image: hmctspublic.azurecr.io/probate/back-office:prod-6707e09-20250109124038 #{"$imagepolicy": "flux-system:probate-back-office"}
schedule: 0 * * * *
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if an environment doesn't override this it will run on the hour every hour?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we expected the env directories to ever contain anything bar these env.yaml files? if not can we have them as just env.yaml in the main directory?

@GEvans-HMCTS GEvans-HMCTS self-assigned this Jan 24, 2025
@GEvans-HMCTS GEvans-HMCTS disabled auto-merge January 24, 2025 10:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants