Code

Test kubernetes in CI#3482

Code

Draft

benclifford wants to merge 100 commits intomasterParsl/parsl:masterfrom benc-k8s-kind-ciParsl/parsl:benc-k8s-kind-ciCopy head branch name to clipboard

+139

Commits

Commits on Jun 7, 2024

• Added options to set annotations and a service account in the Kubernetes worker pod configuration

  shishichen

  committed
  8e3d7cf

  Copy full SHA for 8e3d7cf

• Correct punctuation in debug message. hack out tests that won't fail-as-expected as root

  benclifford

  committed
  45269ed

  Copy full SHA for 45269ed

• Fix a couple of docstrings

  benclifford

  committed
  7ceec7a

  Copy full SHA for 7ceec7a

• a bit of name sanitization for default pod names

  benclifford

  committed
  0cdece2

  Copy full SHA for 0cdece2

Commits on Jun 8, 2024

• fiddle with markings to deal with no shared fs and no staging

  Show description for 87d3454
  benclifford

  committed
  87d3454

  Copy full SHA for 87d3454

Commits on Jun 10, 2024

• add config file i've been using

  benclifford

  committed
  954bad7

  Copy full SHA for 954bad7

• Merge remote-tracking branch 'shishichen/add-k8s-pod-options'

  benclifford

  committed
  d3e3828

  Copy full SHA for d3e3828

• add the dockerfile i've been using

  benclifford

  committed
  17a00dd

  Copy full SHA for 17a00dd

• beginning of kubernetes-in-CI

  benclifford

  committed
  62e0e36

  Copy full SHA for 62e0e36

• push docker image? upgrade ubuntu

  benclifford

  committed
  9086e19

  Copy full SHA for 9086e19

• fiddle with default name

  benclifford

  committed
  26869e6

  Copy full SHA for 26869e6

• Add kubernetes, needed for submitting from inside a cluster

  benclifford

  committed
  16c0a49

  Copy full SHA for 16c0a49

• Add more bits for running everything in a kubernetes cluster

  benclifford

  committed
  b03615a

  Copy full SHA for b03615a

• fix syntax error in github workflow definition

  benclifford

  committed
  e122d19

  Copy full SHA for e122d19

• Tighten timeout, add some debugging info at the end

  benclifford

  committed
  ee14f6e

  Copy full SHA for ee14f6e

• Correct pod name from my test

  benclifford

  committed
  120cf78

  Copy full SHA for 120cf78

• try to stop Job from recreating pod on failure, but instead abort fast

  benclifford

  committed
  5c55fe6

  Copy full SHA for 5c55fe6

• Randomise test order to see if a test failure is specific to a particular test, or something else

  benclifford

  committed
  b56dfd9

  Copy full SHA for b56dfd9

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  dd0f66c

  Copy full SHA for dd0f66c

• Add some memory logging

  benclifford

  committed
  f8f5a27

  Copy full SHA for f8f5a27

• Allocate more memory to workers

  benclifford

  committed
  f4a7300

  Copy full SHA for f4a7300

• Add a staging_required marker that apparently wasn't breaking things always in k8s

  benclifford

  committed
  ffdb021

  Copy full SHA for ffdb021

• messing with backoff limits and restart policy

  benclifford

  committed
  21711e9

  Copy full SHA for 21711e9

• remove apparently invalid restart policy

  benclifford

  committed
  fb1733e

  Copy full SHA for fb1733e

• Flush out some more staging_required tests (by setting storage_access=[] in local htex test...)

  benclifford

  committed
  73b3e1d

  Copy full SHA for 73b3e1d

Commits on Jun 12, 2024

• Switch the Kubernetes client call to read_namespaced_pod_status() to read_namespaced_pod(), which is functionally the same but requires fewer permissions.

  shishichen

  committed
  31bc958

  Copy full SHA for 31bc958

Commits on Jun 13, 2024

• Fixed Kubernetes worker container launch command to remove trailing semicolon.

  shishichen

  committed
  8b39024

  Copy full SHA for 8b39024

Commits on Jun 14, 2024

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  4538763

  Copy full SHA for 4538763

• Merge remote-tracking branch 'shishichen/fix-k8s-launch-cmd' into benc-k8s-kind-ci

  benclifford

  committed
  5f43aeb

  Copy full SHA for 5f43aeb

• Merge remote-tracking branch 'shishichen/swap-k8s-pod-status' into benc-k8s-kind-ci

  benclifford

  committed
  299de99

  Copy full SHA for 299de99

Commits on Jun 18, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  68e3a5d

  Copy full SHA for 68e3a5d

Commits on Jun 24, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  fe3c55e

  Copy full SHA for fe3c55e

Commits on Jul 2, 2024

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  064b833

  Copy full SHA for 064b833

• Merge remote-tracking branch 'origin/benc-k8s-kind-ci' into benc-k8s-kind-ci

  benclifford

  committed
  9c6a04e

  Copy full SHA for 9c6a04e

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  ba5f047

  Copy full SHA for ba5f047

Commits on Jul 7, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  69fbf03

  Copy full SHA for 69fbf03

Commits on Jul 31, 2024

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  75b7c02

  Copy full SHA for 75b7c02

• Merge remote-tracking branch 'origin/benc-k8s-kind-ci' into benc-k8s-kind-ci

  benclifford

  committed
  780fbb0

  Copy full SHA for 780fbb0

Commits on Aug 5, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  2324744

  Copy full SHA for 2324744

Commits on Aug 19, 2024

• function data in temp

  colinthomas-z80

  committed
  b75a3ae

  Copy full SHA for b75a3ae

• use getpass for username

  colinthomas-z80

  committed
  2c18d6c

  Copy full SHA for 2c18d6c

Commits on Aug 20, 2024

• use tempfile module

  colinthomas-z80

  committed
  c201ec1

  Copy full SHA for c201ec1

• flake etc

  colinthomas-z80

  committed
  9f6b037

  Copy full SHA for 9f6b037

Commits on Aug 21, 2024

• Merge branch 'master' into tmp_function_data

  benclifford

  authored
  5ec7cdb

  Copy full SHA for 5ec7cdb

Commits on Aug 22, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  c3f6d45

  Copy full SHA for c3f6d45

Commits on Aug 26, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  edf870f

  Copy full SHA for edf870f

Commits on Sep 3, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  811b8e5

  Copy full SHA for 811b8e5

Commits on Sep 4, 2024

• Merge remote-tracking branch 'refs/remotes/origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  7347f64

  Copy full SHA for 7347f64

• Merge branch 'master' into tmp_function_data

  benclifford

  authored
  cd7229f

  Copy full SHA for cd7229f

Commits on Sep 5, 2024

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  08f8ce9

  Copy full SHA for 08f8ce9

• Merge remote-tracking branch 'refs/remotes/origin/benc-k8s-kind-ci' into benc-k8s-kind-ci

  benclifford

  committed
  5967f01

  Copy full SHA for 5967f01

• Build cctools and run a probably-broken taskvine vs kubernetes test config

  benclifford

  committed
  4938dbf

  Copy full SHA for 4938dbf

• fix repr in taskvine

  benclifford

  committed
  98d7693

  Copy full SHA for 98d7693

• install cloudpickle explicitly for taskvine

  benclifford

  committed
  dfc94a8

  Copy full SHA for dfc94a8

• Add more time onto job timeout, because more is happening in job with taskvine now

  benclifford

  committed
  47378f3

  Copy full SHA for 47378f3

• revert to 180s test time

  benclifford

  committed
  43af8ef

  Copy full SHA for 43af8ef

• Log more to the console, kubernetes style

  benclifford

  committed
  6a32f0f

  Copy full SHA for 6a32f0f

• Note a (documentation?) bug in taskvine address selection

  benclifford

  committed
  d4fab6a

  Copy full SHA for d4fab6a

• force hostname based address config, in line with comment in previous commit

  benclifford

  committed
  21dcae6

  Copy full SHA for 21dcae6

• now we're starting taskvine test successfully, give it time to complete

  benclifford

  committed
  e1cce03

  Copy full SHA for e1cce03

• Make taskvine shutdown scale-in more like htex shutdown scale-in

  benclifford

  committed
  4d4b4ba

  Copy full SHA for 4d4b4ba

• enable staging_required tests in taskvine, because taskvine might be able to deal with staging those files properly (or it might be good to flush out more test problems)

  benclifford

  committed
  2e42e5c

  Copy full SHA for 2e42e5c

• Output timestamps in kubernetes log to help diagnose hangs

  benclifford

  committed
  3ba7e12

  Copy full SHA for 3ba7e12

• failed to get non-staging tests working, made a note in comments

  benclifford

  committed
  084d797

  Copy full SHA for 084d797

• correct duplicated 'and' in pytest -k option

  benclifford

  committed
  f34f2b8

  Copy full SHA for f34f2b8

Commits on Sep 6, 2024

• Merge remote-tracking branch 'colinthomas-z80/tmp_function_data' into benc-k8s-kind-ci

  benclifford

  committed
  60a8611

  Copy full SHA for 60a8611

Commits on Oct 21, 2024

• Add utils to sanitize strings for DNS compliance

  Show description for 1f09e5c
  rjmello

  committed
  1f09e5c

  Copy full SHA for 1f09e5c

• Ensure k8s pod names/labels are RFC 1123 compliant

  Show description for 83278c2
  rjmello

  committed
  83278c2

  Copy full SHA for 83278c2

• Use hex value for k8s job ID instead of pod name

  rjmello

  committed
  86ade32

  Copy full SHA for 86ade32

• Add tests for KubernetesProvider submit

  rjmello

  committed
  0c4d541

  Copy full SHA for 0c4d541

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  08693ab

  Copy full SHA for 08693ab

• Merge remote-tracking branch 'origin/rjmello-kube-pod-names' into benc-k8s-kind-ci, with conflicts

  benclifford

  committed
  415f780

  Copy full SHA for 415f780

• Fix some bad merge

  benclifford

  committed
  c78defa

  Copy full SHA for c78defa

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  54ea143

  Copy full SHA for 54ea143

Commits on Oct 31, 2024

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  535289f

  Copy full SHA for 535289f

Commits on Nov 1, 2024

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  fd26ddd

  Copy full SHA for fd26ddd

Commits on Jan 6, 2025

• Merge branch 'master' into benc-k8s-kind-ci

  benclifford

  authored
  419ba64

  Copy full SHA for 419ba64

• remove spurious whitespace add

  benclifford

  committed
  61e65fa

  Copy full SHA for 61e65fa

• Merge remote-tracking branch 'refs/remotes/origin/benc-k8s-kind-ci' into benc-k8s-kind-ci

  benclifford

  committed
  0d089db

  Copy full SHA for 0d089db

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  803ccc7

  Copy full SHA for 803ccc7

• Merge remote-tracking branch 'refs/remotes/origin/benc-k8s-kind-ci' into benc-k8s-kind-ci

  benclifford

  committed
  f418ebf

  Copy full SHA for f418ebf

• remove channels

  benclifford

  committed
  b688f3b

  Copy full SHA for b688f3b

• Remove repr fix that isn't needed

  benclifford

  committed
  bb76cac

  Copy full SHA for bb76cac

• Remove spurious paste

  benclifford

  committed
  0f0d5b9

  Copy full SHA for 0f0d5b9

Commits on Jan 7, 2025

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  benclifford

  committed
  8966056

  Copy full SHA for 8966056

• remove commented out override that has been fixed elsewhere

  benclifford

  committed
  2dc45ab

  Copy full SHA for 2dc45ab

Commits on Jan 15, 2025

• Merge remote-tracking branch 'origin/master' into benc-k8s-kind-ci

  Show description for 78b7f2d
  benclifford

  committed
  78b7f2d

  Copy full SHA for 78b7f2d

• move to bookworm (debian stable) with a single python version

  benclifford

  committed
  a9d147e

  Copy full SHA for a9d147e

• restore partial python version

  benclifford

  committed
  17d5115

  Copy full SHA for 17d5115

• fix swig install

  benclifford

  committed
  722a828

  Copy full SHA for 722a828

• restore python version in path

  benclifford

  committed
  eebdedc

  Copy full SHA for eebdedc

• move some files around

  benclifford

  committed
  4a497f6

  Copy full SHA for 4a497f6

• move taskvine config

  benclifford

  committed
  4cae154

  Copy full SHA for 4cae154

• move htex config

  benclifford

  committed
  d98c7d8

  Copy full SHA for d98c7d8

• making dockerfile build faster, hopefully no significant change in what is built

  benclifford

  committed
  8fb5f30

  Copy full SHA for 8fb5f30

• move swig install alongside other apt. fix git clone cli

  benclifford

  committed
  f74037b

  Copy full SHA for f74037b

• isort

  benclifford

  committed
  5854b9e

  Copy full SHA for 5854b9e

• add mandatory init

  benclifford

  committed
  bd076fe

  Copy full SHA for bd076fe

• fix linting

  benclifford

  committed
  230cee0

  Copy full SHA for 230cee0

• remove a note to self about taskvine vs non-shared fs

  benclifford

  committed
  88c5f17

  Copy full SHA for 88c5f17