Code

Refactor DCC configuration#122

Code

Merged

afwillia merged 78 commits intodevSage-Bionetworks/data_curator_config:devfrom config-refactorSage-Bionetworks/data_curator_config:config-refactorCopy head branch name to clipboardNov 2, 2023

+557-21

Commits

Commits on Oct 26, 2023

• Move DCC config folders in dcc folder

  afwillia

  committed
  e96532e

  Copy full SHA for e96532e

• Add dcc tenant config file

  afwillia

  committed
  a55507b

  Copy full SHA for a55507b

• Move test fileview configs to test directory

  afwillia

  committed
  5f53c3d

  Copy full SHA for 5f53c3d

• Add github link to each dcc config file

  afwillia

  committed
  96574bd

  Copy full SHA for 96574bd

• Update links in dca demo config

  afwillia

  committed
  4302edf

  Copy full SHA for 4302edf

• Remove dcc_config.csv in favour of tenants.json

  afwillia

  committed
  68edd2b

  Copy full SHA for 68edd2b

• Update paths to template config files and logos for each DCC

  afwillia

  committed
  93bff11

  Copy full SHA for 93bff11

• Fix name of EL dca config file

  afwillia

  committed
  62009b2

  Copy full SHA for 62009b2

• Remove brackets from demo upsert config

  afwillia

  committed
  814bd4c

  Copy full SHA for 814bd4c

• Remove out array from json config menu

  afwillia

  committed
  f46d35d

  Copy full SHA for f46d35d

• Update links to template drop down for tests

  afwillia

  committed
  128e4d1

  Copy full SHA for 128e4d1

Commits on Oct 27, 2023

• Use relative filepaths instead of githubusercontent link to config files in demo config

  afwillia

  committed
  e8afa6c

  Copy full SHA for e8afa6c

• Use relative filepath to dca demo config in tenants.json

  afwillia

  committed
  ddf6f9a

  Copy full SHA for ddf6f9a

• Use relative filepath instead of url to reference config files.

  afwillia

  committed
  68a5435

  Copy full SHA for 68a5435

• Update configs to use relative filepath instead of githubusercontent url

  afwillia

  committed
  2944916

  Copy full SHA for 2944916

• Add schemas for tenants.json and dca_config.json

  afwillia

  committed
  3a47c1c

  Copy full SHA for 3a47c1c

Commits on Oct 30, 2023

• Change tenants array to an oject

  afwillia

  committed
  a31c9ed

  Copy full SHA for a31c9ed

• change tenants from array to object and add tenant object

  afwillia

  committed
  9aa8655

  Copy full SHA for 9aa8655

• Update dca config schema

  afwillia

  committed
  71d8732

  Copy full SHA for 71d8732

• correct manifest submit type for GF

  afwillia

  committed
  f1a9b78

  Copy full SHA for f1a9b78

• Add uri-refernce format to the URL properties of dca config schema

  afwillia

  committed
  b80ee85

  Copy full SHA for b80ee85

Commits on Oct 31, 2023

• Change tenant schema to an array of objects. Define object specifications in defs

  afwillia

  committed
  e895dd5

  Copy full SHA for e895dd5

• Update tenants.json to an object of arrays

  afwillia

  committed
  ac8a50d

  Copy full SHA for ac8a50d

• Add github workflow to validate tenants.json

  afwillia

  committed
  f836c11

  Copy full SHA for f836c11

• Fix whitespace

  afwillia

  committed
  15e118d

  Copy full SHA for 15e118d

• remove - from workflow name

  afwillia

  committed
  71c2fc9

  Copy full SHA for 71c2fc9

• remove path trigger for testing

  afwillia

  committed
  297f6ee

  Copy full SHA for 297f6ee

• use docker form of json validate

  afwillia

  committed
  c3ab00f

  Copy full SHA for c3ab00f

• switch json inputs to env vars

  afwillia

  committed
  2fc6ddb

  Copy full SHA for 2fc6ddb

• fix variable name

  afwillia

  committed
  5f53faf

  Copy full SHA for 5f53faf

• add repo to json filepath

  afwillia

  committed
  96ab333

  Copy full SHA for 96ab333

• tenants.schema not tenant.schema

  afwillia

  committed
  e528808

  Copy full SHA for e528808

• maybe we don't need the repo dir

  afwillia

  committed
  bab0acc

  Copy full SHA for bab0acc

• remove filler ID field and try using http for schema

  afwillia

  committed
  db8f49f

  Copy full SHA for db8f49f

• test removing the schema

  afwillia

  committed
  41695dd

  Copy full SHA for 41695dd

• commit an invalid tenants.json to test validation

  afwillia

  committed
  da00d16

  Copy full SHA for da00d16

• Fix tenants.json

  afwillia

  committed
  4e45521

  Copy full SHA for 4e45521

• Add tenant json paths to trigger workflow

  afwillia

  committed
  267bb30

  Copy full SHA for 267bb30

• Update workflow description

  afwillia

  committed
  a60c528

  Copy full SHA for a60c528

• test validate dcc config workflow

  afwillia

  committed
  f7ae9b4

  Copy full SHA for f7ae9b4

• fix indentation

  afwillia

  committed
  1f689b2

  Copy full SHA for 1f689b2

• Modify smoke test config for test

  afwillia

  committed
  8a3fb9f

  Copy full SHA for 8a3fb9f

• try filtering modified files

  afwillia

  committed
  98f50bc

  Copy full SHA for 98f50bc

• fix smoke test config

  afwillia

  committed
  16b1015

  Copy full SHA for 16b1015

• test smoke test change

  afwillia

  committed
  0c46ad8

  Copy full SHA for 0c46ad8

• Change file to trigger dca config validation workflow

  afwillia

  committed
  a52c757

  Copy full SHA for a52c757

• remove extra echo

  afwillia

  committed
  68bef96

  Copy full SHA for 68bef96

• test edit to config

  afwillia

  committed
  fcb1259

  Copy full SHA for fcb1259

• remove extra parenthesis from echo

  afwillia

  committed
  1bfd74d

  Copy full SHA for 1bfd74d

• test edit to smoke test json

  afwillia

  committed
  0e2402a

  Copy full SHA for 0e2402a

• print config var

  afwillia

  committed
  162bf97

  Copy full SHA for 162bf97

• change smoke test

  afwillia

  committed
  eea8bb9

  Copy full SHA for eea8bb9

• remove single quotes in grep

  afwillia

  committed
  c729f60

  Copy full SHA for c729f60

• test formatting output with xargs

  afwillia

  committed
  b19544b

  Copy full SHA for b19544b

• remove schema from dca config schema for compatibility

  afwillia

  committed
  93b145b

  Copy full SHA for 93b145b

• set output to json

  afwillia

  committed
  ac68636

  Copy full SHA for ac68636

Commits on Nov 1, 2023

• try splitting file changed list with tr

  afwillia

  committed
  002213e

  Copy full SHA for 002213e

• use space-delimited string for changed file output

  afwillia

  committed
  3062033

  Copy full SHA for 3062033

• quote echo

  afwillia

  committed
  e0d6f3f

  Copy full SHA for e0d6f3f

• echo the changed files variable

  afwillia

  committed
  a2f2521

  Copy full SHA for a2f2521

• create filter_file step id to reference varible

  afwillia

  committed
  b41183b

  Copy full SHA for b41183b

• fix smoke test config

  afwillia

  committed
  c3649f5

  Copy full SHA for c3649f5

• remove printing config variable for debugging

  afwillia

  committed
  1f6846a

  Copy full SHA for 1f6846a

• Update template path for mrt test config

  afwillia

  committed
  5c14d21

  Copy full SHA for 5c14d21

• change files changed string to csv instead of space-delimited to handle filenames with spaces

  afwillia

  committed
  e2a8a35

  Copy full SHA for e2a8a35

• pipe changed files output to tr again to remove new liens

  afwillia

  committed
  7aee822

  Copy full SHA for 7aee822

• make json files list comma-separated

  afwillia

  committed
  6a4d69d

  Copy full SHA for 6a4d69d

• print env variables to debug

  afwillia

  committed
  aa89ff4

  Copy full SHA for aa89ff4

• print env variables to debug 2

  afwillia

  committed
  d12f4f1

  Copy full SHA for d12f4f1

• print env variables to debug 3

  afwillia

  committed
  7974ad5

  Copy full SHA for 7974ad5

• use sed to remove last comma

  afwillia

  committed
  165c5d0

  Copy full SHA for 165c5d0

• Remove printing of env var for debugging

  afwillia

  committed
  670dce1

  Copy full SHA for 670dce1

• test adding template config workflow in R

  afwillia

  committed
  a5ed142

  Copy full SHA for a5ed142

Commits on Nov 2, 2023

• remove template config generation workflow. This is in the example data model repo.

  afwillia

  committed
  587f767

  Copy full SHA for 587f767

• Add json schema for template dropdown config

  afwillia

  committed
  590bb91

  Copy full SHA for 590bb91

• Add workflow to validate template config files

  afwillia

  committed
  a2b6ff1

  Copy full SHA for a2b6ff1

• filter files that trigger validation workflows on PR

  afwillia

  committed
  b75e649

  Copy full SHA for b75e649

• only run validation if template json files have changed

  afwillia

  committed
  10687ce

  Copy full SHA for 10687ce