Move config into data-models repo

[adamjtaylor]

We will move the config file looked at by DCA from the data_curator_config repo to the data-models repo.

@elv-sb and I discussed and agree that we will keep here under manual control and not implement its generation in CI.

[adamjtaylor]

Slack discussion here.

https://sagebionetworks.slack.com/archives/C01ANC02U59/p1698873125444109

Need confirmation that DCA will look at a URL in the template_menu_config_file field of https://github.com/Sage-Bionetworks/data_curator_config/blob/main/dcc_config.csv

[aclayton555]

From Anthony on 2023.12.01 - Not urgent - only the staging DCA has been migrated to the new config setup, so we can check this out. Prioritize this (and providing any feedback to FAIR) through December.

FAIR expecting to migrate configs and update prod in January, so any feedback we provide through December, can be taken into consideration in this process.

[aclayton555]

Actions:

• move config to our repo and updating link to new config file in data curator config staging branch
• perform testing (this sprint)
• decide how this config is updated. Could leave this as static, with no CI to generate config. If adding a new component, have author update manually.
• Longer term/TBD: Have separate issue for automating config generation via CI after JSON generation? Or, at least run a check to compare DIFF between JSON and config without pushing any changes

[adamjtaylor]

Reopening while waiting on this PR to be merged: Sage-Bionetworks/data_curator_config#133

[adamjtaylor]

This is now deployed on the DCA staging instance. It seems to be working (I can select the expected templates in the drop downs)

[adamjtaylor]

decide how this config is updated. Could leave this as static, with no CI to generate config. If adding a new component, have author update manually.

I think we agreed to leave static for now

[aclayton555]

Discuss during 2024.01.04 mid-sprint - what, if any, action remaining here (and file tickets accordingly). From Anthony on 2024.01.02:

Hi everybody,
This Thursday, January 4th, DCA prod will switch to the new json config files. The main branch of data_curator_config will be changed to prod for consistency with DCA. No immediate action is required, though anyone contributing to their config files will need to use the prod branch. Let me know if you have any questions.

[adamjtaylor]

No change needed. The HTAN prod config is pointing towards the same data model version as the main branch

prod branch:
https://github.com/Sage-Bionetworks/data_curator_config/blob/4cb12b0367885fd79649d872430d684ad1360494/HTAN/dca_config.json#L5

main branch:
https://github.com/Sage-Bionetworks/data_curator_config/blob/ddd0ae98ce9cc38cdb1f24b5ade0ce60c610cabd/dcc_config.csv#L7

[aclayton555]

Confirm that there are no breaking changes and everything looks okay when we make our next release

[adamjtaylor]

Waiting on Sage-Bionetworks/data_curator_config#139 for our latest version to be deployed with the new prod branch in the data curator config repo

[aclayton555]

Remaining actions:

• is to update config to point to latest release. Filed here: Update HTAN config URL to release version Sage-Bionetworks/data_curator_config#145
• also deprecate this file: https://github.com/Sage-Bionetworks/data_curator_config/blob/prod/HTAN/dca-template-config.json

[adamjtaylor]

Waiting on Sage-Bionetworks/data_curator_config#147 to be merged to close this

[adamjtaylor]

Merged!