Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support business prepop survey launch #11

Merged
merged 25 commits into from
May 8, 2024

Conversation

liamtoozer
Copy link
Contributor

@liamtoozer liamtoozer commented Mar 27, 2024

What is the context of this PR?

We want to be able to launch business schemas that use supplementary data. This service has been altered to serve SDS metadata and unit data back to the caller.

A similar pattern to cache the mock unit data has been used, as in our mock CIR API. Happy to change this pattern to something else, as there isn't much SDS mock data at the moment. We could simply preload the data at startup instead, but see what you think might be best for now.

For this PR, we now do the following:

  • Load the mock data from the SDS schema examples at startup
  • Dynamically create metadata and deterministically generate a dataset_id for each metadata chunk (similar to the mock CIR repo for its data)
  • Load unit data for each survey ID with SDS schema examples. This maps internally to the dataset IDs Note: Survey IDs are hardcoded so we can map them to the folder names that contain the unit data.
  • Identifiers and period IDs are still unused at this point.

How to review

  1. Run make load-mock-unit-data to pull down the example unit data from the SDS schema repo
  2. Run the API
  3. Make some requests to the API endpoints, e.g.: /v1/dataset_metadata?survey_id=014&period_id=1 and /v1/unit_data?dataset_id=d8afa921-1305-d553-d2c6-955a6db2cc2d&identifier=1. Ensure that the data returned is as expected
  4. Checkout the Launcher PR branch and try launching various business surveys that make use of supplementary data. Ensure that the test SDS schemas still load correctly as well.

Note: As mentioned in the comments, Prodcom and BRES/BRS schemas cannot currently be tested

Jira ticket

app/main.py Show resolved Hide resolved
app/main.py Show resolved Hide resolved
app/main.py Show resolved Hide resolved
app/main.py Show resolved Hide resolved
app/main.py Show resolved Hide resolved
Dockerfile Show resolved Hide resolved
scripts/pr_convenience.sh Outdated Show resolved Hide resolved
@liamtoozer liamtoozer merged commit bf49ba1 into main May 8, 2024
2 checks passed
@liamtoozer liamtoozer deleted the support-real-prepop-survey-launch branch May 8, 2024 09:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants