Merge branch 'main' into hostsd-326-infinite-refresh

.vscode/settings.json

@@ -8,4 +8,7 @@
   "notebook.codeActionsOnSave": {
     "source.organizeImports": true,
   },
+  "editor.rulers": [
+    100
+  ]
 }

README.md

@@ -19,7 +19,9 @@ bash do help
 
 ## Get Started Developing
 
-Currently you'll need to install the following.
+### Prerequisites
+
+You'll need to install the following.
 
 | Dependency                                                                | Link                                                               |
 | ------------------------------------------------------------------------- | ------------------------------------------------------------------ |
@@ -35,23 +37,92 @@ Run `nvm install 20.8.1` to install the version of Node required for this projec
 Once it is installed run `nvm use 20.8.w1`.
 If it installed correctly you can run `node -v` and it will display the correct version.
 
-Execute the following command to initialize your local environment and start the required Docker containers.
+#### Mac requirements
+
+You will need to install `coreutils`, `gnu-getopt`, and `gsed` using [Homebrew](https://brew.sh):
+
+> brew install coreutils gnu-getopt gsed
 
-> If you run into errors with a Mac it is most likely a script needs execute permission.
->
-> You will also need to install coreutils and gnu-getopt using [Homebrew](https://brew.sh):
->
-> brew install coreutils gnu-getopt
+If you run into other errors during the steps below, it is most likely a script needs execute permission.
+
+### Container initialization
+
+Execute the following command to initialize your local environment.
 
 ```bash
+# Generate a local Keycloak admin username and password.
 # Generate .env files.
 # Start the database and run the migration.
 # Spin up all other required containers.
-# This process will ask you to input usernames and passwords.
+#
 bash do init
 ```
 
-The default configuration will initialize the database and run the web application.
+The script will initialize the database and various Docker containers.
+
+### Update .env files and restart environment
+
+For the app to be fully functional, we will need to update the values of some `.env` file secrets.
+
+#### Obtain the local Keycloak Client Secret
+
+1. With the application running (check in Docker), navigate to the local Keycloak admin interface: [http://localhost:30001](http://localhost:30001).
+2. Enter the username and password you created for the local Keycloak admin in the previous step.
+3. From the dropdown (select) menu in the upper-right (currently showing "Keycloak"), choose "Host Services Branch Dashboard".
+4. From the sidebar on the left, click Clients.
+5. In the table, click `hsb-app`.
+6. Click the Credentials tab.
+7. In the Client Secret section of the page, click the clipboard icon to copy the Client Secret to your clipboard. This is the **Client Secret**. Take note of it, because it will be entered in several places below. (In the examples below, we will use the pretend key `Abc123`.)
+
+#### Update API env file
+
+In `/src/api/.env`:
+
+Update the line `Keycloak__Secret={GET FROM KEYCLOAK}` with the **Client Secret**, e.g.
+
+```bash
+Keycloak__Secret=Abc123
+```
+
+#### Update API-CSS env file
+
+In `/src/api-css/.env`, update two lines with `hsb-app` and the **Client Secret** respectively:
+
+```bash
+Keycloak__ClientId=hsb-app
+Keycloak__Secret=Abc123
+```
+
+#### Update Dashboard env file
+
+In `/src/dashboard/.env`:
+
+Update the line `KEYCLOAK_SECRET={GET FROM KEYCLOAK}` with the **Client Secret**, e.g.:
+
+```bash
+KEYCLOAK_SECRET=Abc123
+```
+
+Note that you can also **uncomment** the following lines to skip Keycloak authentication altogether:
+
+```bash
+# NEXT_PUBLIC_AUTH_STATUS=authenticated
+# NEXT_PUBLIC_AUTH_ROLES=hsb
+```
+
+This should only be done for development purposes when testing authentication is not necessary.
+
+#### Restart environment
+
+The following command will rebuild the Docker containers to pick up all the `.env` file changes you've made above:
+
+```bash
+bash do up
+```
+
+### Run the web application
+
+Now we can start the web application:
 
 ```bash
 # Open the web application in your default browser
@@ -60,6 +131,16 @@ bash do go
 
 The Dashboard web application is setup for hot-reload within a Docker container.
 
+## Tips
+
+### Find all .env files
+
+When recreating the environment, .env files are left behind by default. But this can cause issues when attempting a fresh install. To find .env files:
+
+```bash
+find . -name '*.env'
+```
+
 ## Helpful Documentation
 
 - [API Swagger](https://localhost:30005/api-docs)
@@ -91,6 +172,13 @@ There are a few other helpful database migration commands that can help with dev
 | db-refresh   | Drops the database and runs all the migrations              |
 | db-redo      | Rollback and reapply the migration                          |
 
+If you get an error on a Mac with an M chip, try running:
+
+```bash
+dotnet tool uninstall dotnet-ef --global
+dotnet tool install dotnet-ef --global -a arm64
+```
+
 ### CI/CD Pipelines
 
 When a Pull Request is created Github Actions will build and test the images to provide feedback on issues.

devops/kustomize/base/data-service/config-map.yaml

@@ -15,7 +15,7 @@ metadata:
     created-by: jeremy.foster
 data:
   API_URL: http://api:8080
-  ENVIRONMENT: "DEV"
+  ENVIRONMENT: 'DEV'
 ---
 kind: ConfigMap
 apiVersion: v1
@@ -35,6 +35,6 @@ data:
   AUTH_URL: https://loginproxy.gov.bc.ca/auth/realms/comsvcauth/protocol/openid-connect/token
   HOST_URI: https://ches.api.gov.bc.ca/api/v1
   FROM: Hosting Service Dashboard <[email protected]>
-  TO: [email protected],[email protected]
-  EMAIL_ENABLED: "true"
-  EMAIL_AUTHORIZED: "true"
+  TO: [email protected],[email protected]
+  EMAIL_ENABLED: 'true'
+  EMAIL_AUTHORIZED: 'true'

devops/kustomize/overlays/dev/data-service/kustomization.yaml

@@ -40,7 +40,7 @@ patches:
         value: Hosting Service Dashboard <[email protected]>
       - op: replace
         path: /data/TO
-        value: [email protected]
+        value: [email protected]
       - op: replace
         path: /data/ENVIRONMENT
         value: DEV

devops/kustomize/overlays/dev/kustomization.yaml

@@ -52,7 +52,7 @@ patches:
         value: (DEV) Hosting Service Dashboard <[email protected]>
       - op: replace
         path: /data/TO
-        value: [email protected]
+        value: [email protected]
   - target:
       kind: ConfigMap
       name: dashboard

devops/kustomize/overlays/prod/data-service/kustomization.yaml

@@ -40,7 +40,7 @@ patches:
         value: Hosting Service Dashboard <[email protected]>
       - op: replace
         path: /data/TO
-        value: [email protected],[email protected]
+        value: [email protected],[email protected]
       - op: replace
         path: /data/ENVIRONMENT
         value: PROD

devops/kustomize/overlays/prod/kustomization.yaml

@@ -52,7 +52,7 @@ patches:
         value: Hosting Service Dashboard <[email protected]>
       - op: replace
         path: /data/TO
-        value: [email protected],[email protected]
+        value: [email protected],[email protected]
   - target:
       kind: ConfigMap
       name: dashboard

devops/kustomize/overlays/test/data-service/kustomization.yaml

@@ -40,7 +40,7 @@ patches:
         value: Hosting Service Dashboard <[email protected]>
       - op: replace
         path: /data/TO
-        value: [email protected],[email protected]
+        value: [email protected],[email protected]
       - op: replace
         path: /data/ENVIRONMENT
         value: TEST

devops/kustomize/overlays/test/kustomization.yaml

@@ -52,7 +52,7 @@ patches:
         value: (TEST) Hosting Service Dashboard <[email protected]>
       - op: replace
         path: /data/TO
-        value: [email protected],[email protected]
+        value: [email protected],[email protected]
   - target:
       kind: ConfigMap
       name: dashboard

scripts/db-migration.sh

@@ -4,26 +4,35 @@ MNAME=$1;
 FILE1=./Migrations/$(basename ./Migrations/*_$MNAME.cs);
 echo "Updating migration '$FILE1'";
 
-sed -i "2iusing HSB.DAL;" $FILE1;
+# Check for Mac OS first, and use gsed if so. The built-in getopt is not
+# the same as the GNU version, and it does not support long options.
+if [ "$(uname)" == "Darwin" ]; then
+    echo 'Note: Mac OS X detected. Using gsed. If you do not have this' \
+      'installed, run `brew install gsed`.'
+    SED_CMD=gsed
+else
+    SED_CMD=sed
+fi
+
+${SED_CMD} -i "2iusing HSB.DAL;" $FILE1;
 
 search=":\ Migration";
 replace=":\ PostgresSeedMigration";
-sed -i "s/$search/$replace/" $FILE1;
+${SED_CMD} -i "s/$search/$replace/" $FILE1;
 
 fl1=$(grep -n "protected override void Up(MigrationBuilder migrationBuilder)" $FILE1 | head -n 1 | cut -d: -f1);
 l1=$(($fl1 + 2));
-sed -i "${l1}i\ \ \ \ \ \ \ \ \ \ \ \ PreUp(migrationBuilder);" $FILE1;
+${SED_CMD} -i "${l1}i\ \ \ \ \ \ \ \ \ \ \ \ PreUp(migrationBuilder);" $FILE1;
 
 fl=$(grep -n "protected override void Down(MigrationBuilder migrationBuilder)" $FILE1 | head -n 1 | cut -d: -f1);
 l2=$(($fl - 3));
-sed -i "${l2}i\ \ \ \ \ \ \ \ \ \ \ \ PostUp(migrationBuilder);" $FILE1;
+${SED_CMD} -i "${l2}i\ \ \ \ \ \ \ \ \ \ \ \ PostUp(migrationBuilder);" $FILE1;
 
 l3=$(($fl + 3));
-sed -i "${l3}i\ \ \ \ \ \ \ \ \ \ \ \ PreDown(migrationBuilder);" $FILE1;
+${SED_CMD} -i "${l3}i\ \ \ \ \ \ \ \ \ \ \ \ PreDown(migrationBuilder);" $FILE1;
 
 eofl=$(wc -l $FILE1 | awk '{ print $1 }');
 l4=$(($eofl - 2));
-sed -i "${l4}i\ \ \ \ \ \ \ \ \ \ \ \ PostDown(migrationBuilder);" $FILE1;
+${SED_CMD} -i "${l4}i\ \ \ \ \ \ \ \ \ \ \ \ PostDown(migrationBuilder);" $FILE1;
 
 code -r $FILE1
-

scripts/help.sh

@@ -16,6 +16,8 @@ nuke:                                                                 Teardown w
 ssh:            {service}                                             SSH into docker container
 go:                                                                   Open a browser and view app
 
+data-service:                                                         Run the data service
+
 npm-install:                                                          Install latest npm packages locally.
 
 tool-update:                                                          Updates the dotnet tool (default ef)

scripts/oc.sh

@@ -292,10 +292,6 @@ oc-run () {
                     }
                   }
                 },
-                # {
-                #   \"name\":\"Service__Actions__0\",
-                #   \"value\": \"clean-organizations\"
-                # },
                 {
                   \"name\":\"Service__SendSuccessEmail\",
                   \"value\": \"true\"

scripts/setup.sh

@@ -26,22 +26,22 @@ gen_env () {
   if [[ -z "$dbPassword" ]]
   then
       # Generate a random password that satisfies password requirements.
-      echo 'A password is randomly being generated.'
+      echo 'A random database password is being generated.'
       dbPassword=$(date +%s | sha256sum | base64 | head -c 29)A8!
       echo "Your generated password is: $dbPassword"
       export dbPassword
   fi
 
   if [[ -z "$keycloakUser" ]]
   then
-      echo 'Enter a username for keycloak.'
+      echo 'Enter an admin username for your local Keycloak installation:'
       read -p 'Username: ' keycloakUser
       export keycloakUser
   fi
 
   if [[ -z "$keycloakPassword" ]]
   then
-      echo 'Enter a password for the keycloak user $keycloakUser.'
+      echo "Enter a password for the local Keycloak user \"$keycloakUser\":"
       read -p 'Password: ' keycloakPassword
       export keycloakPassword
   fi

scripts/variables.sh

@@ -2,7 +2,15 @@
 
 . ./scripts/os.sh
 
-export dockerHost=host.docker.internal
+# On Mac, use localhost instead of host.docker.internal
+
+if [ "$(uname)" == "Darwin" ]; then
+    export dockerHost=localhost
+else
+    export dockerHost=host.docker.internal
+fi
+
+
 
 #######################################################
 # Database Variables

src/api/Dockerfile

@@ -1,5 +1,5 @@
 ARG ASPNETCORE_ENVIRONMENT=Release
-FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base
+FROM mcr.microsoft.com/dotnet/aspnet:8.0-bookworm-slim-amd64 AS base
 
 RUN apt-get update && apt-get -y upgrade
 RUN apt-get -y install curl libc6-dev libgdiplus
@@ -17,7 +17,7 @@ WORKDIR /src/api
 COPY src/api/ .
 COPY src/libs/ /src/libs/
 
-RUN dotnet restore
+RUN dotnet restore "HSB.API.csproj"
 RUN dotnet publish "HSB.API.csproj" -c "$ASPNETCORE_ENVIRONMENT" -o /app/publish
 
 # Runtime image

src/api/Keycloak/CssHelper.cs

@@ -155,6 +155,8 @@ private Task AddOrUpdateUserAsync(string username, Entities.User? user, HSB.CSS.
             }
 
             // Fetch the roles for the user
+            _logger.LogDebug("User activation: Fetching roles for user: {key}", key);
+
             var userRoles = await _cssService.GetRolesForUserAsync(key.ToString());
             if (userRoles.Users.Length > 1) throw new NotAuthorizedException($"Keycloak has multiple users with the same username '{key}'");
             if (user == null)

src/dashboard/Dockerfile.prod

@@ -1,4 +1,4 @@
-FROM node:20-bullseye as BUILD_IMAGE
+FROM --platform=linux/amd64 node:20-bullseye as BUILD_IMAGE
 
 WORKDIR /usr/dashboard
 COPY package*.json ./
@@ -11,7 +11,7 @@ COPY . .
 RUN npm run build
 RUN npm prune --production
 
-FROM node:20-slim
+FROM --platform=linux/amd64 node:20-slim
 
 RUN apt-get update && apt-get install curl nano -y
 

src/dashboard/src/app/login/page.tsx

@@ -29,10 +29,10 @@ export default function Page() {
         <div>
           <h3>Need access to the Storage Dashboard?</h3>
           <p>
-            Please email <a href="[email protected]">[email protected]</a> to request
-            access to your organization&apos;s dashboard.
+            Please email <a href="[email protected]">[email protected]</a> to
+            request access to your organization&apos;s dashboard.
           </p>
-          <h3>If you are a first time user please note:</h3>
+          <h3>If you are a first time user, please note:</h3>
           <ul>
             <li>
               Your first login will include a registration step within the BCGov Single Sign-On

src/data-service/Dockerfile

@@ -1,5 +1,5 @@
 ARG ASPNETCORE_ENVIRONMENT=Release
-FROM mcr.microsoft.com/dotnet/aspnet:8.0 AS base
+FROM mcr.microsoft.com/dotnet/aspnet:8.0-bookworm-slim-amd64 AS base
 
 RUN apt-get update && apt-get -y upgrade
 RUN apt -y install curl libc6-dev libgdiplus
@@ -16,7 +16,7 @@ WORKDIR /src/data-service
 COPY src/data-service/ .
 COPY src/libs/ /src/libs/
 
-RUN dotnet restore
+RUN dotnet restore "HSB.DataService.csproj"
 RUN dotnet publish "HSB.DataService.csproj" -c "$ASPNETCORE_ENVIRONMENT" -o /app/publish
 
 # Runtime image