Merge branch 'release-0.62.0' into 000-update-templates

.github/workflows/pull-request-test.yml

@@ -58,6 +58,7 @@ jobs:
       renku-ui: ${{ steps.deploy-comment.outputs.renku-ui}}
       renku-data-services: ${{ steps.deploy-comment.outputs.renku-data-services}}
       amalthea: ${{ steps.deploy-comment.outputs.amalthea}}
+      amalthea-sessions: ${{ steps.deploy-comment.outputs.amalthea-sessions}}
       test-enabled: ${{ steps.deploy-comment.outputs.test-enabled}}
       extra-values: ${{ steps.deploy-comment.outputs.extra-values}}
     steps:
@@ -99,6 +100,7 @@ jobs:
           renku_ui: "${{ needs.check-deploy.outputs.renku-ui }}"
           renku_data_services: "${{ needs.check-deploy.outputs.renku-data-services }}"
           amalthea: "${{ needs.check-deploy.outputs.amalthea }}"
+          amalthea_sessions: "${{ needs.check-deploy.outputs.amalthea-sessions }}"
           extra_values: "${{ needs.check-deploy.outputs.extra-values }}"
       - name: Check existing renkubot comment
         if: needs.check-deploy.outputs.pr-contains-string == 'true'

CHANGELOG.rst

@@ -1,13 +1,304 @@
 .. _changelog:
 
+0.62.0
+------
+
+This release removes the Gitlab omnibus Helm chart that we created and used to have as a dependency
+of the Renku Helm chart. We have been discouraging anyone from using
+this chart in production and we specified this in our documentation as well.
+
+If you are using the internal Gitlab Helm chart then ensure to migrate to a separate
+Gitlab deployment as specified in our `documentation <https://renku.readthedocs.io/en/stable/how-to-guides/admin/gitlab.html#migrate-from-renku-bundled-omnibus-gitlab-to-cloud-native-gitlab-helm-chart>`_.
+before installing this or any subsequent Renku version. Gitlab publishes an official Helm chart and
+that is what should be used for deploying Gitlab with Helm.
+
+Internal Changes
+~~~~~~~~~~~~~~~~
+
+**Improvements**
+
+- **Infrastructure Components**: ``redis`` has been upgraded from version ``7.0.7`` to ``7.4.1``
+- **Helm chart**: remove the custom-made Gitlab Omnibus Helm chart from Renku dependencies
+- **Search services**: Add support for sentry
+
+
+**Bug Fixes**
+
+- **Search services**: Don't return results without linked namespaces
+
+
+Individual Components
+~~~~~~~~~~~~~~~~~~~~~
+
+- `renku-search 0.7.0 <https://github.com/SwissDataScienceCenter/renku-search/releases/tag/v0.7.0>`_
+
+
+0.61.1
+------
+
+Renku 0.61.1 introduces a few bug fixes for the previous release.
+
+
+User-Facing Changes
+~~~~~~~~~~~~~~~~~~~
+
+**🐞 Bug Fixes**
+
+- **Sessions**: Correctly launch sessions that request dedicated resource classes
+
+Internal Changes
+~~~~~~~~~~~~~~~~
+
+**Bug Fixes**
+
+- **Amalthea**: Add resource requests for the authentication proxy containers
+- **Amalthea**: Add support for setting priority classes for sessions
+- **Data services**: Use the working directory to mount cloud storage if the mount path is relative
+- **Data services**: Use HTTPS in the redirect URL for the authentication proxy
+- **Data services**: Use GPU resource limits when GPUs are requested
+- **Helm chart**: Do not set the default storage class to empty string if it is not set in the values file
+- **Helm chart**: Restart the data services and notebooks pods when the mounted secret changes
+
+Individual components
+~~~~~~~~~~~~~~~~~~~~~~
+
+- `amalthea 0.14.5 <https://github.com/SwissDataScienceCenter/amalthea/releases/tag/0.14.5>`_
+- `amalthea 0.14.6 <https://github.com/SwissDataScienceCenter/amalthea/releases/tag/0.14.6>`_
+- `renku-data-services 0.27.1 <https://github.com/SwissDataScienceCenter/renku-data-services/releases/tag/v0.27.1>`_
+
+
+0.61.0
+------
+
+Renku 0.61.0 introduces a new version of Amalthea that supports running sessions with Docker images
+that do not contain Jupyter server.
+
+NOTES to administrators:
+
+- This upgrade introduces a brand new CRD for sessions. All services that support
+  sessions for Renku v2 will switch to this new CRD. Renku v1 sessions remain unchanged.
+  Therefore any old sessions for Renku v2 will not be visible to users after this upgrade. The sessions
+  themselves will not be immediately deleted and as long as users have saved links to their old sessions they
+  should be able to access their sessions and save data. However we recommend that administrators
+  notify users of the change and allow for enough time so that existing Renku v2 sessions can be saved and
+  cleaned up, rather than asking users to save the url to their sessions. In addition to users not being able
+  to see old Renku v2 sessions, they will also not be able to pause, resume or delete old Renku v2 sessions.
+  Therefore it's best if most sessions are properly saved and cleaned up before this update is rolled out. In order
+  to support the new CRD we have also created a new operator that will manage the new `amaltheasession` resources.
+
+- The network policies for Renku have been consolidated and revamped. The most notable change here is the
+  removal of the egress policy that prevented egress to internal IP addresses from sessions. Now we disallow
+  all ingress in the Renku release namespace by default and explicitly grant permissions to any pods that need
+  to access other pods inside the Renku release namespace. Two properties relevant to this have been added to the
+  Helm chart values file that allows administrators to grant access to all Renku services from a specific namespace
+  or to do the same for specific pods within the Renku namespace. These are not needed for Renku to function and the
+  default network policies should be sufficient, they have been added so that administrators can allow ingress for
+  other services that may not come with the Renku Helm chart such as logging or monitoring. This change will result in
+  the removal of some network policies and the creation of several new policies.
+
+User-Facing Changes
+~~~~~~~~~~~~~~~~~~~
+
+**✨ Improvements**
+
+- **UI**: Enable the use of custom images that don’t contain Jupyter, streamlining the image-building process and allowing for the use of “off-the-shelf” images (`#3341 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3341>`__).
+- **Sessions**: Enable running session images that do not contain Jupyter in them.
+
+Internal Changes
+~~~~~~~~~~~~~~~~
+
+**Improvements**
+
+- **Data services**: Add support for OAuth storage providers
+- **Data services**: Move notebooks code to data services
+- **Helm chart**: Consolidate and revamp network policies
+- **Data services**: Add support for project documentation
+- **Data services**: Add support for cloning projects
+
+**Bug Fixes**
+
+- **Gateway**: Pass on session cookie to data services for anonymous session authentication
+- **Data services**: Correct pagination for namespaces
+- **Data services**: Add creation date and created_by for namespaces
+- **Data services**: Pin RClone version in data services image
+- **Data services**: Properly handle multi-architecture docker images when getting working directory
+- **Data services**: Make environment working directory and mount directory optional
+- **Amalthea**: Add readiness and health checks to sessions.
+- **Amalthea**: Do not authenticate the authentication proxy health check
+- **Amalthea**: Do not mount the Kubernetes service account in sessions
+- **Amalthea**: Do not add Kubernetes specific environment variables in sessions
+
+Individual Components
+~~~~~~~~~~~~~~~~~~~~~
+
+- `renku-gateway 1.3.1 <https://github.com/SwissDataScienceCenter/renku-gateway/releases/tag/1.3.1>`_
+- `renku-ui 3.42.0 <https://github.com/SwissDataScienceCenter/renku-ui/releases/tag/3.42.0>`_
+- `renku-data-services 0.26.0 <https://github.com/SwissDataScienceCenter/renku-data-services/releases/tag/v0.26.0>`_
+- `renku-data-services 0.27.0 <https://github.com/SwissDataScienceCenter/renku-data-services/releases/tag/v0.27.0>`_
+- `amalthea 0.13.0 <https://github.com/SwissDataScienceCenter/amalthea/releases/tag/0.13.0>`_
+- `amalthea 0.14.0 <https://github.com/SwissDataScienceCenter/amalthea/releases/tag/0.14.0>`_
+- `amalthea 0.14.1 <https://github.com/SwissDataScienceCenter/amalthea/releases/tag/0.14.1>`_
+- `amalthea 0.14.2 <https://github.com/SwissDataScienceCenter/amalthea/releases/tag/0.14.2>`_
+- `amalthea 0.14.3 <https://github.com/SwissDataScienceCenter/amalthea/releases/tag/0.14.3>`_
+- `amalthea 0.14.4 <https://github.com/SwissDataScienceCenter/amalthea/releases/tag/0.14.4>`_
+
 0.60.0
 ------
 
+Renku 0.60.0 squashes an issue that has been tripping up many users when connecting their GitHub
+account, as well as various UX polish and bug fixes.
+
+NOTE to administrators: Upgrading the `csi-rclone` component will unmount all cloud storage for all
+active or hibernated sessions. Therefore, we recommend notifying your users ahead of time when you
+deploy this version of Renku and also if possible deploying the upgrade when there are fewer
+sessions that use cloud storage or just fewer sessions in general. Once the upgrade is complete
+users will be able to mount cloud storage as usual.
+
+User-Facing Changes
+~~~~~~~~~~~~~~~~~~~
+
+**✨ Improvements**
+
+- **UI**: Allow for changing the role of members of groups, and hide membership edit buttons for
+  users with insufficient permissions on project and group settings pages (`#3374
+  <https://github.com/SwissDataScienceCenter/renku-ui/pull/3374>`__).
+- **UI**: Improve UX for data connector side sheets and modals (`#3368 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3368>`__).
+
+**🐞 Bug Fixes**
+
+- **UI & Data services**: Direct users to complete the GitHub integration by installing the Renku app in the desired namespace (`#3332 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3332>`__).
+- **UI**: Avoid unexpected redirects when clicking on the 2.0 dashboard session buttons (`#3378 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3378>`__).
+
+Internal Changes
+~~~~~~~~~~~~~~~~
+
+**Improvements**
+
+- **UI**: Update the Admin page to set up Connected services (`#3332 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3332>`__).
+- **Gateway**: Add support for enabling debug logs from the gateway (`#730 <https://github.com/SwissDataScienceCenter/renku-gateway/pull/730>`__).
+
+**Bug Fixes**
+
+- **csi-rclone**: Do not log potentially sensitive data in error messages.
+- **csi-rclone**: Properly handle encrypted secrets with the new annotation-based storage class.
+
+Individual Components
+~~~~~~~~~~~~~~~~~~~~~
+
+- `renku-data-services 0.25.0 <https://github.com/SwissDataScienceCenter/renku-data-services/releases/tag/v0.25.0>`_
+- `renku-gateway 1.3.0 <https://github.com/SwissDataScienceCenter/renku-gateway/releases/tag/1.3.0>`_
+- `renku-ui 3.41.0 <https://github.com/SwissDataScienceCenter/renku-ui/releases/tag/3.41.0>`_
+- `csi-rclone 0.3.4 <https://github.com/SwissDataScienceCenter/csi-rclone/releases/tag/v0.3.4>`__
+- `csi-rclone 0.3.5 <https://github.com/SwissDataScienceCenter/csi-rclone/releases/tag/v0.3.5>`__
+
+0.59.2
+------
+
+Renku ``0.59.2`` is a bugfix release that fixes a bug in Renku 2.0 where project editors could not edit project information.
+
+User-Facing Changes
+~~~~~~~~~~~~~~~~~~~
+
+**🐞 Bug Fixes**
+
+- **Data services**: Allow project editors to send patches with the current namespace (`#483 <https://github.com/SwissDataScienceCenter/renku-data-services/pull/483>`__).
+- **Data services**: Allow project editors to send patches with the current visibility (`#484 <https://github.com/SwissDataScienceCenter/renku-data-services/pull/484>`__).
+
+Internal Changes
+~~~~~~~~~~~~~~~~
+
+**Improvements**
+
+- **Data services**: Return 409 error when creating a project with a conflicting slug (`#471 <https://github.com/SwissDataScienceCenter/renku-data-services/pull/471>`__).
+- **Data services**: Change all serial id columns to be GENERATED AS IDENTITY (`#461 <https://github.com/SwissDataScienceCenter/renku-data-services/pull/461>`__).
+- **Data services**: Include ``is_admin`` in the self ``/user`` endpoint (`#472 <https://github.com/SwissDataScienceCenter/renku-data-services/pull/472>`__).
+
+**Bug Fixes**
+
+- **Data services**: Handle spaces in ``provider_id`` for connected services (`#482 <https://github.com/SwissDataScienceCenter/renku-data-services/pull/482>`__).
+
+Individual Components
+~~~~~~~~~~~~~~~~~~~~~
+
+- `renku-data-services 0.24.2 <https://github.com/SwissDataScienceCenter/renku-data-services/releases/tag/v0.24.2>`__
+
+0.59.1
+------
+
+Renku ``0.59.1`` is a bugfix release that improves stability and performance with renku notebooks when under heavy load.
+
+Internal Changes
+~~~~~~~~~~~~~~~~
+
+**Improvements**
+
+- **Notebooks**: Use gevent methods in notebooks api (`#1996 <https://github.com/SwissDataScienceCenter/renku-notebooks/pull/1996>`__).
+
+Individual Components
+~~~~~~~~~~~~~~~~~~~~~
+
+- `renku-notebooks 1.27.1 <https://github.com/SwissDataScienceCenter/renku-notebooks/releases/tag/1.27.1>`_
 
 
 0.59.0
 ------
 
+Renku ``0.59.0`` introduces the ability to reuse data connectors in multiple projects!
+When you add a data connector (previously called a data source) to your project, you now have the new option to select other data connectors on RenkuLab,
+for example those shared in your group, rather than having to re-enter the data connection details.
+
+User-Facing Changes
+~~~~~~~~~~~~~~~~~~~
+
+**🌟 New Features**
+
+- **UI**: Add and re-use data connectors in Renku 2.0 projects (`#3323 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3323>`__).
+
+**✨ Improvements**
+
+- **UI**: Add a playful design for the 404 and application error pages (`#3248 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3248>`__).
+- **UI**: Update redirect page styles (`#3257 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3257>`__).
+- **UI**: Remove Renku 2.0 beta warning alert from dashboard (`#3357 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3357>`__).
+
+**🐞 Bug Fixes**
+
+- **UI**: Fix how permissions are checked in Renku 2.0, notably group members can perform actions according to their role in projects. (`#3351 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3351>`__).
+- **UI**: Fix styles for the edit launcher environment list (`#3360 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3360>`__).
+- **UI**: Allow opening a project from Renku 2.0 search if the namespace is missing in the result (`#3353 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3353>`__).
+- **UI**: Fix update file and download buttons in Renku 1.0 (`#3363 <https://github.com/SwissDataScienceCenter/renku-ui/pull/3363>`__).
+
+Internal Changes
+~~~~~~~~~~~~~~~~
+
+**New Features**
+
+- **Data services**: Add support for data connectors (`#407 <https://github.com/SwissDataScienceCenter/renku-data-services/pull/407>`__).
+- **Data services**: Do not synchronize blocked users from Keycloak (`#393 <https://github.com/SwissDataScienceCenter/renku-data-services/pull/393>`__).
+- **Data services**: Support getting permissions (`#454 <https://github.com/SwissDataScienceCenter/renku-data-services/pull/454>`__).
+- **Notebooks**: Add support for data connectors (`#1991 <https://github.com/SwissDataScienceCenter/renku-notebooks/pull/1991>`__).
+- **Notebooks**: Cache Amalthea sessions (`#1983 <https://github.com/SwissDataScienceCenter/renku-notebooks/pull/1983>`__).
+
+**Improvements**
+
+- **Data services**: Handle errors in background jobs nicely (`#463 <https://github.com/SwissDataScienceCenter/renku-data-services/pull/463>`__).
+- **Gateway**: Add support for signing cookies (`#734 <https://github.com/SwissDataScienceCenter/renku-gateway/pull/734>`__).
+
+**Bug Fixes**
+
+- **Data services**: Treat invalid JWT as 401 HTTP error.
+- **Data services**: Change user preferences id sequence to proper value.
+
+Individual Components
+~~~~~~~~~~~~~~~~~~~~~
+
+- `renku-data-services 0.23.0 <https://github.com/SwissDataScienceCenter/renku-data-services/releases/tag/v0.23.0>`__
+- `renku-data-services 0.24.0 <https://github.com/SwissDataScienceCenter/renku-data-services/releases/tag/v0.24.0>`__
+- `renku-gateway 1.2.0 <https://github.com/SwissDataScienceCenter/renku-gateway/releases/tag/1.2.0>`_
+- `renku-notebooks 1.27.0 <https://github.com/SwissDataScienceCenter/renku-notebooks/releases/tag/1.27.0>`_
+- `renku-ui 3.38.0 <https://github.com/SwissDataScienceCenter/renku-ui/releases/tag/3.38.0>`_
+- `renku-ui 3.39.0 <https://github.com/SwissDataScienceCenter/renku-ui/releases/tag/3.39.0>`_
+- `renku-ui 3.40.0 <https://github.com/SwissDataScienceCenter/renku-ui/releases/tag/3.40.0>`_
+- `renku-ui 3.40.1 <https://github.com/SwissDataScienceCenter/renku-ui/releases/tag/3.40.1>`_
 
 
 0.58.1

cypress-tests/cypress/e2e/dashboardV2.cy.ts

@@ -72,9 +72,7 @@ describe("Dashboard v2 - Authenticated user", () => {
 describe("Dashboard v2 - Non-Authenticated user", () => {
   it("Cannot see projects and groups on Dashboard when logged out", () => {
     cy.visit("v2");
-    cy.getDataCy("projects-container").contains("No 2.0 projects.");
-    cy.getDataCy("view-other-projects-btn").should("be.visible");
-    cy.getDataCy("groups-container").contains("No 2.0 groups.");
-    cy.getDataCy("view-other-groups-btn").should("be.visible");
+    cy.getDataCy("user-container").should("be.visible");
+    cy.getDataCy("user-container").should("contain.text", "You are not logged in.");
   });
 });

cypress-tests/cypress/e2e/privateProject.cy.ts

@@ -38,7 +38,11 @@ describe("Basic public project functionality", () => {
       },
       validateLogin
     );
-    cy.createProjectIfMissing({templateName: "Python", ...projectIdentifier, visibility: "private"});
+    cy.createProjectIfMissing({
+      templateName: "Python",
+      ...projectIdentifier,
+      visibility: "private",
+    });
     cy.visitAndLoadProject(projectIdentifier);
   });
 
@@ -50,11 +54,11 @@ describe("Basic public project functionality", () => {
       .should("be.checked");
     cy.searchForProject(projectIdentifier, true);
 
-    // logout and search for the project and log back in
-    cy.logout();
+    // Check as an anonymous user
+    cy.session(["anonymous", getRandomString()], () => {});
+    cy.visit("/");
     cy.get("#nav-hamburger").should("be.visible").click();
     cy.searchForProject(projectIdentifier, false);
-    cy.robustLogin();
   });
 
   it("Can always search for project after changing the visibility", () => {
@@ -86,10 +90,11 @@ describe("Basic public project functionality", () => {
 
     // Search the project as both logged in and logged out.
     cy.searchForProject(projectIdentifier, true);
-    cy.logout();
+    // Check as an anonymous user
+    cy.session(["anonymous", getRandomString()], () => {});
+    cy.visit("/");
     cy.get("#nav-hamburger").should("be.visible").click();
     cy.searchForProject(projectIdentifier, false);
-    cy.robustLogin();
   });
 
   it("Deleting the project removes it from the search page", () => {

cypress-tests/cypress/e2e/publicProject.cy.ts

@@ -48,11 +48,11 @@ describe("Basic public project functionality", () => {
     cy.waitMetadataIndexing();
     cy.searchForProject(projectIdentifier);
 
-    // logout and search for the project and log back in
-    cy.logout();
+    // Check as an anonymous user
+    cy.session(["anonymous", getRandomString()], () => {});
+    cy.visit("/");
     cy.get("#nav-hamburger").should("be.visible").click();
     cy.searchForProject(projectIdentifier);
-    cy.robustLogin();
   });
 
   it("Can see overview content and check the clone URLs", () => {