Skip to content

Commit

Permalink
Release notes for 1.6 (#2850)
Browse files Browse the repository at this point in the history
* Release notes for 1.6

Co-authored-by: Simon Dumas <[email protected]>
  • Loading branch information
imsdu and Simon Dumas authored Oct 12, 2021
1 parent 6b94d9b commit ca80068
Show file tree
Hide file tree
Showing 8 changed files with 167 additions and 93 deletions.
1 change: 1 addition & 0 deletions docs/ignore-paths.txt
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
https://bluebrain.github.io/nexus/contexts/metadata.json
https://book.validatingrdf.com/
https://dl.acm.org/.*
http://example.com/.*
https://github.com/BlueBrain/nexus-web/blob/main/README.md.*
Expand Down
4 changes: 4 additions & 0 deletions docs/src/main/paradox/docs/delta/api/storages-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -52,6 +52,10 @@ local file-system structure and that Nexus has read and write access to the targ

### Remote disk storage

@@@ warning
The Remote disk storage and it remote service implementation are now deprecated and will be removed in an upcoming release.
@@@

This storage type relies on a remote HTTP service that exposes basic file operations on an underlying POSIX file-system.
This is particularly handy if your organization is running some kind of distributed network storage (such as Ceph,
Gluster, GPFS, Lustre, ...) that you don't want to mount directly on the system where Nexus Delta runs.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,10 @@ There are many other configuration parameters to customize the behaviour of the

Before running Nexus Delta, the @link:[expected tables](https://github.com/BlueBrain/nexus/blob/v1.5.0/delta/sourcing/src/main/resources/scripts/postgres.ddl){ open=new } should be created. However, one can let Nexus Delta automatically them using the following configuration parameters: `app.database.postgres.tables-autocreate=true`

# RDF parser

The underlying @link:[Apache Jena](https://jena.apache.org/) parser used to validate incoming data is @link:[now configurable](https://github.com/BlueBrain/nexus/blob/master/delta/app/src/main/resources/app.conf#L84) to enable different levels of strictness.

## Service account configuration

Nexus Delta uses a service account to perform automatic tasks under the hood. Examples of it are:
Expand Down Expand Up @@ -86,6 +90,8 @@ The most important flag are:
* `plugins.elasticsearch.base` which defines the endpoint where the Elasticsearch service is running.
* `plugins.elasticsearch.credentials.username` and `plugins.elasticsearch.credentials.password` to allow to access to a secured Elasticsearch cluster. The user provided should have the privileges to create/delete indices and read/index from them.

Please refer to the @link[Elasticsearch configuration](https://www.elastic.co/guide/en/elasticsearch/reference/current/secure-cluster.html) which describes the different steps to achieve this.

### Blazegraph views plugin configuration

The blazegraph plugin configuration can be found @link:[here](https://github.com/BlueBrain/nexus/blob/v1.5.0/delta/plugins/blazegraph/src/main/resources/blazegraph.conf){ open=new }.
Expand Down
122 changes: 29 additions & 93 deletions docs/src/main/paradox/docs/releases/index.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
@@@ index

- @ref:[v1.6 Release Notes](v1.6-release-notes.md)
- @ref:[v1.5 To v1.6 Migration](v1.5-to-v1.6-migration.md)
- @ref:[v1.5 Release Notes](v1.5-release-notes.md)
- @ref:[v1.4 To v1.5 Migration](v1.4-to-v1.5-migration.md)
- @ref:[v1.4 Release Notes](v1.4-release-notes.md)
Expand All @@ -17,99 +19,33 @@

This section of the documentation lists the significant BlueBrain Nexus releases across all services and web applications.

The latest stable release is **v1.5.1** released on **04.06.2021**.

## v1.6.0-M5 (05.10.2021)

- Add basic auth to Elasticsearch client [#2794](https://github.com/BlueBrain/nexus/issues/2794)
- Introduced graph-analytics plugin [#2800](https://github.com/BlueBrain/nexus/issues/2800), [#2814](https://github.com/BlueBrain/nexus/issues/2814), [#2812](https://github.com/BlueBrain/nexus/issues/2812), [#2827](https://github.com/BlueBrain/nexus/issues/2827), [#2837](https://github.com/BlueBrain/nexus/issues/2837)
- Introduced automatic project-deletion plugin [#2778](https://github.com/BlueBrain/nexus/issues/2778), [#2826](https://github.com/BlueBrain/nexus/issues/2826)
- Documentation for /v1/events endpoint [#2813](https://github.com/BlueBrain/nexus/issues/2813)
- Display values for which the count is null as greyed out (not selectable) in the column filter [#2642](https://github.com/BlueBrain/nexus/issues/2642)
- Search bar behaviour improvements [#2731](https://github.com/BlueBrain/nexus/issues/2731)
- Pagination improvements [#2732](https://github.com/BlueBrain/nexus/issues/2732)
- Table rows can be selected [#2742](https://github.com/BlueBrain/nexus/issues/2742)
- Filter improvements [#2783](https://github.com/BlueBrain/nexus/issues/2783)
- BUGFIX: Youtube video plugin crashes if payload is missing some properties [#2765](https://github.com/BlueBrain/nexus/issues/2765)
- BUGFIX: The Search column dropdown should always render in the viewport [#2782](https://github.com/BlueBrain/nexus/issues/2782)
- BUGFIX: Project deletion fixes [#2821](https://github.com/BlueBrain/nexus/issues/2821), [#2832](https://github.com/BlueBrain/nexus/issues/2832), [#2833](https://github.com/BlueBrain/nexus/issues/2833)

## v1.6.0-M4 (16.09.2021)

- Record and present deployment usage stats [#2528](https://github.com/BlueBrain/nexus/issues/2528)
- Ability to clear all search results customisations [#2616](https://github.com/BlueBrain/nexus/issues/2616)
- Allow re-ordering of columns for presentation purposes [#2643](https://github.com/BlueBrain/nexus/issues/2643)
- Allow listings on resources outside the project scope [#2737](https://github.com/BlueBrain/nexus/issues/2737)
- Add information about supported operations on the config field [#2740](https://github.com/BlueBrain/nexus/issues/2740)
- Allow streams other than those related to views to leverage Kamon [#2771](https://github.com/BlueBrain/nexus/issues/2771)
- Allow rdf parser to be configurable [#2792](https://github.com/BlueBrain/nexus/pull/2792)
- BUGFIX: Invalid IRIs should be properly rejected [#2760](https://github.com/BlueBrain/nexus/issues/2760)

## v1.6.0-M3 (02.09.2021)

This is a milestone release for v1.6.0, containing a series of bugfixes and improvements:

- Restrict volume of data through quotas (static configuration) [#2525](https://github.com/BlueBrain/nexus/issues/2525)
- Delete projects implementation [#2720](https://github.com/BlueBrain/nexus/pull/2720)
- Delete Projects: Check if project is being used anywhere before allowing deletion [#2723](https://github.com/BlueBrain/nexus/issues/2723)
- Added endpoint to list projects deletions [#2744](https://github.com/BlueBrain/nexus/pull/2744)
- Document projects deletions [#2746](https://github.com/BlueBrain/nexus/issues/2746)
- /v1/events endpoint is missing [#2397](https://github.com/BlueBrain/nexus/issues/2397)
- Ability to customise visible/hidden fields in search results [#2613](https://github.com/BlueBrain/nexus/issues/2613)
- Ability to filter search results based on known values [#2614](https://github.com/BlueBrain/nexus/issues/2614)
- Ability to sort search results on multiple columns [#2615](https://github.com/BlueBrain/nexus/issues/2615)
- Show only Home and Studios in the sidebar for anonymous users [#2637](https://github.com/BlueBrain/nexus/issues/2637)
- BUGFIX: Login is broken in Safari when you login after opening a link [#2493](https://github.com/BlueBrain/nexus/issues/2493)
- BUGFIX: Opening File resources results in JSON payload error [#2569](https://github.com/BlueBrain/nexus/issues/2569)
- BUGFIX: Delete Projects: Delete ProjectsCounts from deleted project [#2722](https://github.com/BlueBrain/nexus/issues/2722)

## v1.6.0-M2 (17.08.2021)

This is a milestone release for v1.6.0, containing a series of bugfixes and improvements:

- Initial full text search on all projects (based on access) [#2610](https://github.com/BlueBrain/nexus/issues/2610)
- Restrict volume of data through quotas (static configuration) [#2525](https://github.com/BlueBrain/nexus/issues/2525)
- Repurpose the search bar to enable jump to project [#2609](https://github.com/BlueBrain/nexus/issues/2609)
- Ability to paginate search results based on user view port or preference [#2623](https://github.com/BlueBrain/nexus/issues/2623)
- Mooc tutorial update [#2652](https://github.com/BlueBrain/nexus/pull/2652)
- YouTube Video Viewer Plugin [#1697](https://github.com/BlueBrain/nexus/issues/1697)
- BUGFIX: Fix database initialization failures [#2689](https://github.com/BlueBrain/nexus/issues/2689)
- BUGFIX: Fix SVG import in Fusion [#1553](https://github.com/BlueBrain/nexus/issues/1553)
- BUGFIX: Create Blazegraph namespace on view updates [#2668](https://github.com/BlueBrain/nexus/issues/2668)
- BUGFIX: Cannot edit the JSON payload of a View [#1407](https://github.com/BlueBrain/nexus/issues/1407)
- BUGFIX: Search Result Table doesn't always show values even though it should from (Facet Config) [#2160](https://github.com/BlueBrain/nexus/issues/2160)
- BUGFIX: Indexing storages via synchronous indexing does not properly encrypt the source [#2658](https://github.com/BlueBrain/nexus/issues/2658)


## v1.6.0-M1 (02.08.2021)

This is a milestone release for v1.6.0, containing a series of bugfixes and improvements:

- Add support for Accept-Encoding on delta [#2589](https://github.com/BlueBrain/nexus/issues/2589)
- Add support for compression on the HttpClient [#2588](https://github.com/BlueBrain/nexus/issues/2588)
- PDF Viewer Plugin [#2566](https://github.com/BlueBrain/nexus/issues/2566)
- CSV, TSV Table Viewer [#2565](https://github.com/BlueBrain/nexus/issues/2565)
- Support JWT "aud" (audience) claim [#2546](https://github.com/BlueBrain/nexus/issues/2546)
- Enable consistent resource writes through the API [#2537](https://github.com/BlueBrain/nexus/issues/2537), [#2573](https://github.com/BlueBrain/nexus/issues/2573), [#1395](https://github.com/BlueBrain/nexus/issues/1395)
- Use separate blazegraph client for API queries [#2530](https://github.com/BlueBrain/nexus/issues/2530)
- Support multiple versions in the website documentation [#2529](https://github.com/BlueBrain/nexus/issues/2529)
- Provision a default project for a user upon his first interaction with the API while authenticated [#2520](https://github.com/BlueBrain/nexus/issues/2520)
- Passivate view streams after an idleTimeout [#2461](https://github.com/BlueBrain/nexus/issues/2461), [#2518](https://github.com/BlueBrain/nexus/issues/2518)
- Reduce integration tests footprint on a local machine [#2491](https://github.com/BlueBrain/nexus/issues/2491)
- Blazegraph query timeout [#2469](https://github.com/BlueBrain/nexus/issues/2469)
- BUGFIX: ACLs payload not handled as expected [#2626](https://github.com/BlueBrain/nexus/issues/2626)
- BUGFIX: Deprecating resources using the generic endpoint doesn't work [#2597](https://github.com/BlueBrain/nexus/issues/2597)
- BUGFIX: Listings sort parameter only allows for metadata sorting [#2596](https://github.com/BlueBrain/nexus/issues/2596), [#2592](https://github.com/BlueBrain/nexus/issues/2592)
- BUGFIX: Cannot download metadata or data through data cart [#2581](https://github.com/BlueBrain/nexus/issues/2581)
- BUGFIX: Unsupported new grant types in keycloak [#2580](https://github.com/BlueBrain/nexus/issues/2580)
- BUGFIX: Fusion: Saving/Deprecating views results in 404 [#2570](https://github.com/BlueBrain/nexus/issues/2570)
- BUGFIX: Fix data table actions and layout [#2556](https://github.com/BlueBrain/nexus/issues/2556)
- BUGFIX: Use enhanced notifications everywhere [#2550](https://github.com/BlueBrain/nexus/issues/2550)
- BUGFIX: Defaultquery text search is acting up [#2536](https://github.com/BlueBrain/nexus/issues/2536)
- BUGFIX: When a prefixmapping that collides with a valid Uri schema is defined, it is not used [#2532](https://github.com/BlueBrain/nexus/issues/2532)
- BUGFIX: Delta client crashes at startup [#2521](https://github.com/BlueBrain/nexus/issues/2521)
- BUGFIX: Metadata plugin doesn't show up when there is no `contribution.agent` [#2506](https://github.com/BlueBrain/nexus/issues/2506)
- BUGFIX: Studios don't work with AggregateSparqlView [#2484](https://github.com/BlueBrain/nexus/issues/2484)
//TODO update release date
The latest stable release is **v1.6.0** released on **12.10.2021**.

## v1.6.0

### Deprecations

- @ref:[Nexus client](../utilities/index.md#nexus-cli)
- @ref:[Indexing data in other systems](../utilities/index.md#indexing-data-in-other-systems)
- @ref:[RemoteDiskStorage](../delta/api/storages-api.md#remote-disk-storage)

### New features / enhancements

- Introduced a plugin to search among different projects
- Introduced PDF, CSV, TSV and Youtube Viewer Fusion Plugins
- Add basic authentication to access a secured Elasticsearch cluster
- Handle user-defined queries to Blazegraph with a dedicated client
- Introduced a plugin to analyze properties and relationships of resources within a project
- Synchronous indexing
- Listing of resources outside the project scope
- The RDF parser to validate resources is now configurable
- Automatic project provisioning
- Introduced quotas on projects
- Project deletion (on demand and automatic)
- Tagging resources after deprecation

A detailed list of changes included in the release can be found in the @ref:[release notes](v1.6-release-notes.md).

## v1.5.1 (04.06.2021)

Expand Down
11 changes: 11 additions & 0 deletions docs/src/main/paradox/docs/releases/v1.5-to-v1.6-migration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# v1.5 To v1.6 Migration

The v1.6 release of Delta includes a change on how project statistics are computed which requires to run a query against the Cassandra primary store.

## Delete the project statistics row in the projections_progress table

This command has to be run after stopping Delta 1.5 and before starting Delta 1.6

DELETE FROM {delta_1_5}.projections_progress where projection_id = 'ProjectsCounts';

Note that as project statistics will be recomputed according to all events in the system, it may take some time for the statistics computation to reach back to the most recent events.
104 changes: 104 additions & 0 deletions docs/src/main/paradox/docs/releases/v1.6-release-notes.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# v1.6 Release Notes

> This is an incremental software release that adds several improvements across the board. The API is backwards
compatible to v1.5.

//TODO update the issue link if needed
For the detailed list of updates in this release, see the
@link:[list of addressed issues](https://github.com/BlueBrain/nexus/issues?q=is%3Aissue+is%3Aclosed+created%3A2021-05-01..2021-10-12+){ open=new }
since v1.5.

If you are currently using Nexus Delta from 1.5.x, please
visit the @ref:[migration instructions](v1.5-to-v1.6-migration.md) for the necessary steps to successfully upgrade.

## Nexus Delta

The main goal of this release is to allow users to search across multiple projects through @ref:[global search](../getting-started/running-nexus/search-configuration.md)

Secondary goals include:

* The possibility for Delta to query a secured Elasticsearch cluster
* The introduction of a dedicated Blazegraph client for user-defined queries with the possibility to define a timeout to improve Blazegraph stability
* Synchronous indexing
* The analysis of properties and relationships of resources within a project through the @ref:[graph analytics plugin](../delta/api/graph-analytics-api.md)
* The listing of resources within an organization and within all projects
* The configuration of the RDF parser
* @ref:[The deletion of projects and all its resources](../delta/api/projects-api.md#delete)
* Automatic project provisioning
* @ref:[The introduction of quotas](../delta/api/quotas.md)
* Tagging resources after deprecation

A series of bugs have also been fixed with this new release.

### Deprecations

These two features are now marked as deprecated and will be removed in an upcoming release:
- @ref:[Nexus client](../utilities/index.md#nexus-cli)
- @ref:[Indexing data in other systems](../utilities/index.md#indexing-data-in-other-systems)
- @ref:[RemoteDiskStorage](../delta/api/storages-api.md#remote-disk-storage)

### Global search

Before 1.6, searching across several projects was not available in Nexus. The first version of Global search included in this release
aims to fill this blank.

Global search is composed of 2 parts:

* A Delta plugin which allows to index and query resources according to a given configuration
* A matching plugin in Fusion allows to query the endpoints provided by Delta to filter/sort results and customize how these results are displayed to the users

More details about search and how to configure it is provided @ref:[here](../getting-started/running-nexus/search-configuration.md)

Please note that this first version is experimental in both Delta and Fusion and that improvements and possibly breaking
changes may be introduced in the upcoming releases.

### Secure the Delta Elasticsearch client

With 1.6, it is now possible to define credentials to query a secured Elasticsearch cluster: @ref:[Elasticsearch configuration](../getting-started/running-nexus/configuration/index.md#elasticsearch-views-plugin-configuration).

### Dedicated Blazegraph client for user-defined queries

User-defined queries could bring instability to Blazegraph because of their complexity. In 1.6, a @link:[dedicated client with a timeout](https://github.com/BlueBrain/nexus/blob/master/delta/plugins/blazegraph/src/main/resources/blazegraph.conf#L34) which triggers a circuit breaker in Blazegraph has been introduced in order to solve that problem.

### Synchronous indexing

From 1.6, the different types of resources can be @ref:[indexed directly after creation/modification](../delta/api/resources-api.md#indexing) without waiting for the background indexing process to pick it up.

### Graph analytics

This @ref:[new plugin](../delta/api/graph-analytics-api.md) helps users to inspect the properties and relationships of the different resources in a given project.

Only the Delta endpoints are available in this release but a Nexus Fusion plugin will be released soon to make use of it.

### Listings

The different types of resources can now be listed at the @ref:[organization level](../delta/api/resources-api.md#within-an-organization) and @ref:[within all the projects](../delta/api/resources-api.md#within-all-projects) the current user has access to.

### Configurable RDF parser

The RDF parser allowing to validate incoming data can now be @link:[configured](https://github.com/BlueBrain/nexus/blob/master/delta/app/src/main/resources/app.conf#L83) in order to be more or less strict.

### Automatic project provisioning

When enabled, a dedicated project is created for the current user on its first access to the Nexus platform.

How to enable and configure it is detailed @ref:[here](../getting-started/running-nexus/configuration/index.md#automatic-project-provisioning)

### Deletion of projects

The deletion of projects and all its enclosed resources is now possible by enabling a configuration flag via a @ref:[dedicated endpoint](../delta/api/projects-api.md#delete).

### Quotas

It is now possible to define via static configurations @ref:[quotas on projects](../delta/api/quotas.md).

### Tagging resources after deprecation

The different types of resources can now be tagged after their deprecation.

## Nexus Fusion
TODO

## Nexus Forge

See @link:[Nexus Forge](https://nexus-forge.readthedocs.io/en/latest/index.html){ open=new } documentation for release notes.
8 changes: 8 additions & 0 deletions docs/src/main/paradox/docs/utilities/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,10 @@ neuroscience data.

### Nexus CLI

@@@ warning
The Nexus client is now deprecated and will be removed in an upcoming release.
@@@

A command line interface (CLI) to perform basic operations on a Nexus deployment.

@link:[Source Code](https://github.com/BlueBrain/nexus-cli){ open=new }
Expand Down Expand Up @@ -111,5 +115,9 @@ Another utility packages was written to facilitate chained calling behavior, whi

### Indexing data in other systems

@@@ warning
This feature is now deprecated and will be removed in an upcoming release.
@@@

Nexus also provides a CLI tool to index data into other systems. PostgreSQL and InfluxDB are currently supported.
More information about this tool can be found @ref:[here](projections.md).
4 changes: 4 additions & 0 deletions docs/src/main/paradox/docs/utilities/projections.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Indexing data in other systems

@@@ warning
This feature is now deprecated and will be removed in an upcoming release.
@@@

## Introduction

Blue Brain Nexus supports out of the box document and RDF graph based indices of the data through Elasticsearch, Sparql and
Expand Down

0 comments on commit ca80068

Please sign in to comment.