Implement STAC and OGC API compatibility improvements #905
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Expand the STAC catalogue coverage links to provide links for multiple formats. - For TIFF and PNG OGC coverages, apply the squeeze method to remove any singleton dimensions.
- Add support for the scale-factor parameter to the main coverage endpoint. - Improve handling of bbox parameters: they are now applied to the axes in the order that they appear in the dataset, as required in the specification. Inverted y-axes are also handled. - Handle half-open datetime intervals. - More parameter checking: various invalid parameters will now result in a 400 response rather than a server-side error.
For the coverages endpoint, scale-axes, scale-size, subset-crs, and bbox-crs are not yet implemented. The server now makes this explicit by returning an HTTP 501 Not Implemented response if any of these parameters are supplied.
pont-us
changed the title
The improved bbox parameter handling for coverages had mistakenly also retained the previous implementation, causing incorrect behaviour. This commit removes the old implementation.
This makes it easier for clients to find a coverage in a particular format.
If a client attempts supplies a coverage subset parameter specifying an axis that doesn't exist in the dataset, the server now returns a 400 bad request response rather than a server error.
- Strip quotation marks from time specifiers in coverages subset parameter, if present. - Check more robustly for invalid axis names (some cases that previously produced server errors now produce a 400 Bad Request response).
Previously, a dataset's CRS was reported on the basis of the spatial_ref attribute of any crs and spatial_ref variables in the dataset. Now the crs_wkt attribute is also examined if the spatial_ref attribute is absent.
- Give a 400 Bad Request response on requests for non-existent properties (i.e. variables). - Change default subsetting and bbox CRS from EPSG:4326 to OGC:CRS84. - Implement (as yet untested) reprojection for bbox.
Initial implementation was extremely inefficient; bbox now works better for large datasets.
- Run datetime parameter values through ensure_time_index_compatible function to fix timezone-aware / -naive incompatibilities - Handle quotations around single time values when subsetting (previously only done for ranges) - If writing an image from a Dataset with a single variable, write the relevant DataArray rather than the whole Dataset (avoids errors due to additional "bnds" dimensions).
- Handle subset indices with quotation marks. - Ensure that time-based subset specifiers have the same timezone awareness as the datasets they are indexing. - Produce a more informative error for an unparseable subset specifier.
bboxes are now re-applied after transformation into the final CRS to ensure that the crop is as small as possible.
The OGC API - Coverages implementation now allows more flexibility in axis specification; in particular, axis identifiers from the CRS can be used. Handling of latitude ranges has also been improved.
As a final fallback, subsetting in OGC API - Coverages support now tries interpreting the subsetting axis name directly as the name of a dataset dimension.
The OGC API Coverages implementation now produces a "Bad Request" response if the client requests a coverage with zero size, or one larger than a predefined limit.
Subsetting behaviour now conforms more closely to the specification, performing geographical subsetting in the subsetting CRS rather than the native CRS.
Subsetting now handles geographic subset specifiers more cleanly, and reapplies geographic subsetting in the final CRS to ensure that the returned area is no larger than necessary. Some coverages code has also been refactored.
Previously, requests for too large coverages produced a general "400 Bad Request" response. They now produce the more appropriate "413 Content Too Large".
The main coverages endpoint now includes the required Content-Bbox and Content-Crs headers in its response.
The get_coverage_data method and the implementing methods that it calls have become increasingly complex and is becoming difficult to maintain as more of the Coverages draft standard is implemented. In part, this is because it mixes syntactic parsing of the request parameters with the actual processing operations. This commit adds a class CoveragesRequest which parses a coverage HTTP request into a clean Python data structure. So far it is not being used: a subsequent commit will update get_coverage_data et al. to work with a pre-parsed CoveragesRequest instance rather than values extracted directly from the HTTP request.
- Fix a bug which caused ordering of bbox elements to be interpreted incorrectly in some cases. - Add/expand unit tests to improve test coverage.
The OGC API - Coverages implementation no longer tries to map geographic subsetting axis specifiers onto corresponding dataset dimensions; instead, it just parses their names directly. Determination of the relevant dataset dimensions is now done after transformation of the geographic subsetting bounding box into the dataset's native CRS. The _crs_axis_name_to_dim_name has been removed.
TonioF
requested changes
Dec 1, 2023
Co-authored-by: Tonio Fincke <[email protected]>
- Rename the util module to request. - Prefix various non-public implementation methods of CoveragesRequest with _.
If a client makes a coverage request including the datetime parameter when the dataset has no time dimension, the parameter will be ignored. Previously, an error was raised.
Several minor changes requested in PR review: - Update docstring for get_coverage_data - Remove a commented-out line of code - rename _correct_inverted_y_range to _correct_inverted_y_range_if_necessary - Minor refactoring of_transform_bbox
Co-authored-by: Tonio Fincke <[email protected]>
thomasstorm
approved these changes
Dec 5, 2023
forman
reviewed
Dec 7, 2023
xcube/webapi/ows/stac/context.py
Outdated
Comment on lines
36
to
38
| # TODO: decide what else to include in this list. | ||
| # Listing all 11396 CRSs currently known to pyproj seems | ||
| # impractical. Make it a configuration option, perhaps? |
There was a problem hiding this comment.
Is this really a TODO? If so, say why this is needed.
Actually I don't understand why we limit it to geographic. xcube can reproject to any CRS, see tiles generation.
There was a problem hiding this comment.
I've downgraded it to a "check".
Co-authored-by: Norman Fomferra <[email protected]>
Co-authored-by: Norman Fomferra <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes #904 .
ensure_time_index_compatiblefunction to fix timezone-aware / -naive incompatibilitiescrsandcrs_storageproperties to STAC data.Checklist:
docs/source/*CHANGES.md