From f3717fd75976f250d7d6ea21536763977411f156 Mon Sep 17 00:00:00 2001 From: Hugo Ledoux Date: Fri, 21 Jun 2024 10:05:30 +0200 Subject: [PATCH] Update CityJSONSeq page --- cityjsonseq/index.md | 68 +++++++++++++++++++++++--------------------- 1 file changed, 36 insertions(+), 32 deletions(-) diff --git a/cityjsonseq/index.md b/cityjsonseq/index.md index dc4be8d347..6beee4d625 100644 --- a/cityjsonseq/index.md +++ b/cityjsonseq/index.md @@ -13,21 +13,21 @@ permalink: /cityjsonseq/ 1. TOC {:toc} -- - - +- - - -The CityJSON Text Sequence---*CityJSONSeq* for short---is a format based on [JSON Text Sequences](https://datatracker.ietf.org/doc/html/rfc7464) and CityJSON, inspired by [GeoJSON Text Sequences](https://datatracker.ietf.org/doc/html/rfc8142). -The idea is to decompose a (often large) CityJSON file into its features (eg each building, each bridge, each road, etc.), to create several JSON objects (of type [`CityJSONFeature`](https://www.cityjson.org/specs/#text-sequences-and-streaming-with-cityjsonfeature)), and stream/store them in a JSON Text Sequence (for instance [ndjson -- newline delimited JSON](https://github.com/ndjson/ndjson-spec/)). +CityJSON Text Sequence---*CityJSONSeq* for short---is a format based on [JSON Text Sequences](https://datatracker.ietf.org/doc/html/rfc7464) and CityJSON, inspired by [GeoJSON Text Sequences](https://datatracker.ietf.org/doc/html/rfc8142). +The idea is to decompose a (often large) CityJSON dataset into its features (eg each building, each bridge, each road, etc.), to create several JSON objects (of type [`CityJSONFeature`](https://www.cityjson.org/specs/#text-sequences-and-streaming-with-cityjsonfeature)), and stream/store them in a JSON Text Sequence (for instance [ndjson -- newline delimited JSON](https://github.com/ndjson/ndjson-spec/)). ## CityJSONSeq specifications -We follow the specifications of *[ndjson -- newline delimited JSON](https://github.com/ndjson/ndjson-spec/)* and we add 2 constraints for handling CityJSON: +We follow the specifications of *[ndjson -- newline delimited JSON](https://github.com/ndjson/ndjson-spec/)* and we add 2 constraints (#4 and #5): 1. each JSON Object must conform to the [JSON Data Interchange Format specifications](https://datatracker.ietf.org/doc/html/rfc8259) and be written as a UTF-8 string; 2. each JSON Object must be followed by a new-line (LF: `'\n'`) character, and it may be preceded by a carriage-return (CR: `'\r'`); 3. a JSON Object must not contain the new-line or carriage-return characters; 4. the first JSON Object must be of type `'CityJSON'` (see below for details); - 5. the following JSON Objects are of type `'CityJSONFeature'`; + 5. the following JSON Objects must be of type `'CityJSONFeature'`; {: .highlight } **Suggested convention**: we recommend using the extension `.city.jsonl` when saving the JSON Objects to a file. @@ -37,7 +37,7 @@ We follow the specifications of *[ndjson -- newline delimited JSON](https://gith A `CityJSONFeature` object represents **one** feature in a CityJSON object, for instance a `"Building"` (with eventually its children `"BuildingPart"` and/or `"BuildingInstallation"`). The idea is to decompose a large area into each of its features, and each feature is a stored as a `CityJSONFeature`. -Each feature is independent, and has its own list of vertices (which is thus local). +Each feature is independent and has its own list of vertices (which is thus local). See the [full specifications for a CityJSONFeature](https://www.cityjson.org/specs/#text-sequences-and-streaming-with-cityjsonfeature). @@ -73,16 +73,17 @@ See the [full specifications for a CityJSONFeature](https://www.cityjson.org/spe ## Streaming 3D cities with CityJSONSeq -Since we want to have access to some properties, eg [`"transform"`](https://www.cityjson.org/specs/#transform-object), the [CRS](https://www.cityjson.org/specs/#referencesystem-crs) or [`"geometry-templates"`](https://www.cityjson.org/specs/#geometry-templates), those need to be known by the client/software parsing the stream. +To be able to reconstruct the features and geo-reference them, some properties are necessary, eg [`"transform"`](https://www.cityjson.org/specs/#transform-object), the [CRS](https://www.cityjson.org/specs/#referencesystem-crs) or [`"geometry-templates"`](https://www.cityjson.org/specs/#geometry-templates). +Thus those need to be known by the client/software parsing the stream. The first JSON Object should therefore be of type `"CityJSON"` and contain the necessary information. Notice that the properties `"CityObjects"` and `"vertices"` are mandatory (for the [JSON Object to be valid](https://www.cityjson.org/specs/#cityjson-object)) but should be respectively an empty JSON object and an empty array. -One example would be: +Here's one example: ```json {"type":"CityJSON","version":"2.0","transform": {"scale":[1.0,1.0,1.0],"translate": [0.0, 0.0, 0.0]},"metadata":{"referenceSystem":"https://www.opengis.net/def/crs/EPSG/0/7415"},"CityObjects":{},"vertices":[]} ``` -The subsequent JSON Objects must all be of type `"CityJSONFeature"`, which means a CityJSONSeq with 3 features could look like this one: +The subsequent JSON Objects must all be of type `"CityJSONFeature"`, which means a CityJSONSeq with 3 features looks like this one (it has 4 lines): ```json {"type":"CityJSON","version":"2.0","transform": {"scale":[1.0,1.0,1.0],"translate": [0.0, 0.0, 0.0]},"metadata":{"referenceSystem":"https://www.opengis.net/def/crs/EPSG/0/7415"},"CityObjects":{},"vertices":[]} @@ -94,21 +95,24 @@ The subsequent JSON Objects must all be of type `"CityJSONFeature"`, which means ## Reading and writing CityJSONSeq with cjseq -The software [cjseq](https://github.com/cityjson/cjseq) allows us to convert between CityJSON and CityJSONSeq (both directions). -cjseq has thus 2 commands: - +The software [cjseq](https://github.com/cityjson/cjseq) allows us to convert between CityJSON and CityJSONSeq, and vice-versa +. +cjseq has at the moment 3 commands: + 1. cat: CityJSON ==> CityJSONSeq 2. collect: CityJSONSeq ==> CityJSON + 3. filter: to filter a stream based on feature type, bounding box, distance to a location, etc. We can create a CityJSONSeq stream (with the first line containing the metadata) this way: -``` +```sh cjseq cat -f myfile.city.json > myfile.city.jsonl ``` And conversely convert a stream to a CityJSON file: -``` -cat myfile.city.jsonl | cjseq collect > myfile_2.city.json + +```sh +cat myfile.city.jsonl | cjseq collect > myfile.city.json ``` @@ -117,10 +121,10 @@ cat myfile.city.jsonl | cjseq collect > myfile_2.city.json | dataset | CityJSONSeq file | description | | ------- | ---------------- | ----------- | | 3DBAG | [3dbag_b2.city.jsonl](https://3d.bk.tudelft.nl/opendata/cityjson/cityjsonl/3dbag_b2.city.jsonl) | 2 buildings randomly selected from the 3DBAG, LoD2.2 only | -| Montréal | [montréal_b4.city.jsonl](https://3d.bk.tudelft.nl/opendata/cityjson/cityjsonl/montréal_b4.city.jsonl) | 4 buildings randomly selected from the Montréal dataset | +| Montréal | [montréal_b4.city.jsonl](https://3d.bk.tudelft.nl/opendata/cityjson/cityjsonl/montréal_b4.city.jsonl) | 4 buildings randomly selected from the Montréal dataset | -## Validating a stream +## Validating a stream ### With the online validator @@ -131,24 +135,24 @@ You can just drop those files and the validator will indicate, *per line*, if th [![](validator.png)](https://validator.cityjson.org) -### Locally with cjseqval +### Locally with cjval -The official [schema-validator of CityJSON (called cjval)](https://github.com/cityjson/cjval) can validate CityJSONSeq streams with its binary `cjseqval`. +The official [schema-validator of CityJSON (called cjval)](https://github.com/cityjson/cjval) can validate CityJSONSeq streams. Each line is individually validated and errors reported: -```bash -cjseq cat -f myfile.city.json | cjseqval --verbose -l.1 ✅ -l.2 ❌ {"attributes":{"function":"something"},"geometry":[{"boundaries":[[[[0,1,2,3]],[[4,5,0,3]],[[5,6,1,0]],[[6,7,2,1]],[[3,2,7,4]],[[7,6,5,4]]]],"lod":"1","type":"Solid"}],"type":"+99999GnericCityObject"} is not valid under any of the given schemas [path:/CityObjects/id-1] | -l.3 ✅ -l.4 🟡 Vertex (0, 1000, 0) duplicated | Vertex #8 is unused | -l.5 ✅ -l.6 ✅ +```sh +cjseq cat -f myfile.city.json | cjval --verbose +1 ✅ [1st-line for metadata] +2 ✅ [NL.IMBAG.Pand.0503100000019581] +3 ❌ [NL.IMBAG.Pand.0503100000014639] Additional properties are not allowed ('CityObject' was unexpected) [path:] "CityObjects" is a required property [path:] | +4 ✅ [NL.IMBAG.Pand.0503100000005139] +... +61 🟡 [NL.IMBAG.Pand.0503100000020017] Vertex (185470, 295278, 4110) duplicated | Vertex #24 is unused | +62 ✅ [NL.IMBAG.Pand.0503100000034978] ``` -## cjseqview: a small viewer for CityJSONSeq - +## cjseqview: a small viewer for CityJSONSeq ![](https://raw.githubusercontent.com/cityjson/viewcjl/main/demo.png) @@ -157,10 +161,10 @@ The [cjseqview GitHub repository](https://github.com/cityjson/cjseqview/) has mo It reads a [CityJSONSeq](https://cityjson.org/cityjsonseq) from stdin. -```bash +```sh cat ./data/b2.city.jsonl | python ./src/cjseqview.py ``` -```bash -cjio --suppress_msg Vienna_102081.city.json subset --random 5 export jsonl stdout | python ./src/cjseqview.py +```sh +cat NYC.city.jsonl | cjseq filter --random 10 | python ./src/cjseqview.py` ```