extract functions (Geojson to Topojson) #46

Priceless-P · 2023-02-04T22:03:02Z

Extracts lines and rings from Geojson's Objects

Priceless-P · 2023-02-04T22:06:05Z

@patricoferris what do i think about this? I also intend to add extract_geomteries and extract_features_collection (if necessary).

patricoferris · 2023-02-06T21:36:58Z

src/topojson/topojson.ml

+        loop [] coordinates
+    | _ -> failwith "Not a valid LineString or Polygon"
+
+  let extract_lines = function


This comment applies to pretty much most of the other functions defined here.

I don't think we want to reimplement all of the JSON parsing logic again, GeoJSON does this already for us so we should just uses GeoJSON to extract the coordinates etc. from the geometries (since we are converting from GeoJSON to TopoJSON).

e.g.

https://github.com/geocaml/ocaml-geojson/blob/d45db9166a050da2e61ff3f04856c4788c13edc8/src/geojson/geojson_intf.ml#L124

patricoferris · 2023-02-06T21:37:50Z

src/topojson/topojson_intf.ml

@@ -282,6 +282,21 @@ module type S = sig

    val to_json : ?bbox:float array -> t -> json
    val of_json : json -> (t, [ `Msg of string ]) result
+
+    val extract_lines :


We won't actually ever expose these internal functions. Instead, eventually, we will just have something like:

val geojson_to_topojson : Geojson.t -> Topojson.t

Priceless-P · 2023-02-11T09:53:29Z

Hi @patricoferris,

module Geojson = Geojson.Make (Ezjsonm_parser)

keep giving error Unbound module Ezjsonm_parser

Also with the current implementation, will extract_lines_from_geometry also extract rings from Polgon and MultiPolygon?

patricoferris · 2023-02-11T23:16:28Z

Also with the current implementation, will extract_lines_from_geometry also extract rings from Polgon and MultiPolygon?

I think so, they are after all just lines.

keep giving error Unbound module Ezjsonm_parser

Yep, we still want to be agnostic to the JSON parser the user decided to use. You should be able to pass the same one that gets passed to our Make "functor". This is a special name for modules that are parameterised by other modules.

module Make (J : Intf.Json) = struct
  module Geojson = Geojson.Make(J)
  ...

Priceless-P · 2023-02-15T12:08:08Z

Hi @patricoferris what do you think?

patricoferris · 2023-02-15T23:05:52Z

src/topojson/topojson.ml

+            let arr = Array.map LineString.coordinates lines in
+            Array.to_list arr
+        | Polygon p ->
+            let rings = Polygon.rings p in


Hmm thinking about this a little more, perhaps we have to treat rings slightly differently because the first and last point are technically connected.

patricoferris · 2023-02-15T23:09:41Z

src/topojson/topojson.ml

+    List.iter
+      (fun line ->
+        let line_length = Array.length line in
+        for i = 0 to line_length - 2 do


Points on a line other than the first and the last have two neighbours: n - 1 and n + 1.

patricoferris · 2023-02-15T23:10:51Z

src/topojson/topojson.ml

+  let find_junctions (lines : Geojson.Geometry.Position.t array list) :
+      Geojson.Geometry.Position.t list =
+    let open Geojson.Geometry in
+    let junction_table : (Position.t, Position.t list) Hashtbl.t =


Why not make this a position to position set hash table to stay closer to how the blog post does it?

patricoferris · 2023-02-15T23:12:52Z

src/topojson/topojson.ml

+        for i = 0 to line_length - 2 do
+          let point1 = line.(i) in
+          let point2 = line.(i + 1) in
+          if Hashtbl.mem junction_table point1 then


I think for every point we visit we want to put in the hash table it's neighbours as a set. So if we are at point N then we but N, { N - 1, N - 2} as a key-value pair into the hash table.

Then if we come across N again we check that Ns neighbours are different (therefore a junction) or if they are the same then it isn't.

Priceless-P · 2023-02-20T08:33:01Z

@patricoferris I am having a lot of type incompatibility errors. Please help me out 😥

patricoferris · 2023-02-20T17:33:14Z

Ah, most of the issues were just related to giving functions a little too much generality and also we didn't expose that the positions were float arrays. Feel free to cherry-pick this commit 7b31814

Priceless-P · 2023-02-20T19:41:18Z

The type errors are gone now. :)) Thank you @patricoferris

Priceless-P · 2023-02-20T19:52:27Z

Do you think the find_junction is correction? I changed the MultiLineString coordinates to

 "coordinates": [
        [
			[1, 1], [2, 2],[3, 3],
			[4, 4], [3, 1],[4, 2]

        ], [
			[4, 2], [4, 4], [4, 6]
        ]
    ]

And the test didn't print out any junctions

patricoferris · 2023-02-27T09:57:47Z

I don't think the function is correct, see my comments inline on the code.

patricoferris

These changes are heading in the right direction, nice! Comments are inline.

patricoferris · 2023-02-27T22:30:24Z

test/topojson/test.ml

@@ -127,7 +127,7 @@ let pp_transform ppf t =

 let transform = Alcotest.testable pp_transform Stdlib.( = )

-let to_string pp a =
+let _to_string pp a =


If this function isn't being used we can probably remove it ?

patricoferris · 2023-02-27T22:35:37Z

src/topojson/topojson.ml

          let point1 = line.(i) in
-          let point2 = line.(i - 1) in
-          let point3 = line.(i + 1) in
+          let point2 = if i > 0 then line.(i - 1) else line.(line_length - 1) in


Isn't this only true if we're talking about a ring rather than any line?

patricoferris · 2023-02-27T22:35:52Z

src/topojson/topojson.ml

-          let point2 = line.(i - 1) in
-          let point3 = line.(i + 1) in
+          let point2 = if i > 0 then line.(i - 1) else line.(line_length - 1) in
+          let point3 = if i < line_length - 1 then line.(i + 1) else line.(0) in


Same question here?

patricoferris · 2023-02-27T22:37:58Z

src/topojson/topojson.ml

@@ -698,24 +696,23 @@ module Make (J : Intf.Json) = struct
    let junction_table : (Position.t, Set.t) Hashtbl.t =


We really want to know every set of neighbours to begin with (let's get the algorithm correct before optimising). So I think we want something like: (Position.t, Set.t list) Hashtbl.t.

patricoferris · 2023-02-27T22:39:24Z

src/topojson/topojson.ml

-          let point2 = line.(i - 1) in
-          let point3 = line.(i + 1) in
+          let point2 = if i > 0 then line.(i - 1) else line.(line_length - 1) in
+          let point3 = if i < line_length - 1 then line.(i + 1) else line.(0) in
          if Hashtbl.mem junction_table point1 then
            let neighbors = Hashtbl.find junction_table point1 in
            Hashtbl.replace junction_table point1


Instead of adding the neighbours to an ever growing set, I think we want to keep them distinct so we know what two neighbours belong together. Consider the problem of having two lines ABC and CBA. B in this case is not a junction because the neighbours A and C are the same in both lines.

patricoferris · 2023-02-27T22:40:25Z

src/topojson/topojson.ml

          if Hashtbl.mem junction_table point1 then
            let neighbors = Hashtbl.find junction_table point1 in
            Hashtbl.replace junction_table point1
              (Set.add point2 (Set.add point3 neighbors))
          else
            Hashtbl.add junction_table point1 (Set.of_list [ point2; point3 ]);
          let neighbors = Hashtbl.find junction_table point1 in
-          if Set.cardinal neighbors > 2 then
-            ref junction := point1 :: !(ref junction)
+          if Set.cardinal neighbors > 2 then junction := point1 :: !junction


With the above changes this will become List.length when we store the list of possible neighbours and store a new pair iff they are different as defined in the algorithm.

extract functions

5507ac7

fix typo error & fmt

ff766d5

patricoferris reviewed Feb 6, 2023

View reviewed changes

fix

35fc7a7

implement find junction

cb14f25

patricoferris reviewed Feb 15, 2023

View reviewed changes

find_junction test

4b35e92

fix find_junction

ad771c3

patricoferris reviewed Feb 27, 2023

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

extract functions (Geojson to Topojson) #46

extract functions (Geojson to Topojson) #46

Priceless-P commented Feb 4, 2023

Priceless-P commented Feb 4, 2023

patricoferris Feb 6, 2023

patricoferris Feb 6, 2023

patricoferris Feb 6, 2023

Priceless-P commented Feb 11, 2023

patricoferris commented Feb 11, 2023 •

edited

Loading

Priceless-P commented Feb 15, 2023

patricoferris Feb 15, 2023

patricoferris Feb 15, 2023

patricoferris Feb 15, 2023

patricoferris Feb 15, 2023

Priceless-P commented Feb 20, 2023

patricoferris commented Feb 20, 2023

Priceless-P commented Feb 20, 2023

Priceless-P commented Feb 20, 2023

patricoferris commented Feb 27, 2023

patricoferris left a comment

patricoferris Feb 27, 2023

patricoferris Feb 27, 2023

patricoferris Feb 27, 2023

patricoferris Feb 27, 2023

patricoferris Feb 27, 2023

patricoferris Feb 27, 2023

		@@ -698,24 +696,23 @@ module Make (J : Intf.Json) = struct
		let junction_table : (Position.t, Set.t) Hashtbl.t =

extract functions (Geojson to Topojson) #46

Are you sure you want to change the base?

extract functions (Geojson to Topojson) #46

Conversation

Priceless-P commented Feb 4, 2023

Priceless-P commented Feb 4, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Priceless-P commented Feb 11, 2023

patricoferris commented Feb 11, 2023 • edited Loading

Priceless-P commented Feb 15, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Priceless-P commented Feb 20, 2023

patricoferris commented Feb 20, 2023

Priceless-P commented Feb 20, 2023

Priceless-P commented Feb 20, 2023

patricoferris commented Feb 27, 2023

patricoferris left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

patricoferris commented Feb 11, 2023 •

edited

Loading