Geocoder: Pelias: remove exact duplicates #807
Conversation
| rewriteAutocompleteResponse(response: unknown): MultiGeocoderResponse { | ||
| const features = (response as FeatureCollection)?.features; | ||
|
|
||
| const filtered = features?.filter( |
There was a problem hiding this comment.
is there some fun CSy way of sorting this first to make it more efficient?
There was a problem hiding this comment.
If we were writing in a language with actual arrays yes you could sort by name keep a fixed array iterate by index and then stop the check once you got past the first letter you're checking against. But given that we will never be sorting more than ~50 items I think we're ok to prioritize code cleanliness. Also I don't think JS supports native pointers so how would you maintain a real array of sorted strings? Maybe a second array where you store string lengths but then you'd have to allocate a large enough string array and you'd somehow have to sync up a separate array of coordinates... yikes
| f => | ||
| f?.properties?.name === feature?.properties?.name && | ||
| // This is unclean, but we know geometry will never contain objects | ||
| JSON.stringify((feature?.geometry as Point)?.coordinates) === |
There was a problem hiding this comment.
would it not be faster to compare the individual lat lon numbers rather than stringifying the coordinates object?
There was a problem hiding this comment.
if you put them together in an AND then 99% of the time it won't even evaluate the second half of the AND because the first half of the coordinate won't match
There was a problem hiding this comment.
You are right but again I prioritized cleanliness. If we compare coordinates we'd have to cast 4 times and we'd probably have to break into a function block
daniel-heppner-ibigroup
assigned miles-grant-ibigroup and unassigned daniel-heppner-ibigroup
It seems like we've been seeing some exact duplicates coming back from Pelias in some cases. This PR removes them.
This will slow things down, but hopefully not too much.