Skip to content

Commit

Permalink
correct typo and remove not needed space
Browse files Browse the repository at this point in the history
  • Loading branch information
defuneste committed Aug 16, 2024
1 parent b167bb5 commit fff064e
Show file tree
Hide file tree
Showing 6 changed files with 56 additions and 30 deletions.
32 changes: 30 additions & 2 deletions docs/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -311,7 +311,7 @@ <h3 class="no-anchor listing-title">
</a>
</div>
</div>
<div class="quarto-post image-right" data-index="4" data-categories="cli,code,geojson,json,spatial data" data-listing-date-sort="1710046800000" data-listing-file-modified-sort="1723129180185" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="5" data-listing-word-count-sort="983">
<div class="quarto-post image-right" data-index="4" data-categories="cli,code,geojson,json,spatial data,broadband" data-listing-date-sort="1710046800000" data-listing-file-modified-sort="1723820227654" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="5" data-listing-word-count-sort="974">
<div class="thumbnail">
<p><a href="./posts/awesomejq/index.html" class="no-external"></a></p><a href="./posts/awesomejq/index.html" class="no-external">
<p><img src="./posts/awesomejq/thumbnail.png" class="thumbnail-image"></p>
Expand All @@ -325,7 +325,7 @@ <h3 class="no-anchor listing-title">
<a href="./posts/awesomejq/index.html" class="no-external"></a>
</div>
<div class="listing-description">
<a href="./posts/awesomejq/index.html" class="no-external">If you are manipulating a lot of <code>GeoJSON</code> eatures/objects and want a quick CLI tool to filter and slice them, you should give jq a try! Since there are not many tutorials…</a>
<a href="./posts/awesomejq/index.html" class="no-external">If you are manipulating a lot of <code>GeoJSON</code> features/objects and want a quick CLI tool to filter and slice them, you should give jq a try! Since there are not many tutorials…</a>
</div>
</div>
<div class="metadata">
Expand All @@ -339,6 +339,34 @@ <h3 class="no-anchor listing-title">
</a>
</div>
</div>
<div class="quarto-post image-right" data-index="5" data-categories="cli,code,geojson,json,spatial data" data-listing-date-sort="1710046800000" data-listing-file-modified-sort="1723819532924" data-listing-date-modified-sort="NaN" data-listing-reading-time-sort="5" data-listing-word-count-sort="983">
<div class="thumbnail">
<p><a href="./posts/fcc-duckdb/index.html" class="no-external"></a></p><a href="./posts/fcc-duckdb/index.html" class="no-external">
<p><img src="./posts/fcc-duckdb/thumbnail.png" class="thumbnail-image"></p>
</a><p><a href="./posts/fcc-duckdb/index.html" class="no-external"></a></p>
</div>
<div class="body">
<h3 class="no-anchor listing-title">
<a href="./posts/fcc-duckdb/index.html" class="no-external">Awesome jq and GeoJSON</a>
</h3>
<div class="listing-subtitle">
<a href="./posts/fcc-duckdb/index.html" class="no-external"></a>
</div>
<div class="listing-description">
<a href="./posts/fcc-duckdb/index.html" class="no-external"></a>
</div>
</div>
<div class="metadata">
<a href="./posts/fcc-duckdb/index.html" class="no-external">
<div class="listing-date">
Mar 10, 2024
</div>
<div class="listing-author">
Olivier Leroy
</div>
</a>
</div>
</div>
</div>
<div class="listing-no-matching d-none">
No matching items
Expand Down
3 changes: 2 additions & 1 deletion docs/listings.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
"/posts/formd_intro/index.html",
"/posts/urisa-2023/index.html",
"/posts/mapping_rural_tips/index.html",
"/posts/awesomejq/index.html"
"/posts/awesomejq/index.html",
"/posts/fcc-duckdb/index.html"
]
}
]
21 changes: 11 additions & 10 deletions docs/posts/awesomejq/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -150,6 +150,7 @@ <h1 class="title">Awesome jq and GeoJSON</h1>
<div class="quarto-category">geojson</div>
<div class="quarto-category">json</div>
<div class="quarto-category">spatial data</div>
<div class="quarto-category">broadband</div>
</div>
</div>
</div>
Expand Down Expand Up @@ -196,7 +197,7 @@ <h2 id="toc-title">On this page</h2>



<p>If you are manipulating a lot of <a href="https://en.wikipedia.org/wiki/GeoJSON"><code>GeoJSON</code></a> eatures/objects and want a quick CLI tool to filter and slice them, you should give jq a try! Since there are not many tutorials that exist on using jq to manage objects in the GeoJSON family, we hope that these few tricks will help you on your learning journey.</p>
<p>If you are manipulating a lot of <a href="https://en.wikipedia.org/wiki/GeoJSON"><code>GeoJSON</code></a> features/objects and want a quick CLI tool to filter and slice them, you should give jq a try! Since there are not many tutorials that exist on using jq to manage objects in the GeoJSON family, we hope that these few tricks will help you on your learning journey.</p>
<!-- You can find plenty of great guides online, but the [documentation](https://jqlang.github.io/jq/manual/) and the [playground](https://jqplay.org/) are always a good start. -->
<div class="callout callout-style-default callout-note callout-titled">
<div class="callout-header d-flex align-content-center">
Expand Down Expand Up @@ -229,9 +230,9 @@ <h2 id="toc-title">On this page</h2>
<li><p><code>data/vt-bb.geojson</code> is the path of our file as the last argument</p></li>
<li><p><code>'.features | length'</code> is a <strong>jq filter</strong>. Remember, we are in the shell and whitespace has meaning. Not quoting it would mean that we have 4 arguments when we just have 2.</p>
<ul>
<li><p><code>.features</code> will return the json array containing every feature</p></li>
<li><p><code>|</code> will pipe the stream of features into a new filter</p></li>
<li><p><code>length</code> is a built-in function that behaves differently depending the object input. Given an array, it will return the number of elements</p></li>
<li><code>.features</code> will return the json array containing every feature</li>
<li><code>|</code> will pipe the stream of features into a new filter</li>
<li><code>length</code> is a built-in function that behaves differently depending the object input. Given an array, it will return the number of elements</li>
</ul></li>
</ul>
<section id="jq-and-small-examples" class="level2">
Expand Down Expand Up @@ -296,9 +297,9 @@ <h4 class="anchored" data-anchor-id="second-make-them-into-a-csv">Second make th
<li><p><code>(.features[0].properties | keys_unsorted)</code> here nothing new we added parentheses to enforce precedence. We are getting the header of our csv</p></li>
<li><p><code>(.features[].properties | to_entries | map(.value))</code> :</p>
<ul>
<li><p>we are starting from all our properties (not the first one)</p></li>
<li><p>passing it to <code>to_entries</code> convert our object to multiple objects with “key” / “value” (see margin)</p></li>
<li><p>finally, <code>map(.value)</code> gets all “value” for every selected features</p></li>
<li>we are starting from all our properties (not the first one)</li>
<li>passing it to <code>to_entries</code> convert our object to multiple objects with “key” / “value” (see margin)</li>
<li>finally, <code>map(.value)</code> gets all “value” for every selected features</li>
</ul></li>
</ul>

Expand All @@ -320,9 +321,9 @@ <h4 class="anchored" data-anchor-id="second-make-them-into-a-csv">Second make th
</ul>
<p>We have just explored the surface! <code>jq</code> can help to filter some specific features:</p>
<ul>
<li><p>every geometries “served” in our file?</p></li>
<li><p>the first node in every geometries)?</p></li>
<li><p>etc!</p></li>
<li>every geometries “served” in our file?</li>
<li>the first node in every geometries)?</li>
<li>etc!</li>
</ul>
<p><code>jq</code> is a generic tool for filtering json and lot of people are following the JSON spec in GeoJSON, so we can build on top of all their monumental work!</p>

Expand Down
4 changes: 2 additions & 2 deletions docs/search.json
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@
"href": "index.html",
"title": "Recent posts",
"section": "",
"text": "The top 10 micropolitan areas for raising venture capital\n\n\n\n\n\n\n\n\n\n\n\nJul 3, 2024\n\n\nBrittany Kainen\n\n\n\n\n\n\n\n\n\n\n\n\nUsing SEC Form D to estimate venture capital\n\n\n\n\n\n\n\n\n\n\n\nJul 1, 2024\n\n\nBrittany Kainen\n\n\n\n\n\n\n\n\n\n\n\n\nMDA’s URISA-2023 presentation\n\n\n\n\n\n\n\n\n\n\n\nJun 20, 2024\n\n\nOlivier Leroy, Drew Rosebush\n\n\n\n\n\n\n\n\n\n\n\n\nSix tips for mapping rural data\n\n\n\n\n\n\n\n\n\n\n\nMay 24, 2024\n\n\nCamden Blatchly\n\n\n\n\n\n\n\n\n\n\n\n\nAwesome jq and GeoJSON\n\n\n\n\n\n\n\n\n\n\n\nMar 10, 2024\n\n\nOlivier Leroy\n\n\n\n\n\n\nNo matching items"
"text": "The top 10 micropolitan areas for raising venture capital\n\n\n\n\n\n\n\n\n\n\n\nJul 3, 2024\n\n\nBrittany Kainen\n\n\n\n\n\n\n\n\n\n\n\n\nUsing SEC Form D to estimate venture capital\n\n\n\n\n\n\n\n\n\n\n\nJul 1, 2024\n\n\nBrittany Kainen\n\n\n\n\n\n\n\n\n\n\n\n\nMDA’s URISA-2023 presentation\n\n\n\n\n\n\n\n\n\n\n\nJun 20, 2024\n\n\nOlivier Leroy, Drew Rosebush\n\n\n\n\n\n\n\n\n\n\n\n\nSix tips for mapping rural data\n\n\n\n\n\n\n\n\n\n\n\nMay 24, 2024\n\n\nCamden Blatchly\n\n\n\n\n\n\n\n\n\n\n\n\nAwesome jq and GeoJSON\n\n\n\n\n\n\n\n\n\n\n\nMar 10, 2024\n\n\nOlivier Leroy\n\n\n\n\n\n\n\n\n\n\n\n\nAwesome jq and GeoJSON\n\n\n\n\n\n\n\n\n\n\n\nMar 10, 2024\n\n\nOlivier Leroy\n\n\n\n\n\n\nNo matching items"
},
{
"objectID": "about.html",
Expand All @@ -39,7 +39,7 @@
"href": "posts/awesomejq/index.html",
"title": "Awesome jq and GeoJSON",
"section": "",
"text": "If you are manipulating a lot of GeoJSON eatures/objects and want a quick CLI tool to filter and slice them, you should give jq a try! Since there are not many tutorials that exist on using jq to manage objects in the GeoJSON family, we hope that these few tricks will help you on your learning journey.\nIn these examples, we are using a GeoJSON file of Vermont census blocks with attributes related to our work on broadband data. While it is not a deeply nested JSON, it is perfect to illustrate some common use cases.\nA quick check lets us know that it is 94 MB. Not “that” big but still decent.\nFirst, let’s see how many features it has. Here’s how we can approximate that:\nThis is a decent estimate, but we are counting some rows at the top and bottom of the file that are not features (try head -n 5 and tail on it if you are curious).\nWe can also use jq:\nThis is the correct number of blocks! How did that magic work? Let’s decompose our one-liner:"
"text": "If you are manipulating a lot of GeoJSON features/objects and want a quick CLI tool to filter and slice them, you should give jq a try! Since there are not many tutorials that exist on using jq to manage objects in the GeoJSON family, we hope that these few tricks will help you on your learning journey.\nIn these examples, we are using a GeoJSON file of Vermont census blocks with attributes related to our work on broadband data. While it is not a deeply nested JSON, it is perfect to illustrate some common use cases.\nA quick check lets us know that it is 94 MB. Not “that” big but still decent.\nFirst, let’s see how many features it has. Here’s how we can approximate that:\nThis is a decent estimate, but we are counting some rows at the top and bottom of the file that are not features (try head -n 5 and tail on it if you are curious).\nWe can also use jq:\nThis is the correct number of blocks! How did that magic work? Let’s decompose our one-liner:"
},
{
"objectID": "posts/awesomejq/index.html#jq-and-small-examples",
Expand Down
18 changes: 3 additions & 15 deletions posts/awesomejq/index.qmd
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@
title: "Awesome jq and GeoJSON"
author: "Olivier Leroy"
date: "2024-03-10"
categories: [cli, code, geojson, json, spatial data]
categories: [cli, code, geojson, json, spatial data, broadband]
image: thumbnail.png
---

If you are manipulating a lot of [`GeoJSON`](https://en.wikipedia.org/wiki/GeoJSON) eatures/objects and want a quick CLI tool to filter and slice them, you should give jq a try!
If you are manipulating a lot of [`GeoJSON`](https://en.wikipedia.org/wiki/GeoJSON) features/objects and want a quick CLI tool to filter and slice them, you should give jq a try!
Since there are not many tutorials that exist on using jq to manage objects in the GeoJSON family, we hope that these few tricks will help you on your learning journey.

<!-- You can find plenty of great guides online, but the [documentation](https://jqlang.github.io/jq/manual/) and the [playground](https://jqplay.org/) are always a good start. -->
Expand Down Expand Up @@ -44,16 +44,12 @@ jq '.features | length' data/vt-bb.geojson
This is the correct number of blocks! How did that magic work? Let’s decompose our one-liner:

- `jq` here we are calling jq like any other CLI tool

- `data/vt-bb.geojson` is the path of our file as the last argument

- `'.features | length'` is a **jq filter**. Remember, we are in the shell and whitespace has meaning.
Not quoting it would mean that we have 4 arguments when we just have 2.

- `.features` will return the json array containing every feature

- `|` will pipe the stream of features into a new filter

- `length` is a built-in function that behaves differently depending the object input. Given an array, it will return the number of elements


Expand Down Expand Up @@ -153,13 +149,10 @@ jq -r '(.features[0].properties | keys_unsorted), (.features[].properties | to_e
```

- `(.features[0].properties | keys_unsorted)` here nothing new we added parentheses to enforce precedence. We are getting the header of our csv

- `(.features[].properties | to_entries | map(.value))` :

* we are starting from all our properties (not the first one)

* passing it to `to_entries` convert our object to multiple objects with "key" / "value" (see margin)

* finally, `map(.value)` gets all "value" for every selected features

::: {.column-margin}
Expand All @@ -179,17 +172,12 @@ jq -r '(.features[0].properties | keys_unsorted), (.features[].properties | to_e
```
:::


- Finally `@csv` convert to a csv and we redirect the output later in `data/sample.csv`


We have just explored the surface! `jq` can help to filter some specific features:


- every geometries "served" in our file?

- every geometries "served" in our file?
- the first node in every geometries)?

- etc!

`jq` is a generic tool for filtering json and lot of people are following the JSON spec in GeoJSON, so we can build on top of all their monumental work!
Expand Down
8 changes: 8 additions & 0 deletions posts/fcc-duckdb/index.qmd
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
title: "Fast analytics on FCC (big) dataset"
author: "Olivier Leroy"
date: "2024-08-16"
categories: [cli, code, duckDB, broadband]
image: thumbnail.png
---

0 comments on commit fff064e

Please sign in to comment.