Skip to content
This repository has been archived by the owner on Jul 11, 2023. It is now read-only.

Geo packages + reboot #2

Merged
merged 18 commits into from
Jan 16, 2018
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Contributing

Thanks for being here.

## Project participation guidelines

Please read our [project participation guidelines](https://okfn.org/project-participation-guidelines/) that apply to all our projects and contributors.

## Contribute

### Tell us your plans

Before contributing, let us know your plans - [raise an issue](https://github.com/frictionlessdata/example-data-packages/issues), [start a topic](https://discuss.okfn.org/c/frictionless-data) on the discussion forum, or [chat with us on Gitter](https://gitter.im/frictionlessdata/chat).

### Fork the repository

To contribute to this project you will need to [create a fork of this repository](https://help.github.com/articles/fork-a-repo/).

### Set up Goodtables.io validation

To run the validation tests each time you commit a change to your GitHub repository, you'll need to:
- [login to Goodtables.io](http://goodtables.io)
- use `Manage Sources` to select your repository
- make a change to the data in a data package
- then check the results. ([sample results](https://goodtables.io/github/frictionlessdata/example-data-packages))

[Learn more about Goodtables.io](http://okfnlabs.org/blog/2017/05/22/introducing-the-new-goodtables-library-and-goodtablesio.html)

### Make your changes

To add a data package:

- Follow the directory structure in the [Readme](./README.md/#data-packages).
- Add your data package example
- Make sure the Goodtables.io tests pass
- Add a `README.md` using the template

### Submit a pull request

- Please be patient while volunteers review your pull request
- We may ask you to make changes to improve improve your contributions

### Celebrate

You're now a [contributor](https://github.com/frictionlessdata/example-data-packages/graphs/contributors) :tada:
25 changes: 25 additions & 0 deletions .github/ISSUE_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
> Use the Issue Title above to provide a summary of the problem, data request or other issue.
> Fill out the relevant sections below. Delete sections as appropriate.
> Provide as much detail as you can to help us help you.

### Desired Data
> Use this section for new data requests. If you're reporting a problem, delete this section.

### Current Behaviour (for problems)
> When reporting a problem, describe what happens that isn't expected behaviour?

### Expected Behaviour
> When reporting a problem, describe what should happen

### Steps to Reproduce
> When reporting a problem, provide a set of steps so others can reproduce the problem.

1.
2.
3.

### Your Environment
> Include details about the environment you experienced the problem - this will help us fix things quicker.

*
*
9 changes: 9 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
> Append "WIP" to the PR if it is "work in progress" and you don't want it merged yet

Fixes #

Changes proposed in this pull request:

-
-
-
125 changes: 121 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,126 @@
# Example Data Packages

[![goodtables.io](https://goodtables.io/badge/github/frictionlessdata/example-data-packages.svg)](https://goodtables.io/github/frictionlessdata/example-data-packages)
![goodtables.io](https://goodtables.io/badge/github/frictionlessdata/example-data-packages.svg)

**These examples are outdated. Please see the current [specs](http://specs.frictionlessdata.io/) site for updated examples)**
This repository contains example data packages to help you learn about [Frictionless Data](https://frictionlessdata.io/). The example data packages:

These Data Packages are used primarily as reference material for the [Guides](http://frictionlessdata.io/guides/) section for the [Frictionless Data](http://frictionlessdata.io/) website.
- illustrate the current Frictionless Data [Specification](https://frictionlessdata.io/specs/)
- support the Frictionless Data [Guides](https://frictionlessdata.io/guides/)
- demonstrate Frictionless Data [Patterns](https://frictionlessdata.io/specs/patterns/) - ways to solve common problems that are not (yet) in the specification

For a comprehensive suite of Data Packages for testing, please see the `/datasets/` directory of [testsuite-py](https://github.com/frictionlessdata/testsuite-py).
The Guides are a great place to [start to learn](https://frictionlessdata.io/guides/). Alternatively, dive in and open a data package directory, view the `README.md`, then view the `datapackage.json` to understand the data.

*Warning: some data packages are out-of-date. (See [Issue #4](https://github.com/frictionlessdata/example-data-packages/issues/4))*

The `datapackage.json` will describe the [data package](https://frictionlessdata.io/specs/data-package/) as a whole, and one or more [data resources](https://frictionlessdata.io/specs/data-resource/) (each optionally with a [schema](https://frictionlessdata.io/specs/data-resource/#resource-schemas) or a [view](https://frictionlessdata.io/specs/views/)).

If all the data is tabular (i.e. [CSV files](https://frictionlessdata.io/guides/csv/)), then it will be described as a [tabular data package](https://frictionlessdata.io/specs/tabular-data-package/) with one or more [tabular data resources](https://frictionlessdata.io/specs/tabular-data-package/) each with a [table schema](https://frictionlessdata.io/specs/table-schema/) and, if needed, a [CSV dialect](https://frictionlessdata.io/specs/csv-dialect/).

There are other specialised [profiles](https://frictionlessdata.io/specs/profiles/) that describe different types of data, such as [fiscal data](https://frictionlessdata.io/specs/fiscal-data-package/).

## Data Packages

Each data package is stored in it's own directory:

```
|- data-package-name-1
|- README.md
|- datapackage.json
|- data
|- data.csv
|- ...
```

It contains a:

- `README.md` to explain the provenance of the data
- `datapackage.json` a machine readable file that explains the structure and meaning of the data
- one or more data files, typically grouped in a `data` directory

The data package directory may also contain [other files or sub-directories](https://frictionlessdata.io/specs/data-package/#illustrative-structure). These files may be scripts used to prepare the data package or other related resources.

## Validation

With each commit to the repository, the data package is validated using [goodtables.io](http://goodtables.io/). The validation results for the whole repository are indicated with a badge: ![goodtables.io](https://goodtables.io/badge/github/frictionlessdata/example-data-packages.svg)

*Ideally an individual badge could also be awarded for each data package and displayed in its `README.md` file but this is not possible yet. (See goodtables.io [issue #285](https://github.com/frictionlessdata/goodtables.io/issues/285))*

The validation is controlled by the [goodtables.yaml](https://github.com/frictionlessdata/example-data-packages/blob/master/goodtables.yml) file. It should be configured to test all the data packages in the repository. This can be changed if your working locally to [validate specific data packages](https://github.com/frictionlessdata/goodtables.io/blob/master/docs/goodtables_yml.md).

## Zipped Data Packages

With each commit to the repository, the data package directory is converted into a .zip file so it can be used with [software that supports Frictionless Data ](https://frictionlessdata.io/software/) such as the [Data Curator](http://data-curator.io) app or the [DataPackage.js](https://github.com/frictionlessdata/datapackage-js) library. The zip files are stored in the `zip` directory.

*To do: script needed*

## Formatted JSON

With each commit, the `datapackage.json` file is [formatted to make it easier to read](https://frictionlessdata.io/guides/publish-faq/#alignment). Although this formatting is not needed by computer programs, it makes it easier for you to read the contents of the file.

*To do: script (or linting) needed*

## Resources

A resources directory contains a template `README.md` and example `datapackage.json` snippets.

### `README.md` template

In this repository, each data package must have a `README.md`. The `README.md` should follow [good practices](https://frictionlessdata.io/guides/publish-faq/#readme).

*To Do. Update good practices to include ideas from [Data Curator](https://github.com/ODIQueensland/data-curator/blob/develop/test/features/tools/sample-provenance-information.md) and [Gitter](https://gitter.im/frictionlessdata/chat?at=5a58aff461a861c92704fb38)*

### `datapackage.json` snippets

JSON snippets provide a fragment of a `datapackage.json` file to help you learn about that specific property or cut and paste into your own data package. E.g. `licenses.json` could include JSON for each [recommended Open Definition conformant license](http://opendefinition.org/licenses/#conformant-licenses).

## Repository Structure

```
|
|- data-package-name-1
| |- README.md
| |- datapackage.json
| |- data
| |- data.csv
| |- data.geojson
| |- ...
|
|- data-package-name-2
| |- etc.
|
|- resources
| |- README-template.md
| |- licenses.json
| |- contributors.json
| |- dialect.json
| |- ...
|
|- zip
| |- data-package-name-1.zip
| |- data-package-name-2.zip
|
|
|- goodtables.yaml
|- README.md

```

## Contributing

We value all types of contributions:
- documentation
- data packages
- [issues](https://github.com/frictionlessdata/example-data-packages/issues) and [requests](https://github.com/frictionlessdata/example-data-packages/issues)
- code

We thank the generous [contributors](https://github.com/your/project/contributors) to this project.

To join them, please read [`CONTRIBUTING.md`](.github/CONTRIBUTING.md) for details on our code of conduct and how to submit a pull request. Each contributed data package should be licensed as openly as possible.

*`Contributing.md` will include the requirement to apply [data package version](https://frictionlessdata.io/specs/patterns/#data-package-versions).*

## Licenses

Data Packages in this project are licensed as specified in each individual `datapackage.json` file. If a license is not specified, it is provided under a [CC0 1.0 Universal](https://creativecommons.org/publicdomain/zero/1.0/) Public Domain Dedication.

The code in this project, unless otherwise noted, is licensed as described in [`LICENSE.md`](LICENSE.md).
8 changes: 8 additions & 0 deletions geo-lat-lon-as-numbers/Readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
A single csv file with location defined by two `number` columns:

- Latitude
- Longitude

This package supports the [example](https://frictionlessdata.io/guides/point-location-data/#4.-numbers-with-constraints) given in the Frictionless Data [Point location data in CSV files](https://frictionlessdata.io/guides/point-location-data/) guide.

You can adjust the `minimum` and `maximum` values in `constraints` to define a minimum bounding rectangle for the data.
4 changes: 4 additions & 0 deletions geo-lat-lon-as-numbers/data/office-locations.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Office,Lat,Lon
Dalby,-27.1944,151.2660
Kingaroy,-26.533333,151.833333
Toowoomba,-27.566667,151.95
53 changes: 53 additions & 0 deletions geo-lat-lon-as-numbers/datapackage.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
{
"name": "office-locations",
"title": "Office Locations",
"description": "Office Locations as a Latitude, Longitude columns",
"profile": "tabular-data-package",
"licenses": [{
"id": "CC0-1.0",
"title": "CC0 1.0",
"url": "https://creativecommons.org/publicdomain/zero/1.0/"
}],
"resources": [{
"name": "offices",
"path": "data/office-locations.csv",
"profile": "tabular-data-resource",
"primaryKeys": ["office"],
"format": "csv",
"mediatype": "text/csv",
"encoding": "utf-8",
"licenses": [{
"id": "CC0-1.0",
"title": "CC0 1.0",
"url": "https://creativecommons.org/publicdomain/zero/1.0/"
}],
"schema": {
"fields": [{
"name": "Office",
"type": "string",
"format": "default"
}, {
"name": "Lat",
"title": "Latitude",
"type": "number",
"format": "default",
"constraints": {
"required": true,
"minimum": "-90",
"maximum": "90"
}
}, {
"name": "Lon",
"title": "Longitude",
"type": "number",
"format": "default",
"constraints": {
"required": true,
"minimum": "-180",
"maximum": "180"
}
}],
"missingValues": [""]
}
}]
}
4 changes: 4 additions & 0 deletions geo-location-fk/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
A single csv file with location defined by a String and Foreign key reference to well-known place-name

This package supports the [example](https://frictionlessdata.io/guides/point-location-data/#5.-string-and-foreign-key-reference-to-well-known-place-name
) given in the Frictionless Data [Point location data in CSV files](https://frictionlessdata.io/guides/point-location-data/) guide.
4 changes: 4 additions & 0 deletions geo-location-fk/data/gazetteer.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
city-or-town,location
Dalby,"{""lon"":151.2660, ""lat"": -27.1944}"
Kingaroy,"{""lon"": 151.833333, ""lat"": -26.533333}"
Toowoomba,"{""lon"": 151.95, ""lat"": -27.566667}"
4 changes: 4 additions & 0 deletions geo-location-fk/data/offices.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Office,Town
Dalby Drop In Centre,Dalby
Peanuts Hot Desking,Kingaroy
Darling Desking,Toowoomba
78 changes: 78 additions & 0 deletions geo-location-fk/datapackage.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
{
"name": "geo-location-fk",
"profile": "tabular-data-package",
"licenses": [{
"id": "CC0-1.0",
"title": "CC0 1.0",
"url": "https://creativecommons.org/publicdomain/zero/1.0/"
}],
"resources": [{
"path": "data/offices.csv",
"name": "offices",
"title": "Offices",
"description": "Hot Desk Offices",
"profile": "tabular-data-resource",
"encoding": "utf-8",
"format": "csv",
"mediatype": "text/csv",
"licenses": [{
"id": "CC0-1.0",
"title": "CC0 1.0",
"url": "https://creativecommons.org/publicdomain/zero/1.0/"
}],
"schema": {
"fields": [{
"name": "Office",
"title": "Office Name",
"description": "Hot Desk Offices",
"type": "string",
"format": "default",
"constraints": {
"required": true
}
}, {
"name": "Town",
"type": "string",
"format": "default",
"constraints": {
"required": true
}
}]
},
"primaryKeys": ["Office"],
"foreignKeys": [{
"fields": ["Town"],
"reference": {
"resource": "gazetteer",
"fields": ["city-or-town"]
}
}]
}, {
"path": "data/gazetteer.csv",
"name": "gazetteer",
"profile": "tabular-data-resource",
"encoding": "utf-8",
"format": "csv",
"mediatype": "text/csv",
"licenses": [{
"name": "CC-BY-4.0",
"title": "Creative Commons Attribution 4.0",
"path": "https://creativecommons.org/licenses/by/4.0/"
}],
"schema": {
"fields": [{
"name": "city-or-town",
"type": "string",
"format": "default",
"constraints": {
"required": true
}
}, {
"name": "location",
"type": "geopoint",
"format": "object"
}],
"primaryKeys": ["city-or-town"]
}
}]
}
6 changes: 6 additions & 0 deletions geo-location-uri/Readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
A single csv file with location defined by:

- `"type": "string"`
- `"format": "uri"`

This package supports the [example](https://frictionlessdata.io/guides/point-location-data/#6.-use-a-uniform-resource-identifier-to-reference-a-location) given in the Frictionless Data [Point location data in CSV files](https://frictionlessdata.io/guides/point-location-data/) guide.
4 changes: 4 additions & 0 deletions geo-location-uri/data/office-locations.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Office,Location uri
Dalby,https://nominatim.openstreetmap.org/details.php?place_id=114595
Kingaroy,https://nominatim.openstreetmap.org/details.php?place_id=3529222
Toowoomba,https://nominatim.openstreetmap.org/details.php?place_id=110709
Loading