Skip to content

Commit

Permalink
Update some docs
Browse files Browse the repository at this point in the history
  • Loading branch information
stuartmcalpine committed Sep 16, 2024
1 parent c6a04d2 commit 031dd48
Show file tree
Hide file tree
Showing 3 changed files with 30 additions and 26 deletions.
12 changes: 12 additions & 0 deletions docs/source/reference_python.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,18 @@ It connects the user to the database, and serves as a wrapper to both the

.. automethod:: dataregistry.registrar.dataset.DatasetTable.register

.. automethod:: dataregistry.registrar.dataset.DatasetTable.replace

.. automethod:: dataregistry.registrar.dataset.DatasetTable.modify

.. automethod:: dataregistry.registrar.dataset.DatasetTable.delete

.. automethod:: dataregistry.registrar.dataset.DatasetTable.add_keywords

.. automethod:: dataregistry.registrar.dataset.DatasetTable.get_modifiable_columns

.. automethod:: dataregistry.registrar.dataset.DatasetTable.get_keywords

.. automethod:: dataregistry.registrar.execution.ExecutionTable.register

.. automethod:: dataregistry.registrar.dataset_alias.DatasetAliasTable.register
32 changes: 11 additions & 21 deletions docs/source/tutorial_cli.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,8 @@ Typing
will list all the metadata properties that can be associated with a dataset
during registration. As when registering datasets using the ``dataregistry``
package, the ``relative_path`` and ``version`` string properties are mandatory,
which will always be the first two parameters passed to the ``dregs register
package, the dataset ``name`` and ``version`` properties are mandatory, which
will always be the first two parameters passed to the ``dregs register
dataset`` command respectively.

For example, say I have produced some data from my latest DESC publication that
Expand All @@ -59,11 +59,9 @@ would run the CLI as follows:
--description "Data from my_paper_dataset"
This will recursively copy the ``/some/place/at/nersc/my_paper_dataset/``
directory into the data registry shared space under the relative path
``my_paper_dataset``. As we did not specify a ``--name`` for the dataset, the
``name`` column in the database will automatically be assigned as
``my_paper_dataset`` (and all other properties we did not specify will keep
their default values).
directory into the data registry shared space with the
``name='my_paper_dataset'`` (other non-specified properties will keep their
default values).

Updating a dataset
------------------
Expand All @@ -76,26 +74,18 @@ initial registration, we need to create a new version of the dataset.
.. code-block:: bash
dregs register dataset \
my_paper_dataset_updated \
my_paper_dataset \
patch \
--old-location /some/place/at/nersc/my_paper_dataset_updated/ \
--owner_type project \
--owner "DESC Generic Working Group" \
--description "Data from my_paper_dataset describing bugfix" \
--name my_paper_dataset
Here we associate it with the previous dataset through ``--name
my_paper_dataset``, and tell the data registry to automatically bump the patch
version to ``1.0.1`` by specifying "patch" as the version string (you could
however have entered "1.0.1" here if you prefer).

.. note::
Remember, if the dataset is non-overwritable, the relative paths in the data
registry need to be unique, which is why we could not have the relative path
of the second entry match the first. But for datasets only the ``name``
plus ``version`` has to be unique, which is how we could associate them with
the same ``name`` column.
Here we associate it with the previous dataset through ``name=
my_paper_dataset`` (and making sure we keep the same `owner` and `owner_type`),
and tell the data registry to automatically bump the patch version to ``1.0.1``
by specifying "patch" as the version string (you could however have entered
"1.0.1" here if you prefer).

Querying the data registry
--------------------------
Expand Down
12 changes: 7 additions & 5 deletions docs/source/tutorial_python.rst
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,16 @@ material by viewing the notebooks on GitHub.
1) Getting started with the data registry
-----------------------------------------

**Getting started: Part 1 - Registering datasets**: `Link to notebook <https://github.com/LSSTDESC/dataregistry/blob/main/docs/source/tutorial_notebooks/getting_started_1_register_datasets.ipynb>`__
**Getting started: Part 1 - Registering datasets**: `Link to notebook <https://github.com/LSSTDESC/dataregistry/blob/main/docs/source/tutorial_notebooks/register_datasets.ipynb.ipynb>`__

**Getting started: Part 2 - Simple queries**: `Link to notebook <https://github.com/LSSTDESC/dataregistry/blob/main/docs/source/tutorial_notebooks/getting_started_2_query_datasets.ipynb>`__
**Getting started: Part 2 - A deeper dive into datasets**: `Link to notebook <https://github.com/LSSTDESC/dataregistry/blob/main/docs/source/tutorial_notebooks/datasets_deeper_look.ipynb.ipynb>`__

**Getting started: Part 3 - Pipeline datasets**: `Link to notebook <https://github.com/LSSTDESC/dataregistry/blob/main/docs/source/tutorial_notebooks/getting_started_3_pipelines.ipynb>`__
**Getting started: Part 3 - Simple queries**: `Link to notebook <https://github.com/LSSTDESC/dataregistry/blob/main/docs/source/tutorial_notebooks/query_datasets.ipynb.ipynb>`__

2) The production schema
------------------------
**Getting started: Part 4 - Pipeline datasets**: `Link to notebook <https://github.com/LSSTDESC/dataregistry/blob/main/docs/source/tutorial_notebooks/pipelines.ipynb>`__

2) Extra material
-----------------

**The prduction schema**: `Link to notebook <https://github.com/LSSTDESC/dataregistry/blob/main/docs/source/tutorial_notebooks/production_schema.ipynb>`__

Expand Down

0 comments on commit 031dd48

Please sign in to comment.