Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update installation docs #154

Merged
merged 4 commits into from
Sep 30, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,8 @@

## Version 1.0.2

Update documentation for release, new installation instructions etc.

## Version 1.0.0 (Release)

- Update default NERSC site to
Expand Down
1 change: 1 addition & 0 deletions docs/source/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,7 @@ them.

dev_notes_spin
dev_notes_database
installation_locally

.. toctree::
:maxdepth: 2
Expand Down
103 changes: 37 additions & 66 deletions docs/source/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -3,76 +3,44 @@
Installation
============

Currently the DESC data registry software can only be used at NERSC (i.e.,
PerlMutter).
Currently the DESC ``dataregistry`` database is only accessible through NERSC
(i.e., PerlMutter).

Main installation steps
-----------------------
Using the ``dataregistry`` at NERSC
------------------------------------

When installing the ``dataregistry`` package, it is recommended to work within
your own Conda or Python virtual environment.
The ``dataregistry`` package is readily available as part of the
``desc-python-bleed`` environment (see `here
<https://confluence.slac.stanford.edu/display/LSSTDESC/Getting+Started+with+Anaconda+Python+at+NERSC>`__
for details about the *Conda* environments available at NERSC). Therefore
before getting started, make sure to activate the ``desc-python-bleed``
environment from the command line, or, when working at the NERSC JupyterHub,
select the ``desc-python-bleed`` kernel.

Creating a Conda environment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~
If you wish to install the ``dataregistry`` package yourself, see the
instructions :ref:`here <local-installation>`.

You can make a new Conda environment via
Access accounts
---------------

.. code-block:: bash

module load conda/Mambaforge-22.11.1-4
conda create -p ./datareg_env psycopg2

where ``./datareg_env`` is the path where the environment will be installed
(change this as required). To activate the environment do

.. code-block:: bash

conda activate <path to your env>

Creating a Python venv
~~~~~~~~~~~~~~~~~~~~~~

or, you can work within a Python virtual environment via

.. code-block:: bash

module load python/3.10
python3 -m venv ./datareg_env

where ``./datareg_env`` is the path where the environment will be installed
(change this as required). To activate the environment do

.. code-block:: bash
Authentication to the DESC ``dataregistry`` database works through two primary
group accounts, ``reg_reader`` and ``reg_writer``. These accounts have
different privileges depending on the database schema you are connected to.
Both ``reg_reader`` and ``reg_writer`` have query (read) access to the primary
working schema (``lsst_desc_working``), but only ``reg_writer`` has write
access to register new entries in the database.

source <path to your env>/bin/activate
Both ``reg_reader`` and ``reg_writer`` accounts have read and write access to
the tutorial schemas (used for the tutorial notebooks, i.e., the
``tutorial_working`` and ``tutorial_production`` schemas).

Note the specific version of Python used above (``3.10``) is only an example,
the ``dataregistry`` package is supported on Python versions ``>3.7``.
Neither ``reg_reader`` or ``reg_writer`` can write to the main production
schema (``lsst_desc_production``), however they both have read access. If you
need to register production entries, please consult one of the data registry
admins.

Installing the ``dataregistry`` package
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Now we can install the DESC data registry software. First clone the GitHub
repository

.. code-block:: bash

git clone https://github.com/LSSTDESC/dataregistry.git

then, navigate to the ``dataregistry`` directory and install via *pip* using

.. code-block:: bash

python3 -m pip install .

You can test to see if the ``dataregistry`` package has installed successfully
by typing

.. code-block:: bash

python3 -c "import dataregistry; print(dataregistry.__version__)"

If you see the current package version printed to the console, success!
Depending on which account you have access to, you will need to perform a
one-time-setup to authenticate, detailed below.

.. _one-time-setup:

Expand All @@ -90,19 +58,22 @@ entry

.. code-block:: yaml

sqlalchemy.url : postgresql://reg_writer@data-registry-dev-loadbalancer.jrb-test.development.svc.spin.nersc.org:5432/desc_data_registry
sqlalchemy.url : postgresql://<username>@dataregistry-release-test-loadbalancer.mcalpine-test.development.svc.spin.nersc.org:5432/desc_data_registry

where ``<username>`` should either be ``reg_writer`` or ``reg_reader``,
depending on what account you have access to.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be good to add a note here (or somewhere) something like this:
Note: reg_writer cannot write to production (schema lsst_desc_production). There is another more privileged account for that. If you need to register production entries, please consult one of the dataregistry admins. Both reg_reader and reg_writer can write to the tutorial version of production (schema tutorial_production).


Then (if you don't have one already), create a file named ``~/.pgpass`` in your
``$HOME`` directory, and append the entry

.. code-block:: bash

# data registry db
data-registry-dev-loadbalancer.jrb-test.development.svc.spin.nersc.org:5432:desc_data_registry:reg_writer:<password>
dataregistry-release-test-loadbalancer.mcalpine-test.development.svc.spin.nersc.org:5432:desc_data_registry:<username>:<password>

where ``<password>`` is provided on demand by the DESC data registry admins. As
a final step, the ``.pgpass`` file must only be readable by you, which you
can ensure by doing
a final step, the ``.pgpass`` file must only be readable by you, which you can
ensure by doing

.. code-block:: bash

Expand Down
78 changes: 78 additions & 0 deletions docs/source/installation_locally.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,78 @@
Local installation
==================

.. _local-installation:

When installing the ``dataregistry`` package locally yourself, it is
recommended to work within your own Conda or Python virtual environment.

Creating a Conda environment
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

You can make a new Conda environment via

.. code-block:: bash

module load conda/Mambaforge-22.11.1-4
conda create -p ./datareg_env psycopg2

where ``./datareg_env`` is the path where the environment will be installed
(change this as required). To activate the environment do

.. code-block:: bash

conda activate <path to your env>

Creating a Python venv
~~~~~~~~~~~~~~~~~~~~~~

or, you can work within a Python virtual environment via

.. code-block:: bash

module load python/3.10
python3 -m venv ./datareg_env

where ``./datareg_env`` is the path where the environment will be installed
(change this as required). To activate the environment do

.. code-block:: bash

source <path to your env>/bin/activate

Note the specific version of Python used above (``3.10``) is only an example,
the ``dataregistry`` package is supported on Python versions ``>=3.9``.

Installing the ``dataregistry`` package
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The easiest way to install the ``dataregistry`` package is via `PyPi
<https://pypi.org/project/lsstdesc-dataregistry/>`__, i.e.,

.. code-block:: bash

pip install lsstdesc-dataregistry

Install from source
~~~~~~~~~~~~~~~~~~~

To install from source, first clone the GitHub repository

.. code-block:: bash

git clone https://github.com/LSSTDESC/dataregistry.git

then, navigate to the ``dataregistry`` directory and install via *pip* using

.. code-block:: bash

python3 -m pip install .

You can test to see if the ``dataregistry`` package has installed successfully
by typing

.. code-block:: bash

python3 -c "import dataregistry; print(dataregistry.__version__)"

If you see the current package version printed to the console, success!
7 changes: 3 additions & 4 deletions docs/source/tutorial_setup.rst
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,8 @@ This section has some tutorials on how to get started using the

Before we begin, make sure that you have the ``dataregistry`` package installed
and available to you at NERSC (installation instructions :ref:`here
<installation>`). This is generally
simplest within a Conda or Python virtual environment (note that the
``dataregistry`` is not yet part of the `desc-python` environment).
<installation>`). This is generally simplest working within the
``desc-python-bleed`` Conda environment.

Also, make sure you have completed the one-time-setup for the data registry
(more details :ref:`here <one-time-setup>`),
Expand All @@ -30,6 +29,6 @@ reference materials, however they can also be used interactively.
To do this at NERSC, first clone the ``dataregistry`` repository to your home
space, then load the notebooks through the NERSC JupyterHub portal. Note that
you will need to be operating within a kernel that has the ``dataregistry``
package installed (see `here
package installed, i.e., ``desc-python-bleed`` (see `here
<https://docs.nersc.gov/services/jupyter/how-to-guides/>`__ how to register
your own Python environments as Jupyter kernels).
2 changes: 1 addition & 1 deletion src/dataregistry/_version.py
Original file line number Diff line number Diff line change
@@ -1 +1 @@
__version__ = "1.0.1"
__version__ = "1.0.2"
Loading