From 47ece732d0d1ba68d30a3df0d5667d1927c579d4 Mon Sep 17 00:00:00 2001 From: Stuart McAlpine Date: Mon, 30 Sep 2024 14:46:37 +0200 Subject: [PATCH 1/4] Update installation docs --- docs/source/index.rst | 1 + docs/source/installation.rst | 91 ++++++++-------------------------- docs/source/tutorial_setup.rst | 7 ++- 3 files changed, 24 insertions(+), 75 deletions(-) diff --git a/docs/source/index.rst b/docs/source/index.rst index 99a10d90..0eabf1d4 100644 --- a/docs/source/index.rst +++ b/docs/source/index.rst @@ -59,6 +59,7 @@ them. dev_notes_spin dev_notes_database + installation_locally .. toctree:: :maxdepth: 2 diff --git a/docs/source/installation.rst b/docs/source/installation.rst index 385974c3..5e3a97b1 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -3,76 +3,22 @@ Installation ============ -Currently the DESC data registry software can only be used at NERSC (i.e., -PerlMutter). +Currently the DESC ``dataregistry`` database is only accessible through NERSC +(i.e., PerlMutter). -Main installation steps ------------------------ +Using the ``dataregistry`` at NERSC +------------------------------------ -When installing the ``dataregistry`` package, it is recommended to work within -your own Conda or Python virtual environment. +The ``dataregistry`` package is readily available as part of the +``desc-python-bleed`` environment (see `here +`__ +for details about the *Conda* environments available at NERSC). Therefore +before getting started, make sure to activate the ``desc-python-bleed`` +environment from the command line, or, when working at the NERSC JupyterHub, +select the ``desc-python-bleed`` kernel. -Creating a Conda environment -~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -You can make a new Conda environment via - -.. code-block:: bash - - module load conda/Mambaforge-22.11.1-4 - conda create -p ./datareg_env psycopg2 - -where ``./datareg_env`` is the path where the environment will be installed -(change this as required). To activate the environment do - -.. code-block:: bash - - conda activate - -Creating a Python venv -~~~~~~~~~~~~~~~~~~~~~~ - -or, you can work within a Python virtual environment via - -.. code-block:: bash - - module load python/3.10 - python3 -m venv ./datareg_env - -where ``./datareg_env`` is the path where the environment will be installed -(change this as required). To activate the environment do - -.. code-block:: bash - - source /bin/activate - -Note the specific version of Python used above (``3.10``) is only an example, -the ``dataregistry`` package is supported on Python versions ``>3.7``. - -Installing the ``dataregistry`` package -~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ - -Now we can install the DESC data registry software. First clone the GitHub -repository - -.. code-block:: bash - - git clone https://github.com/LSSTDESC/dataregistry.git - -then, navigate to the ``dataregistry`` directory and install via *pip* using - -.. code-block:: bash - - python3 -m pip install . - -You can test to see if the ``dataregistry`` package has installed successfully -by typing - -.. code-block:: bash - - python3 -c "import dataregistry; print(dataregistry.__version__)" - -If you see the current package version printed to the console, success! +If you wish to install the ``dataregistry`` package yourself, see the +instructions :ref:`here `. .. _one-time-setup: @@ -90,7 +36,10 @@ entry .. code-block:: yaml - sqlalchemy.url : postgresql://reg_writer@data-registry-dev-loadbalancer.jrb-test.development.svc.spin.nersc.org:5432/desc_data_registry + sqlalchemy.url : postgresql://@dataregistry-release-test-loadbalancer.mcalpine-test.development.svc.spin.nersc.org:5432/desc_data_registry + +where ```` should either be ``reg_writer`` or ``reg_reader``, +depending on what account you have access to. Then (if you don't have one already), create a file named ``~/.pgpass`` in your ``$HOME`` directory, and append the entry @@ -98,11 +47,11 @@ Then (if you don't have one already), create a file named ``~/.pgpass`` in your .. code-block:: bash # data registry db - data-registry-dev-loadbalancer.jrb-test.development.svc.spin.nersc.org:5432:desc_data_registry:reg_writer: + dataregistry-release-test-loadbalancer.mcalpine-test.development.svc.spin.nersc.org:5432:desc_data_registry:: where ```` is provided on demand by the DESC data registry admins. As -a final step, the ``.pgpass`` file must only be readable by you, which you -can ensure by doing +a final step, the ``.pgpass`` file must only be readable by you, which you can +ensure by doing .. code-block:: bash diff --git a/docs/source/tutorial_setup.rst b/docs/source/tutorial_setup.rst index fd7ddf7e..abc6f3ca 100644 --- a/docs/source/tutorial_setup.rst +++ b/docs/source/tutorial_setup.rst @@ -6,9 +6,8 @@ This section has some tutorials on how to get started using the Before we begin, make sure that you have the ``dataregistry`` package installed and available to you at NERSC (installation instructions :ref:`here -`). This is generally -simplest within a Conda or Python virtual environment (note that the -``dataregistry`` is not yet part of the `desc-python` environment). +`). This is generally simplest working within the +``desc-python-bleed`` Conda environment. Also, make sure you have completed the one-time-setup for the data registry (more details :ref:`here `), @@ -30,6 +29,6 @@ reference materials, however they can also be used interactively. To do this at NERSC, first clone the ``dataregistry`` repository to your home space, then load the notebooks through the NERSC JupyterHub portal. Note that you will need to be operating within a kernel that has the ``dataregistry`` -package installed (see `here +package installed, i.e., ``desc-python-bleed`` (see `here `__ how to register your own Python environments as Jupyter kernels). From b563d97d32febde0214a7378f046e5691eae48f6 Mon Sep 17 00:00:00 2001 From: Stuart McAlpine Date: Mon, 30 Sep 2024 19:27:47 +0200 Subject: [PATCH 2/4] Add install docs --- docs/source/installation_locally.rst | 78 ++++++++++++++++++++++++++++ 1 file changed, 78 insertions(+) create mode 100644 docs/source/installation_locally.rst diff --git a/docs/source/installation_locally.rst b/docs/source/installation_locally.rst new file mode 100644 index 00000000..c0d28996 --- /dev/null +++ b/docs/source/installation_locally.rst @@ -0,0 +1,78 @@ +Local installation +================== + +.. _local-installation: + +When installing the ``dataregistry`` package locally yourself, it is +recommended to work within your own Conda or Python virtual environment. + +Creating a Conda environment +~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +You can make a new Conda environment via + +.. code-block:: bash + + module load conda/Mambaforge-22.11.1-4 + conda create -p ./datareg_env psycopg2 + +where ``./datareg_env`` is the path where the environment will be installed +(change this as required). To activate the environment do + +.. code-block:: bash + + conda activate + +Creating a Python venv +~~~~~~~~~~~~~~~~~~~~~~ + +or, you can work within a Python virtual environment via + +.. code-block:: bash + + module load python/3.10 + python3 -m venv ./datareg_env + +where ``./datareg_env`` is the path where the environment will be installed +(change this as required). To activate the environment do + +.. code-block:: bash + + source /bin/activate + +Note the specific version of Python used above (``3.10``) is only an example, +the ``dataregistry`` package is supported on Python versions ``>=3.9``. + +Installing the ``dataregistry`` package +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + +The easiest way to install the ``dataregistry`` package is via `PyPi +`__, i.e., + +.. code-block:: bash + + pip install lsstdesc-dataregistry + +Install from source +~~~~~~~~~~~~~~~~~~~ + +To install from source, first clone the GitHub repository + +.. code-block:: bash + + git clone https://github.com/LSSTDESC/dataregistry.git + +then, navigate to the ``dataregistry`` directory and install via *pip* using + +.. code-block:: bash + + python3 -m pip install . + +You can test to see if the ``dataregistry`` package has installed successfully +by typing + +.. code-block:: bash + + python3 -c "import dataregistry; print(dataregistry.__version__)" + +If you see the current package version printed to the console, success! From 0f51504a558c229a0fe4083d28f66b4d5cf0f0ba Mon Sep 17 00:00:00 2001 From: Stuart McAlpine Date: Mon, 30 Sep 2024 21:25:38 +0200 Subject: [PATCH 3/4] Add info about accounts to docs --- docs/source/installation.rst | 22 ++++++++++++++++++++++ 1 file changed, 22 insertions(+) diff --git a/docs/source/installation.rst b/docs/source/installation.rst index 5e3a97b1..6b9afbd6 100644 --- a/docs/source/installation.rst +++ b/docs/source/installation.rst @@ -20,6 +20,28 @@ select the ``desc-python-bleed`` kernel. If you wish to install the ``dataregistry`` package yourself, see the instructions :ref:`here `. +Access accounts +--------------- + +Authentication to the DESC ``dataregistry`` database works through two primary +group accounts, ``reg_reader`` and ``reg_writer``. These accounts have +different privileges depending on the database schema you are connected to. +Both ``reg_reader`` and ``reg_writer`` have query (read) access to the primary +working schema (``lsst_desc_working``), but only ``reg_writer`` has write +access to register new entries in the database. + +Both ``reg_reader`` and ``reg_writer`` accounts have read and write access to +the tutorial schemas (used for the tutorial notebooks, i.e., the +``tutorial_working`` and ``tutorial_production`` schemas). + +Neither ``reg_reader`` or ``reg_writer`` can write to the main production +schema (``lsst_desc_production``), however they both have read access. If you +need to register production entries, please consult one of the data registry +admins. + +Depending on which account you have access to, you will need to perform a +one-time-setup to authenticate, detailed below. + .. _one-time-setup: Authenticating with the database From 7accbaa662af2aa952c78ef7b68bb9fca67254f6 Mon Sep 17 00:00:00 2001 From: Stuart McAlpine Date: Mon, 30 Sep 2024 21:26:24 +0200 Subject: [PATCH 4/4] Update version --- CHANGELOG.md | 5 +++++ src/dataregistry/_version.py | 2 +- 2 files changed, 6 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 19412155..2fe55bd2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,8 @@ + +## Version 1.0.2 + +Update documentation for release, new installation instructions etc. + ## Version 1.0.0 (Release) - Update default NERSC site to diff --git a/src/dataregistry/_version.py b/src/dataregistry/_version.py index 5c4105cd..7863915f 100644 --- a/src/dataregistry/_version.py +++ b/src/dataregistry/_version.py @@ -1 +1 @@ -__version__ = "1.0.1" +__version__ = "1.0.2"