Skip to content

Commit

Permalink
Deployed 893d1b1 with MkDocs version: 1.1.2
Browse files Browse the repository at this point in the history
  • Loading branch information
Unknown committed Nov 25, 2024
1 parent ed4c996 commit aefdf1d
Show file tree
Hide file tree
Showing 3 changed files with 18 additions and 11 deletions.
27 changes: 17 additions & 10 deletions misc/pelican/index.html
Original file line number Diff line number Diff line change
Expand Up @@ -448,7 +448,14 @@
Upgrade of the OSDF origin to Pelican for Collaborations
</a>

<nav class="md-nav" aria-label="Upgrade of the OSDF origin to Pelican for Collaborations">
</li>

<li class="md-nav__item">
<a href="#htcondor-plugin-users-no-changes-are-needed-in-your-submission-scripts" class="md-nav__link">
HTCondor Plugin Users: No changes are needed in your submission scripts.
</a>

<nav class="md-nav" aria-label="HTCondor Plugin Users: No changes are needed in your submission scripts.">
<ul class="md-nav__list">

<li class="md-nav__item">
Expand Down Expand Up @@ -488,20 +495,20 @@ <h1>Pelican</h1>
<h2 id="upgrade-of-the-osdf-origin-to-pelican-for-collaborations">Upgrade of the OSDF origin to Pelican for Collaborations<a class="headerlink" href="#upgrade-of-the-osdf-origin-to-pelican-for-collaborations" title="Permanent link">&para;</a></h2>
<p>The OSG Collab AP had an <strong>OSDF</strong> door deployed on the access point (ap23.uc.osg-htc.org) that provided users with authenticated access to a Ceph cluster, that provides high capacity storage to shared project directories. HTCondor jobs running in the OSPool at remote <strong>Execution Points (EPs)</strong> (remote worker nodes) can access the filesystem either via a client tool or via an HTCondor plugin that is invoked in their submit scripts. The storage is also mounted on the AP at /ospool/uc-shared/project. </p>
<p>On <strong>11/21/2024</strong>, OSG/PATh staff migrated the OSDF door from the OSG Collab AP to a separate infrastructure to allow upgrading the origin to the Pelican Platform (https://pelicanplatform.org/) and provide shared project access to users at other APs (ap20.uc.osg-htc.org and ap21.uc.osg-htc.org).</p>
<p>The migration should have been transparent to the OSG Collab AP. If using the HTCondor plugin, no changes are needed in your submission scripts. For reference, the general purpose documentation found here (https://portal.osg-htc.org/documentation/htc_workloads/managing_data/osdf/) is also applicable to the users of the OSG Collab AP. It describes using the HTCondor plugin to move data to and from the OSDF/Pelican Origin. </p>
<p>In a nutshell:
1. Include the following in your submit script for an OSPool job to transfer a file from your project directory at the origin to the EP environment:</p>
<div class="codehilite"><pre><span></span><code>transfer_input_files = osdf:///ospool/uc-shared/project/&lt;your_project&gt;/&lt;file&gt;
</code></pre></div>

<p>The migration was designed to be transparent to users of the OSG Collab AP.</p>
<h2 id="htcondor-plugin-users-no-changes-are-needed-in-your-submission-scripts">HTCondor Plugin Users: No changes are needed in your submission scripts.<a class="headerlink" href="#htcondor-plugin-users-no-changes-are-needed-in-your-submission-scripts" title="Permanent link">&para;</a></h2>
<p>The <a href="https://portal.osg-htc.org/documentation/htc_workloads/managing_data/osdf/">OSDF documentation</a> remains applicable and describes how to use the HTCondor plugin to move data to and from the OSDF/Pelican origin.</p>
<p>Here are key examples for transferring files using the HTCondor plugin:</p>
<ol>
<li>
<p>Include the following in your submit script for an OSPool job to transfer a file to your project directory at the origin from an EP enviroment:</p>
<p>Transfer a file from the project directory to the EP environment:</p>
<p>transfer_input_files = osdf:///ospool/uc-shared/project/<your_project>/<file>
5. Transfer a file to the project directory from the EP environment:</p>
<p>OSDF_LOCATION = osdf:///ospool/uc-shared/project/<your_project>/<file>
transfer_input_files = $(OSDF_LOCATION)/<file>
Note: The Pelican platform uses federated URLs for the origin, but the prefix osdf:// remains unchanged. After the upgrade, osdf:// maps to pelican://osg-htc.org/.</p>
transfer_input_files = $(OSDF_LOCATION)/<file></p>
</li>
</ol>
<p>Note: The Pelican platform uses federated URLs for the origin, but the prefix osdf:// remains unchanged. After the upgrade, osdf:// maps to pelican://osg-htc.org/.</p>
<h3 id="transition-from-stashcp-to-pelican">Transition from Stashcp to Pelican<a class="headerlink" href="#transition-from-stashcp-to-pelican" title="Permanent link">&para;</a></h3>
<p>If your runtime scripts at the EP rely on the stashcp tool, it will continue to work (version 6.12 or higher), except for recursive access (-r flag). However, we recommend transitioning to the Pelican client for improved compatibility and features.</p>
<p>Command Comparison</p>
Expand Down
2 changes: 1 addition & 1 deletion search/search_index.json

Large diffs are not rendered by default.

Binary file modified sitemap.xml.gz
Binary file not shown.

0 comments on commit aefdf1d

Please sign in to comment.