Bash script to simplify/automate user setup on hpc

[rlskoeser]

User setup for #15

Based on the instructions we were drafting in the google doc.

I've copied this script to della. you should be able to run it by sshing to della and then entering:

/scratch/gpfs/rkoeser/htr2hpc_setup/user_setup.sh

If you want to test using it to create a new conda env (since you already have an htr2hpc env), you could copy it into your space and edit the name of the conda env it's going to create by modifying the conda_env_name variable on line 26.

[cmroughan]

Ran another test, this time including the conda environment step. Overall things seem to work, albeit with some errors and warnings in the conda process.

• 🟢 handle ssh key
• 🟡 create conda environment
• 🟢 create working directory

More details below.

[cmroughan]

I do get one error here:

['/home/croughan/.conda/envs/htr2hpc_TEST/bin/python', '-m', 'pip', 'install', '-U', '-r', '/tmp/condaenv.z0lgf55i.requirements.txt', '--exists-action=b']
Pip subprocess output:
Processing /scratch/gpfs/croughan/.conda/envs/htr2hpc_TEST

Pip subprocess error:
ERROR: file:///. (from -r /tmp/condaenv.z0lgf55i.requirements.txt (line 2)) does not appear to be a Python project: neither 'setup.py' nor 'pyproject.toml' found.

failed

CondaEnvException: Pip failed

But outside the failure to handle that last line of environment_cuda.yml, line 33 of the script seems to work.

[cmroughan]

I get a CondaError: Run 'conda init' before 'conda activate' warning. This does not cause issues with activating the environment afterwards, but during this script's runtime I think it is preventing these pip installations from working -- I still have the old versions of these libraries, not the version numbers to be installed here.

[cmroughan]

Otherwise, I'll want to remember to mention to people that this script will take a few minutes to run (it took 5 mins in this test) and not to exit prematurely. We could have an echo statement in the script that mentions this, but I imagine it will primarily be part of the instructions I provide to new users on first time setup.

[rlskoeser]

@cmroughan thanks for your careful testing. Do we need to make some adjustments for the errors you found? Echoing a reminder that it will take a little while to install all the dependencies seems like an easy and reasonable thing to add.

[cmroughan]

Further testing revealed that the second issue (CondaError: Run 'conda init' before 'conda activate') should not actually be something our users will likely run into -- it seems I was only getting it because I had previously run module load... in the session. When our script is run right when a user logs in, before they load any environments, we don't get that error and the subsequent pip installations work as they should.

So I think no need to make any adjustments for the above errors. I think we can just toss in a note about installation time and leave it at that!

[rlskoeser]

@cmroughan so are we ready to merge? or do you want to suggest a message to output first?