Skip to content

Setting up your HPC environment

Jump to bottom
Adrien Le Guillou edited this page May 21, 2024 · 16 revisions

Getting familiar with the HPC interface

When you connected to the HPC, you will almost exclusively work with a Command Line Interface (CLI). In order to get familiar with it we suggest you read the following:

A few tips

  • Pressing the Up and Down arrow at the command line will navigate through the command you previously typed
  • Pressing the Tab key at the command line triggers auto completion of command or file paths. Pressing it twice lists the possible completions
  • Pressing Ctrl+R starts the command history search: type some character and the previous matching command will show up. Press Ctrl+R again to cycle the potential matches

Connecting to HPC with SSH

You can use SSH keys to connect to the Emory HPC without the need to enter your password every time you connect. The steps here are first to generate a public/private ssh key pair and second to copy the key to the HPC system. There are two general approaches, the latter of which seems to work for Windows users better.

Approach 1 with ssh-copy-id (Mac/Linux)

ssh-keygen is used to generate the key and ssh-copy-id is used to copy it to the Emory HPC. For example:

ssh-keygen -f ~/.ssh/emory-hpc-key

will create a new key file emory-hpc-key in your ~/.ssh folder. When running this command, you do not need a passphrase for this file (hit enter for no passphrase).

For the second step, you would copy that key over to the Emory HPC with

ssh-copy-id -i ~/.ssh/emory-hpc-key [email protected]

Approach 2 with manual copying (Windows)

First, remember to run set DISPLAY= on the terminal you will be using.

ssh-keygen is used to generate the key and ssh-copy-id is used to copy it to the Emory HPC. For example:

ssh-keygen -f %homepath%/.ssh/emory-hpc-key

Skip the passphrase by pressing enter. After running the above, there should be two new files in that directory: %homepath%\.ssh\emory-hpc-key and %homepath%\.ssh\emory-hpc-key.pub(%homepath% gets translated by windows to your user's home directory).

Once your key is saved, you need to copy the public key to the HPC using this command on your local terminal window (this will require you to enter your HPC password):

scp %homepath%\.ssh\emory-hpc-key.pub [email protected]:.

Finally, log in the HPC as before. In that HPC terminal, you will need to run two additinal commands:

cat emory-hpc-key.pub >> .ssh/authorized_keys
chmod 0600 .ssh/authorized_keys

Exit the HPC and try logging in again to see that it worked.

ssh <user>@clogin01.sph.emory.edu -i .ssh/emory-hpc-key

The -i argument to ssh specifies which rsa key to use

Setting up a SSH config File (Optional)

This is an optional but recommended step. It reduces the amount of typing/code needed to log in to the HPC and transfer files between computers.

Using a config file with your ssh further automate the process for logging in to the system. On your local computer, navigate to the %homepath%/.ssh folder and see if you already have a file there named config. If not, create one and put this block in it. The easiest way is to make the config file in Rstudio (File > New File > Text file) and save it to the "%homepath%/.ssh" folder.

Host sph
    HostName clogin01.sph.emory.edu
    User NETID
    IdentityFile ~/.ssh/emory-hpc-key
    ControlPath ~/.ssh/master-%r@%h:%p
    ControlMaster auto
    ControlPersist no
    ForwardX11 no
    XAuthLocation /opt/X11/bin/xauth
    Compression yes

Once complete, you can now connect to the HPC with only:

ssh sph

Shell Aliases

On linux, aliases are helpful shortcuts to running common commands. On the HPC, you can see a current list of your aliases by just typing alias.

Some helpful aliases for monitoring your HPC jobs and the state of the HPC nodes are:

alias myq='squeue -u NETID'
alias noderep1='sinfo -o '\''%n %e %m %a %c %C'\'' '
alias noderep2='sinfo --Node --long'
alias noderep3='scontrol show node'

Anything that you type interactively on a repeated basis might be a good candidate for an alias.

To create a new alias, you just need to edit the .bashrc file in your home directory. You can do that with the text editor Vim. To edit .bashrc, just type:

vim .bashrc

This will open the file in view mode. To edit the file, type i to enter interactive mode. Use the cursor to edit the file at the bottom of the file (after any existing code you see in the file).

Continue editing the file as needed. When done editing, hit your escape key, then type :wq to save and quit. You can see any changes to your alias file with cat .bashrc.

The next time you log in to the HPC, these will be loaded. You can log out and log in again to check. If loaded correctly, they should should up when you type alias (this will show a listing of all the currently defined aliases).

Clone this wiki locally