Working with files on the HPC

Listing files

The ls function shows all the primary files within a directory. To format that nicely (with file sizes listed more usefully) and also show the hidden files in a directory (those starting with .) use:

ls -lah

To see this in another directory than the current, put the directory name at the end:

ls -lah /projects/epimodel/

Copying files within the HPC

To copy a file from one folder on the HPC to another folder on the HPC, use the cp function:

cp /oldpath/filename /newpath/filename

To move a file, use mv instead.

mv /oldpath/filename /newpath/filename

Note that mv can be used to rename a file too.

Copying files to/from the HPC

The scp function (for secure copy) is used to move files between the HPC and your local computer. This needs to be run in a local terminal window, not logged into the HPC. I recommend having one terminal window logged into the HPC and another window not logged in (meaning it is local). On local, you will run the scp function similarly to the cp function but reference the HPC location.

From HPC to home directory in local:

scp sph:~/file.rda ~/

From home directory in local to HPC:

scp ~/file.rda sph:~/

Note, we typically copy big data to and from our projects directory on the HPC, likely to a project directory on local, so it would look like this:

scp sph:/projects/epimodel/NETID/project1/file.rda ~/project1/
scp ~/project1/file.rda sph:/projects/epimodel/NETID/project1/

Note that this depends on properly setting up the config file in the previous file. If you have not done that, substitute sph above with [email protected].

FileZilla - GUI

Even though it's recommended to get familiar with navigating the HPC filesystem with the command line, there are graphical software that can help with moving files from and to the HPC.

FileZilla is one such software.