File transfer using the terminal

MacOS/Linux

  • Terminal is already included in macOS.
    • Open the Terminal by opening the Applications folder, then the Utilities folder, and then click Terminal.app.
      Alternatively, use the Mac 'Spotlight Search' tool (hit [Command+Space] or click the magnifying glass in the tool bar) and search for Terminal.

Windows

  • MobaXterm

    Download and install the MobaXterm Home Edition (Installer edition).
    Open MobaXterm and click Start local terminal.

  • Click here for more detailed instructions on installing and configuring MobaXterm.
  • Transferring files using SCP:

    Use SCP to transfer single files.

    In this example, the file filename will be copied to/from the user's scratch directory on Henry2 using SCP.

    From the local machine, open a terminal window and use SCP, where user_name is the Unity ID.

    • Local to Henry2
      scp filename user_name@login.hpc.ncsu.edu:/share/group_name/user_name/
    • Henry2 to local
      scp user_name@login.hpc.ncsu.edu:/share/group_name/user_name/filename .
    There will be a prompt for a password and Duo authentication.

    Transferring files using SFTP:

    To transfer multiple files without needing to re-authenticate, or to be able to navigate through the remote file system, use SFTP.

    In this example, the file filename will be copied to/from the user's scratch directory on Henry2 using SFTP.

    From the local machine, open a terminal window and initiate an SFTP session.
    • To initiate an SFTP session, type:
      sftp user_name@login.hpc.ncsu.edu
      where user_name is the Unity ID. There will be a prompt for a password and Duo authentication.
      An SFTP prompt will be opened:
      sftp>
    • Navigate to the share directory:
      sftp> cd /share/group_name/user_name
    • Local to Henry2
      sftp> put filename
    • Henry2 to local
      sftp> get filename

    Use man sftp for a complete list of SFTP commands.

    File transfer using Globus

    Use Globus to transfer files with a GUI interface or to transfer very large files.

    Globus is a web based file transfer application that allows resilient, unattended file transfers between two Globus endpoints. Start the transfer and Globus ensures it completes successfully and sends email when the transfer is done. Globus may be preferable to SCP or SFTP when transferring very large files because it does so unattended, in the background, with status checking and fault tolerance.

    To initiate a data transfer, go to the Globus web site and select the Log In button in top right corner of the page. Select North Carolina State University from the dropdown list of authentication sources and select Continue. On the NC State Shibboleth authentication page, enter the Unity ID and password followed by Duo authentication.

    Select the two endpoints in between which the data transfer will occur. Data may be moved to or from either endpoint; the order does not matter. The Globus endpoint for the Henry2 cluster is NC State Henry2 HPC Cluster. To set a personal location such as a laptop or desktop as an endpoint, the Globus Connect Personal App must be installed on the local machine. Once endpoints are selected, navigate the directory trees from the web interface. Select the file or files to be transferred. Once files are selected, the arrow button for initiating the transfer will be highlighted. Press the button to initiate the transfer. Once initiated the browser window may be closed. Globus will send email when the transfer is complete.

    Video Tutorials
  • Transferring Files with Globus
  • Synchronize local and remote files with rsync

    Use rsync to synchronize the files in a local directory with a directory on Henry2.

    Rsync transfers only the differences between the source files and the existing files in the destination. It is an effective way of backing up files on Henry2 to a local machine. In this example, a directory on Henry2 is /share/group_name/user_name/myfiles and the directory on the local machine is /Users/RealName/HPC/myfiles.

  • For Windows users, first see the instructions and video tutorial for customizing the home directory in MobaXterm to inform how to navigate to the desired local directory.
  • To update the files in the local directory with those that have been modified on Henry2, open a terminal window, navigate to the location of /Users/RealName/HPC/myfiles, and do
    rsync -av user_name@login.hpc.ncsu.edu:/share/group_name/user_name/myfiles/ .

    To see the many additional options and use cases, type man rsync or see the online man pages.

    Before using rsync, use the -n option to test which changes are to be made:
    rsync -anv user_name@login.hpc.ncsu.edu:/share/group_name/user_name/myfiles/ .

    Copyright © 2019 · Office of Information Technology · NC State University · Raleigh, NC 27695 · Accessibility · Privacy · University Policies