Linux

First set up your environment

    If you have an account on Henry2, log in and type the commands into the terminal. If not, set up your environment using Learn Linux without Linux.

Learning path by use case
  • Just the bare minimum

      Go to Essential Linux, try the 7 basic commands, and create a text file using nano. If on Henry2, try Exercise 2.1. If you have problems, go through the links given for more details on those commands.

  • Some background first please!

      Go through the minimal set of the LinkedIn Learning video tutorial. Most of the video content is explaining basic concepts, such as why we would want a command line instead of a GUI, how directories and files are structured, and how commands are formed. It explains Linux in terms of what you already know from using Windows or a Mac. After that, it goes over clearly and in depth - with great examples - all of the 'bare minimum' commands.

  • Background, essentials, and a bit more

      Go through the entire LinkedIn Learning video course. If some sections seem boring to you, still go through them but at higher playback speed so you don't miss any background from skipped sections.

  • Less background, more commands, faster pace

      Going through the text based Full Linux Tutorial is the most efficient way to learn the most commands as quickly as possible.

  • Going to be around for a while?

      Go through the Software Carpentry Tutorial. They start from zero, but go into further depth and introduce shell scripting. After that, do the Full Linux Tutorial. You will skim through the first few sections, pick up some new concepts in the next few, and learn some extras in the last couple sections.

      Also, take advantage of the NC State subscription to LinkedIn Learning. There are several advanced courses on Linux.

Linux/Unix commands and scripting

Text based tutorials
  • Learn Linux without Linux gives instructions for setting up a Linux environment without access to Henry2 or another Linux machine.
  • Essential Linux (from the Quick Start Tutorial) lists the minimal set of Linux to get started: 7 commands and the use of a text editor. It links to tutorial segments covering those commands.
  • Full Linux tutorial is a tutorial developed externally but modified specifically for the Henry2 environment. Completing the Introduction through Tutorial Two will cover the essential commands completely. Since the tutorial means to mimic the Henry2 environment, the default shell is tcsh. This will only cause issues in bash (default for emulators) when completing the later chapters on setting the environment.
  • Software Carpentry tutorial on the Unix Shell. The tutorial explains the basics of the Unix shell (Linux is a Unix-like operating system), including the difference between graphical user interface (GUI) and command line interface, using the prompt, navigating the file system, basic Unix/Linux commands, and the use of a text editor. Setup through Section 3 will cover all basic essentials necessary for interacting with Henry2, with the exception of how to actually look at a file (see cat and less). Note in Section 3, Henry2 users do not have access to the touch command. Authors estimate that time to completion of the first three sections is 95 minutes, not including setting the environment. The remaining sections give an excellent introduction to more advanced file navigation along with basic shell scripting.
  • Video tutorial
  • LinkedIn Learning video tutorial on Linux. Going through the full Linux Tutorial on LinkedIn Learning is well worth the time and consists of less than 2.5 hours total of video instruction and setup. For a quicker start, this collection (visible to NC State users) is a subset of content containing just the essentials needed for basic interaction with Henry2, consisting of about 40 minutes of video instruction plus setting the environment. The collection may not play them in order, so here is an ordered listing of those sections. The listing includes some additional notes and a time estimate to completion.
  • Command line text editors

  • Vi/Vim: See the full LinkedIn Learning video tutorial on Vim or a short section on Vim from LinkedIn Learning Linux tutorial (skip the sudo apt install part).
  • nano: See the full LinkedIn Learning video tutorial on nano or a short section on nano from LinkedIn Learning Linux tutorial.

  • High Performance Computing

    For Henry2 specific instruction, click Get Started or jump to the Quick Start Tutorial.

    Basic concepts

  • Introduction to High-Performance Computing. This Software Carpentry tutorial is customized for the computing environment at the University of Leeds, but much of it is general and applies to the Henry2 cluster. In particular, the setup and lessons 1 and 2 are completely general. In the sections about connecting to the system and transferring files, the only difference is the computer address. On Henry2, that address is login.hpc.ncsu.edu. The section on scheduling is very specific to their scheduler PBS (Henry2 uses LSF). Instead of reading that section, see the documentation on running jobs on Henry2. The section on Software is perfectly applicable, but the exact output from the example commands will understandably be different. (Also, please do not load modules by default by modifying login files as mentioned in the tutorial.) See setting the compute environment for additional detail on the compute environment and loading software modules.
  • Basic HPC hardware terms and definitions. This section from the above tutorial introduces the basic terms of HPC, including nodes, scheduler, cores, interconnect, and processors.
  • Also see the links below under Parallel processing.
  • General hardware

  • Understanding performance and obtaining hardware information. San Diego Supercomputer Center (SDSC), December 2018.

    This webinar discusses how to find CPU specs, memory quantity, cache configuration, file systems, OS version, GPU properties, and instruction sets (ISA). It also discusses why one should understand them, and it includes a short discussion of profiling, monitoring, and optimizing. The target audience is developers - users who write parallel programs but are not necessarily familiar with all of the hardware and performance terms. It is also helpful for users who are familiar with general hardware information but do not know how to find that information on Henry2.

    Parallel processing

  • Intro to Parallel Computing - MPI - 1. XSEDE Monthly Workshop Series: MPI Workshop, April 2019.
    This introductory video explains why parallel programming is necessary, showcases typical exascale computing problems, and gives a demonstration of the basic theory of parallelizing a calculation. It also covers key terminology including the definition of cores, processors, nodes, and PEs (processing elements), the difference between CPU and GPU in terms of proper use cases, and discusses performance and the limitations of parallelization. The video explains which parallel programming platforms are used for each type of parallelism (MPI, OpenMP, OpenACC, CUDA, etc.). The entire video is 1h 15m and is well worth the time.
      • To skip history and Moore's law, start the essentials at 21:32 and watch to at least 35:15. (Less than 15m time investment.)
      • The definitions of Nodes, Cores, etc., is essential to understanding how to properly define and submit parallel jobs on Henry2. To skip to that, go to 43:06 (~3m).
  • Introduction to Parallel Computing. Lawrence Livermore National Laboratory (LLNL).
    This is the material used for the first 2.5 hours of the HPC Training Workshops given by LLNL for their HPC users. Skimming over it will give a great overview of HPC concepts, and reading it carefully will give a comprehensive overview while defining most or all of the HPC terms and definitions a user is likely to ever come across. For HPC users wanting to begin learning and developing parallel applications, go through this tutorial. For an HPC user who only needs to understand parallel architecture enough to run code (i.e. submit HPC jobs) efficiently but does not plan to develop code, check out the diagrams in the Parallel Memory Architectures and Parallel Programming Models sections.
  • The XSEDE HPC Summer Boot Camp was hosted through the NC State University Libraries in June 2019. Contact us with questions or comments about going through the Boot Camp exercises on Henry2.
  • GPU specific

  • Introduction to GPU Architecture and Programming Models. Argonne Training Program on Extreme-Scale Computing (ATPESC) Workshop, 2018.
    This talk is for users already having a solid background in parallel programming but are new to GPU programming. It discusses GPU applications, GPU myths, GPU architecture, and gives some hands on CUDA and OpenCL programming examples.
      • To skip to the heart of the talk, start at timestamp 11:20.
      • The theoretical part ends and CUDA examples begin at 47:17.


    Henry2 Video tutorials

  • The HPC Acceptable Use Policy
  • A Video Tour of the HPC Website (How to find stuff)
  • Log in with with macOS
  • Log in with PuTTY for Windows
  • Navigating the different types of HPC storage (Where do I put my stuff???)
  • Using Globus
  • Requesting specific compute resources with LSF, Part 1: Queues
  • Requesting specific compute resources with LSF, Part 2: GPU Queue
  • See OIT-HPC YouTube Channel to check for new releases.
  • Copyright © 2019 · Office of Information Technology · NC State University · Raleigh, NC 27695 · Accessibility · Privacy · University Policies