This is a quick list of steps to get up and running on the Cray XT3 (sapphire). For more information, follow the supplied links for detailed instructions or access the Cray XT3 User's Guide, the Cray XT3 Beginner's FAQ, or the Cray XT3 Advanced FAQ.
Sapphire can be accessed via Kerberized ssh as follows:
ssh login_node.erdc.hpc.mil
where login_node is one of sapphire01 through sapphire06.
You may also connect using telnet or rlogin. For security purposes, you must have a current Kerberos ticket on your computer before attempting to connect.
Each user has file space available in $HOME, $WORKDIR, and $WORKDIR2 (these are environment variables that point to the appropriate locations in the corresponding file systems). Users are strongly encouraged to use the predefined environment variables $HOME, $WORKDIR, and $WORKDIR2 in their scripts.
Even though the working directories on sapphire are 222 TBytes and 94 TBytes, that space can be consumed very quickly. Users are urged to keep the amount of data in their $WORKDIR and $WORKDIR2 directories to a minimum. Currently, $WORKDIR and $WORKDIR2 files older than 14 days are subject to being purged.
Use Kerberized ftp to transfer necessary files to and from sapphire. Since available storage is limited in your home directory, large data files should be placed in the $WORKDIR or $WORKDIR2 directory.
If you need to move data between machines at the ERDC DSRC, please use the ftp command.
The Portable Batch System (PBS) is the batch system for sapphire. The Cray XT3 User's Guide and Cray XT3 Advanced FAQ contain sample PBS scripts.
To submit a batch job, use the following command:
qsub scriptname
where scriptname is the name of the file containing the batch script. For more information, see the PBS Quick Reference Guide.
Your job script must contain a subproject ID against which the run time is to be charged. Use the show_usage command to generate a list of your current subprojects. For more information on qsub or show_usage, see their respective man pages.
Sapphire has a batch queue structure essentially identical to the other ERDC DSRC systems:
| Queue Name | Max Wall Time | Max Cores per Job |
Priority |
|---|---|---|---|
| urgent | 168:00:00 | 2048 | 50 |
| debug | 1:00:00 | 256 | 40 |
| high | 168:00:00 | 2048 | 35 |
| challenge | 168:00:00 | 2048 | 30 |
| standard | 48:00:00 | 2048 | 20 |
| special | 168:00:00 | 256 | 15 |
| background | 4:00:00 | 2048 | 10 |
You can monitor batch jobs on sapphire by using the qstat, qview, and qhist commands.
You can cancel a job by using "qdel ID", where ID is obtained from the qstat command.
The commands qstat, show_queues, qview, and qlim let you to see the status of the queues.
When your job is finished running, you will want to save any important data. Files in $WORKDIR and $WORKDIR2 are removed on a regular basis, so any important data should be copied to the Data Management System (DMS).
Files can be copied to the DMS using the following command:
archive put -C $ARCHIVE_HOME/subproject filename
and can be retrieved from the DMS using the following command:
archive get -C $ARCHIVE_HOME/subproject filename
For more information, see the archive man page or go to the PST Web site 
.
For available software packages on sapphire, see the Software Version Matrix.
Last update: May 11, 2009
You are accessing a U.S. Government (USG) Information System (IS) that is provided for USG-authorized use only. By using this IS (which includes any device attached to this IS), you consent to the following conditions: * The USG routinely intercepts and monitors communications on this IS for purposes including, but not limited to, penetration testing, COMSEC monitoring, network operations and defense, personnel misconduct (PM), law enforcement (LE), and counterintelligence (CI) investigations. * At any time, the USG may inspect and seize data stored on this IS. * Communications using, or data stored on, this IS are not private, are subject to routine monitoring, interception, and search, and may be disclosed or used for any USG- authorized purpose. * This IS includes security measures (e.g., authentication and access controls) to protect USG interests--not for your personal benefit or privacy. * Not withstanding the above, using this IS does not constitute consent to PM, LE or CI investigative searching or monitoring of the content of privileged communications, or work product, related to personal representation or services by attorneys, psychotherapists, or clergy, and their assistants. Such communications and work product are private and confidential.
