The ERDC DSRC is offering a feature for storage of archival data to the Data Management System (DMS). When sending files to DMS, you have the option to place files in subdirectories specifically designated for the subproject under which the files were created. This option permits the reporting of storage by subproject. Although you are encouraged to archive data in the subproject subdirectories, existing storage commands, procedures, and environment variables remain intact.
The subproject subdirectories are automatically created under your top‑level archive directory, referenced by the environment variable $ARCHIVE_HOME. The names of the subdirectories match the names of their corresponding subprojects. For example, if you are eligible to run jobs at ERDC under subprojects abcde111 and fghij222 on one system and klmno333 on a second system, then subdirectories named abcde111, fghij222, and klmno333 will be created for you on the DMS.
To invoke the storage by subproject process, you must make changes to any scripts and procedures that store archival data on the DMS. For example, if your job is to be run under subproject pqrst444 and you need to store its output data on the DMS, then you would need to change the storage commands within the job as in the following script fragments:
(old)
archive mkdir -C $ARCHIVE_HOME run37
archive put -C $ARCHIVE_HOME/run37 datafile
(new)
archive mkdir -C $ARCHIVE_HOME pqrst444/run37
archive put -C $ARCHIVE_HOME/pqrst444/run37 datafile
In order to successfully operate under the new procedures, your existing DMS files should be relocated under one or more subproject subdirectories. To accomplish this relocation, you can directly log on to the DMS and move the files, or you can use the ERDC Mass Storage utilities. If you prefer to directly log on the DMS, use the command getarchost on any ERDC system to display the address of the DMS server on which your archive directory is located. For example,
$ getarchost
silver.erdc.hpc.mil
You can then use your preferred Kerberized command (ssh, rlogin, telnet) to log on to the designated server. The exceptions to the relocation are any personal login or shell‑history dot‑files such as .login, .cshrc, .profile, .history, etc.; these files should remain in your top‑level $ARCHIVE_HOME directory. If you have multiple subproject subdirectories and you are unsure into which one you should move your files, simply choose one at random. However, you should attempt to place your files correctly so that reporting of storage use will be as accurate as possible.
Use the Unix mv command to relocate your files instead of using the cp command to copy them. mv works almost instantaneously because it simply changes a file pointer. cp must first unmigrate the data from tape to disk before it then makes a second copy of the file in the new location, thus increasing the workload of the archival tape subsystem.
Last update: March 05, 2009
You are accessing a U.S. Government (USG) Information System (IS) that is provided for USG-authorized use only. By using this IS (which includes any device attached to this IS), you consent to the following conditions: * The USG routinely intercepts and monitors communications on this IS for purposes including, but not limited to, penetration testing, COMSEC monitoring, network operations and defense, personnel misconduct (PM), law enforcement (LE), and counterintelligence (CI) investigations. * At any time, the USG may inspect and seize data stored on this IS. * Communications using, or data stored on, this IS are not private, are subject to routine monitoring, interception, and search, and may be disclosed or used for any USG- authorized purpose. * This IS includes security measures (e.g., authentication and access controls) to protect USG interests--not for your personal benefit or privacy. * Not withstanding the above, using this IS does not constitute consent to PM, LE or CI investigative searching or monitoring of the content of privileged communications, or work product, related to personal representation or services by attorneys, psychotherapists, or clergy, and their assistants. Such communications and work product are private and confidential.