Overview of the mpscp File Copy Utility

The mpscp utility is an easy‑to‑use, high‑performance utility for copying files between systems. It achieves high transfer rates by spreading the data stream over multiple IP sockets, and has the basic command‑line usage of rcp and scp.

mpscp is installed on all of our HPC and Mass Storage systems and is included in your default search path upon login. All configuration settings and environment variables are set for you automatically. An on‑line man page is available by typing "man mpscp".

Advantages

mpscp is a useful alternative for copying files between an HPC system and Mass Storage or between two HPC systems (much like ftp, scp, and rcp). A significant advantage of mpscp is that it can be used to recursively copy entire directory trees by using the "-r" option. You can also adjust the number of parallel data streams over which your file is spread by using the option "-w #". Note that values of # beyond 8 will probably not provide any additional performance improvement.

Limitations

mpscp can only be used from an interactive session on a login node of the Mass Storage or HPC systems.

Finally, when using mpscp to copy files to and from Mass Storage, you must specify the name of the Mass Storage server as well as the full directory path on that server. Unlike the archive command, mpscp does not automatically prepend your default archive directory name ($ARCHIVE_HOME) to the directory path on Mass Storage nor does it know on which server ($ARCHIVE_HOST) your default archive directory exists. These values can also be obtained by executing the commands getarchome and getarchost.

Examples

Listed below are several examples of using mpscp. Note the use of { and } to enclose the environment variable names $WORKDIR, $ARCHIVE_HOST and $ARCHIVE_HOME.

  1. Transfer a file from my $WORKDIR directory to my $ARCHIVE_HOME directory on Mass Storage using 4 parallel data streams.

    mpscp -w 4 ${WORKDIR}/datafile ${ARCHIVE_HOST}:${ARCHIVE_HOME}

  2. Using 8 parallel data streams, recursively copy $WORKDIR/output into the subdirectory $ARCHIVE_HOME/projname on Mass Storage.

    mpscp -w 8 -r ${WORKDIR}/output ${ARCHIVE_HOST}:${ARCHIVE_HOME}/projname

  3. Copy all files from $ARCHIVE_HOME/input on Mass Storage into my $WORKDIR/run99/rundata.

    mpscp ${ARCHIVE_HOST}:"${ARCHIVE_HOME}/input/*" ${WORKDIR}/run99/rundata