File transfer mechanism (lsrcp)

The LSF remote file access mechanism (bsub -f) uses lsrcp to process the file transfer. The lsrcp command tries to connect to RES on the submission host to handle the file transfer.

Limitations to lsrcp

Because LSF client hosts do not run RES, jobs that are submitted from client hosts should only specify bsub -f if rcp is allowed. You must set up the permissions for rcp if account mapping is used.

File transfer using lscrp is not supported in the following contexts:

If LSF account mapping is used; lsrcp fails when running under a different user account
LSF client hosts do not run RES, so lsrcp cannot contact RES on the submission host

See the Authentication and Authorization chapter for more information.

Workarounds

In these situations, use the following workarounds:

rcp and scp on UNIX

If lsrcp cannot contact RES on the submission host, it attempts to use rcp to copy the file. You must set up the /etc/hosts.equiv or HOME/.rhosts file in order to use rcp.

If LSF_REMOTE_COPY_CMD is set in lsf.conf, lscrp uses that command instead of rcp to copy the file. You can specify rcp, scp, or a custom copy command and options in this parameter.

See the rcp(1) and rsh(1) man pages for more information on using the rcp command.

Custom file transfer mechanism

You can replace lsrcp with your own file transfer mechanism as long as it supports the same syntax as lsrcp. This might be done to take advantage of a faster interconnection network, or to overcome limitations with the existing lsrcp. sbatchd looks for the lsrcp executable in the LSF_BINDIR directory as specified in the lsf.conf file.

Sample script for file transfer

#!/bin/sh
# lsrcp_fallback_cmd - Sample shell script to perform file copy between hosts.
#                      This script can be used by lsrcp by configuring 
#                      LSF_REMOTE_COPY_CMD in lsf.conf.
# We recommend placing this file in $LSF_BINDIR.
#

SHELL_NAME="lsrcp_fallback_cmd"
RCP="rcp"
SCP="scp"
SOURCE=$1
DESTINATION=$2

ENOENT=2
EACCES=13
ENOSPC=28

noFallback()
{
echo "Do not try fallback commands"
EXITCODE=0
}

tryRcpScpInOrder()
{
    echo "Trying rcp..."
    $RCP $SOURCE $DESTINATION
    EXITCODE=$?
    #The exit code of rcp only indicates whether a connection was made succesfully or not. 
    #An error will be returned if the hostname is not found 
    #or the host refuses the connection. Otherwise, rcp is always successful.
    #So, we only try scp when the exit code is not zero. For other cases, we do nothing,
    #but the error message of rcp can be seen from terminal
    if [ $EXITCODE -ne 0 ]; then
       echo "Trying scp..."
       #If you don't configure SSH authorization and want users to input password,
       #remove the scp option of "-B -o 'strictHostKeyChecking no'"
       $SCP -B -o 'strictHostKeyChecking no' $SOURCE $DESTINATION
       EXITCODE=$?
    fi
}
tryScp()
{
    echo "Trying scp..."
    #If you don't configure SSH authorization and want users to input password,
    #remove the scp option of "-B -o 'strictHostKeyChecking no'"
    $SCP -B -o 'strictHostKeyChecking no' $SOURCE $DESTINATION
    EXITCODE=$?
}
tryRcp()
{
    echo "Trying rcp..."
    $RCP $SOURCE $DESTINATION
    EXITCODE=$?
}

usage()
{    echo "Usage: $SHELL_NAME source destination"
}



if [ $# -ne 2 ]; then
    usage
    exit 2
fi
case $LSF_LSRCP_ERRNO in
        $ENOENT)
        noFallback
        ;;
        $EACCES)
        noFallback
        ;;
        $ENOSPC)
        noFallback
        ;;
        *)
        tryRcpScpInOrder
        ;;
esac
exit $EXITCODE