path: root/doc/sg_xcopy.8

.TH SG_XCOPY "8" "September 2021" "sg3_utils\-1.47" SG3_UTILS
.SH NAME
sg_xcopy \- copy data to and from files and devices using SCSI EXTENDED
COPY (XCOPY)
.SH SYNOPSIS
.B sg_xcopy
[\fIbs=BS\fR] [\fIconv=CONV\fR] [\fIcount=COUNT\fR] [\fIibs=BS\fR]
[\fIif=IFILE\fR] [\fIiflag=FLAGS\fR] [\fIobs=BS\fR] [\fIof=OFILE\fR]
[\fIoflag=FLAGS\fR] [\fIseek=SEEK\fR] [\fIskip=SKIP\fR] [\fI\-\-help\fR]
[\fI\-\-version\fR]
.PP
[\fIapp=\fR0|1] [\fIbpt=BPT\fR] [\fIcat=\fR0|1] [\fIdc=\fR0|1] [\fIfco=\fR0|1]
[\fIid_usage=\fR{hold|discard|disable}] [\fIlist_id=ID\fR] [\fIprio=PRIO\fR]
[\fItime=\fR0|1] [\fIverbose=VERB\fR] [\fI\-\-on_dst|\-\-on_src\fR]
[\fI\-\-verbose\fR]
.SH DESCRIPTION
.\" Add any additional description here
.PP
Copy data to and from any files. Specialized for "files" that are Linux SCSI
devices that support the SCSI EXTENDED COPY (XCOPY) command.
.PP
This utility
has similar syntax and semantics to
.B dd(1)
but with no "conversions" is supported.
.PP
The first group in the synopsis above are "standard" Unix
.B dd(1)
operands. The second group are extra options added by this utility.
Both groups are defined below in combined, alphabetical order.
.PP
By default the XCOPY command is sent to \fIOFILE\fR. This can be changed
with the \fI\-\-on_src\fR or \fIiflag=xflag\fR options which cause the XCOPY
command to be sent to \fIIFILE\fR instead. Also see the section on
ENVIRONMENT VARIABLES.
.PP
In the SPC\-4 standard the T10 committee has expanded the XCOPY command so
that it now has two variants: "LID1" (for a List Identifier length of 1 byte)
and "LID4" (for a List Identifier length of 4 bytes). This utility supports
the older, LID1 variant which is also found in SPC\-3 and earlier. While the
LID1 variant in SPC\-4 is command level (binary) compatible with XCOPY as
defined in SPC\-3, some of the command naming has changed. This utility uses
the older, SPC\-3 XCOPY names.
.PP
The ddpt utility supports the same xcopy(LID1) functionality as this utility
with the same options and flags. Additionally ddpt supports a subset of
xcopy(LID4) functionality variously called "xcopy version 2, lite" or ODX.
ODX is a market name and stands for Offloaded Data Xfer (i.e. transfer).
.SH OPTIONS
.TP
\fBapp\fR={0|1}
if 1 start the destination of the copy at the end of OFILE. This assumes
that OFILE is a regular file. The default is 0 in which case the destination
of the copy starts at the beginning of OFILE (possibly offset be SEEK). This
option cannot be used with the \fIseek=SEEK\fR option.
.TP
\fBbpt\fR=\fIBPT\fR
each IO transaction will be made using \fIBPT\fR blocks (or less if near
the end of the copy). Default is 128 for logical block sizes less that 2048
bytes, otherwise the default is 32. So for bs=512 the reads and writes
will each convey 64 KiB of data by default (less if near the end of the
transfer or memory restrictions). When cd/dvd drives are accessed, the
logical block size is typically 2048 bytes and bpt defaults to 32 which again
implies 64 KiB transfers.
.TP
\fBbs\fR=\fIBS\fR
where \fIBS\fR
.B must
be the logical block size of the physical device (if either the input or
output files are accessed via SCSI commands). Note that this differs from
.B dd(1)
which permits \fIBS\fR to be an integral multiple. Defaults to the
device logical block size.
.TP
\fBcat\fR={0|1}
sets the SCSI EXTENDED COPY command segment descriptor CAT bit to 0 or
1 (default: 0). The CAT bit (in conjunction with the PAD bit) controls
the handling of residual data. See section
.B HANDLING OF RESIDUAL DATA
for details.
.TP
\fBconv\fR=\fBCONV\fR
all \fBCONV\fR arguments are ignored.
.TP
\fBcount\fR=\fICOUNT\fR
copy \fICOUNT\fR blocks from \fIIFILE\fR to \fIOFILE\fR. Default is the
minimum (\fIIFILE\fR if \fIdc=0\fR or \fIOFILE\fR if \fIdc=1\fR)
number of blocks that SCSI devices report from SCSI READ CAPACITY
commands or that block devices (or their partitions) report. Normal
files are not probed for their size. If \fIskip=SKIP\fR or
\fIseek=SEEK\fR are given and the count is derived (i.e. not
explicitly given) then the derived count is scaled back so that the
copy will not overrun the device. If the file name is a block device
partition and \fICOUNT\fR is not given then the size of the partition
rather than the size of the whole device is used. If \fICOUNT\fR is
not given (or \fIcount=\-1\fR) and cannot be derived then an error
message is issued and no copy takes place.
.TP
\fBdc\fR={0|1}
sets the SCSI EXTENDED COPY command segment descriptor DC bit to 0 or
1 (default: 0). The DC bit controls whether \fICOUNT\fR
refers to the source (\fIdc=0\fR) or the target (\fIdc=1\fR) descriptor.
.TP
\fBfco\fR={0|1}
sets the SCSI EXTENDED COPY command segment descriptor FCO bit to 0 or
1 (default: 0). The Fast Copy Only (FCO) bit set will result in the
copy being done but a technique faster than SCSI READ and WRITE commands.
If the copy cannot but done in a faster manner then a sense key of "Copy
aborted" with and additional sense of "Fast copy not possible" is
returned.
.TP
\fBibs\fR=\fIBS\fR
if given must be the same as \fIBS\fR given to 'bs=' option.
.TP
\fBid_usage\fR={hold|discard|disable}
sets the SCSI EXTENDED COPY command parameter list field called LIST ID
USAGE to 0 if the argument is 'hold', to 2 if the argument is 'discard',
or to '3' if the argument is 'disable'.
.br
If the device has the ability to hold data (as indicated by "held data
limit" being greater than zero) then \fIid_usage\fR defaults to 'hold'
otherwise it defaults to 'discard'.
.TP
\fBif\fR=\fIIFILE\fR
read from \fIIFILE\fR instead of stdin. If \fIIFILE\fR is '\-' then stdin
is read. Starts reading at the beginning of \fIIFILE\fR unless \fISKIP\fR
is given.
.TP
\fBiflag\fR=\fIFLAGS\fR
where \fIFLAGS\fR is a comma separated list of one or more flags outlined
below.  These flags are associated with \fIIFILE\fR and are ignored when
\fIIFILE\fR is stdin.
.TP
\fBlist_id\fR=\fIID\fR
sets the SCSI EXTENDED COPY command parameter list field called LIST
IDENTIFIER to \fIID\fR. \fIID\fR should be a value between 0 and
255 (inclusive). \fIID\fR usually defaults to 1 unless
\fIid_usage=disable\fR in which case it defaults to 0.
.TP
\fBobs\fR=\fIBS\fR
if given must be the same as \fIBS\fR given to 'bs=' option.
.TP
\fBof\fR=\fIOFILE\fR
write to \fIOFILE\fR instead of stdout. If \fIOFILE\fR is '\-' then writes
to stdout.  If \fIOFILE\fR is /dev/null then no actual writes are performed.
If \fIOFILE\fR is '.' (period) then it is treated the same way as
/dev/null (this is a shorthand notation). If \fIOFILE\fR exists then it
is _not_ truncated; it is overwritten from the start of \fIOFILE\fR
unless 'oflag=append' or \fISEEK\fR is given.
.TP
\fBoflag\fR=\fIFLAGS\fR
where \fIFLAGS\fR is a comma separated list of one or more flags outlined
below.  These flags are associated with \fIOFILE\fR and are ignored when
\fIOFILE\fR is /dev/null, '.' (period), or stdout.
.TP
\fBprio\fR=\fIPRIO\fR
sets the SCSI EXTENDED COPY command parameter list field called PRIORITY
to \fIPRIO\fR.  The default value is 1.
.TP
\fBseek\fR=\fISEEK\fR
start writing \fISEEK\fR bs\-sized blocks from the start of \fIOFILE\fR.
Default is block 0 (i.e. start of file).
.TP
\fBskip\fR=\fISKIP\fR
start reading \fISKIP\fR bs\-sized blocks from the start of \fIIFILE\fR.
Default is block 0 (i.e. start of file).
.TP
\fBtime\fR={0|1}
when 1, times transfer and does throughput calculation, outputting the
results (to stderr) at completion. When 0 (default) doesn't perform timing.
.TP
\fBverbose\fR=\fIVERB\fR
as \fIVERB\fR increases so does the amount of debug output sent to stderr.
Default value is zero which yields the minimum amount of debug output.
A value of 1 reports extra information that is not repetitive. A value
2 reports cdbs and responses for SCSI commands that are not repetitive
(i.e. other that READ and WRITE). Error processing is not considered
repetitive. Values of 3 and 4 yield output for all SCSI commands (and
Unix read() and write() calls) so there can be a lot of output.
.TP
\fB\-h\fR, \fB\-\-help\fR
outputs usage message and exits.
.TP
\fB\-\-on_dst\fR
send the XCOPY command to the output file/device (i.e. \fIOFILE\fR). This is
the default unless overridden by the \fI\-\-on_src\fR or \fIiflag=xflag\fR
options. Also see the section below on ENVIRONMENT VARIABLES.
.TP
\fB\-\-on_src\fR
send the XCOPY command to the input file/device (i.e. \fIIFILE\fR).
.TP
\fB\-v\fR, \fB\-\-verbose\fR
equivalent to \fIverbose=1\fR. When used twice, equivalent to
\fIverbose=2\fR, etc.
.TP
\fB\-V\fR, \fB\-\-version\fR
outputs version number information and exits.
.SH FLAGS
Here is a list of flags and their meanings:
.TP
append
causes the O_APPEND flag to be added to the open of \fIOFILE\fR. For regular
files this will lead to data appended to the end of any existing data.
Cannot be used together with the \fIseek=SEEK\fR option as they conflict.
The default action of this utility is to overwrite any existing data
from the beginning of the file or, if \fISEEK\fR is given, starting at
block \fISEEK\fR. Note that attempting to 'append' to a device file (e.g.
a disk) will usually be ignored or may cause an error to be reported.
.TP
excl
causes the O_EXCL flag to be added to the open of \fIIFILE\fR and/or
\fIOFILE\fR.
.TP
flock
after opening the associated file (i.e. \fIIFILE\fR and/or \fIOFILE\fR)
an attempt is made to get an advisory exclusive lock with the flock()
system call. The flock arguments are "FLOCK_EX | FLOCK_NB" which will
cause the lock to be taken if available else a "temporarily unavailable"
error is generated. An exit status of 90 is produced in the latter case
and no copy is done.
.TP
null
has no affect, just a placeholder.
.TP
pad
sets the SCSI EXTENDED COPY command segment descriptor PAD bit. The
PAD bit (in conjunction with the CAT bit) controls the handling of
residual data.(See section
.B HANDLING OF RESIDUAL DATA
for details.
.TP
xcopy
has no affect; for compatibility with ddpt.
.SH HANDLING OF RESIDUAL DATA
The \fIpad\fR and \fIcat\fR bits control the handling of residual
data. As the data can be specified either in terms of source or target
logical block size and both might have different block sizes residual data
is likely to happen in these cases.
If both logical block sizes are identical these bits have no effect as
residual data will not occur.
.PP
If none of these bits are set, the EXTENDED COPY command will be
aborted with additional sense 'UNEXPECTED INEXACT SEGMENT'.
.PP
If only the \fIcat\fR bit is set the residual data will be retained
and made available for subsequent segment descriptors. Residual data
will be discarded for the last segment descriptor.
.PP
If the \fIpad\fR bit is set for the source descriptor only, any
residual data for both source or destination will be discarded.
.PP
If the \fIpad\fR bit is set for the target descriptor only any
residual source data will be handled as if the \fIcat\fR bit is set,
but any residual destination data will be padded to make a whole block
transfer.
.PP
If the \fIpad\fR bit is set for both source and target any residual
source data will be discarded, and any residual destination data will
be padded.
.SH ENVIRONMENT VARIABLES
If the command line invocation does not explicitly (and unambiguously)
indicate whether the XCOPY SCSI command should be sent to \fIIFILE\fR (i.e.
the source) or \fIOFILE\fR (i.e. the destination) then a check is
made for the presence of the XCOPY_TO_SRC and XCOPY_TO_DST environment
variables. If either one exists (but not both) then it indicates where
the SCSI XCOPY command will be sent. By default the XCOPY command is
sent to \fIOFILE\fR.
.SH RETIRED OPTIONS
Here are some retired options that are still present:
.TP
append=0 | 1
when set, equivalent to 'oflag=append'. When clear the action is
to overwrite the existing file (if it exists); this is the default.
See the 'append' flag.
.SH NOTES
Copying data behind an Operating System's back can cause problems. In the
case of Linux, users should look at this link:
https://linux\-mm.org/Drop_Caches
.br
This command sequence may be useful:
.br
  sync; echo 3 > /proc/sys/vm/drop_caches
.PP
Various numeric arguments (e.g. \fISKIP\fR) may include multiplicative
suffixes or be given in hexadecimal. See the "NUMERIC ARGUMENTS" section
in the sg3_utils(8) man page.
.PP
The \fICOUNT\fR, \fISKIP\fR and \fISEEK\fR arguments can take 64 bit
values (i.e. very big numbers). Other values are limited to what can fit in
a signed 32 bit number.
.PP
All informative, warning and error output is sent to stderr so that
dd's output file can be stdout and remain unpolluted. If no options
are given, then the usage message is output and nothing else happens.
.PP
If a device supports xcopy operations then it should set the 3PC
field (3PC stands for Third Party Copy) in its standard INQUIRY response.
This utility will attempt a xcopy operation irrespective of the value
in the 3PC field but if it is zero (cleared) one would expect the
xcopy operation to fail.
.PP
The status of the SCSI EXTENDED COPY command can be queried with
.B sg_copy_results(sg3_utils)
.PP
Currently only block\-to\-block transfers are implemented; \fIIFILE\fR
and \fIOFILE\fR must refer to a SCSI block device.
.PP
No account is taken of partitions so, for example, /dev/sbc2, /dev/sdc,
/dev/sg2, and /dev/bsg/3:0:0:1 would all refer to the same thing: the
whole logical unit (i.e. the whole disk) starting at LBA 0. So any
partition indication (e.g. /dev/sdc2) is ignored. The user should set
\fISKIP\fR,  \fISEEK\fR and \fICOUNT\fR with information obtained
from a command like 'fdisk \-l \-u /dev/sdc' to account for partitions.
.PP
XCOPY (LID1) capability has been added to the ddpt utility which is in
a package of the same name. The ddpt utility will run on other
OSes (e.g. FreeBSD and Windows) while sg_xcopy only runs on Linux. Also
ddpt permits the arguments to \fIibs=\fR and \fIibs=\fR to be different.
.SH EXAMPLES
Copy 2M of data from the start of one device to another:
.PP
# sg_xcopy if=/dev/sdo of=/dev/sdp count=2048 list_id=2 dc=1
.br
sg_xcopy: if=/dev/sdo skip=0 of=/dev/sdp seek=0 count=1024
.br
Start of loop, count=1024, bpt=65535, lba_in=0, lba_out=0
.br
sg_xcopy: 1024 blocks, 1 command
.PP
Check the status of the EXTENDED COPY command:
.PP
# sg_copy_results \-\-status \-\-list_id=2 /dev/sdp
.br
Receive copy results (copy status):
    Held data discarded: Yes
    Copy manager status: Operation completed without errors
    Segments processed: 1
    Transfer count units: 0
    Transfer count: 0
.SH SIGNALS
The signal handling has been borrowed from dd: SIGINT, SIGQUIT and
SIGPIPE output the number of remaining blocks to be transferred and
the records in + out counts; then they have their default action.
SIGUSR1 causes the same information to be output yet the copy continues.
All output caused by signals is sent to stderr.
.SH EXIT STATUS
The exit status of sg_xcopy is 0 when it is successful. Otherwise see
the sg3_utils(8) man page.
.PP
An additional exit status of 90 is generated if the flock flag is given
and some other process holds the advisory exclusive lock.
.SH AUTHORS
Written by Hannes Reinecke and Douglas Gilbert.
.SH "REPORTING BUGS"
Report bugs to <dgilbert at interlog dot com>.
.SH COPYRIGHT
Copyright \(co 2000\-2021 Hannes Reinecke and Douglas Gilbert
.br
This software is distributed under the GPL version 2. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
.SH "SEE ALSO"
There is a web page discussing sg_dd at https://sg.danny.cz/sg/sg_dd.html
.PP
A POSIX threads version of this utility called
.B sgp_dd
is in the sg3_utils package. Another version from that package is called
.B sgm_dd
and it uses memory mapped IO to speed transfers from sg devices.
.PP
The lmbench package contains
.B lmdd
which is also interesting. For moving data to and from tapes see
.B dt
which is found at https://www.scsifaq.org/RMiller_Tools/index.html
.PP
To change mode parameters that effect a SCSI device's caching and error
recovery see
.B sdparm(sdparm)
.PP
See also
.B dd(1), sg_copy_results(sg3_utils), ddrescue(GNU), ddpt,ddptctl(ddpt)