path: root/README

                      README for sg3_utils
                      ====================
Introduction
============
This package contains low level utilities for devices that use a SCSI
command set. Apart from SCSI parallel interface (SPI) devices, the SCSI
command set is used by ATAPI devices (CD/DVDs and tapes), USB
mass storage devices, Fibre Channel disks and IEEE 1394 storage
devices (that use the "SBP" protocol).

This package targets the Linux SCSI subsystem. Recent changes
allow many utilities within this package to be used on other operating
systems which have Unix like support. Most operating systems have
a pass through mechanism for sending SCSI commands and receiving
the corresponding response. This README mainly concentrates on
Linux; see the README.freebsd file to information about the FreeBSD
port.

Most utilities within the sg3_utils package work at the SCSI command
level. For example the sg_inq utility issues a SCSI INQUIRY command
and decodes the response. To view the relationship between the sg3_utils'
utility and the main SCSI command(s) that it issues see the COVERAGE file.
The only utilities that interface at a slightly higher level are sg_dd,
sgm_dd and sgp_dd. These are closely related to the Unix dd command and
typically issue a sequence of SCSI READ and WRITE commands to copy data.

Description
===========
A web site supporting the sg3_utils package, its predecessor sg_utils,
and the Linux SCSI generic driver (and the scsi_debug driver) can be
found at http://www.torque.net/sg . The most recent release version
of sg3_utils and the most recent beta is on that page. There is also
a page describing the utilities in the sg3_utils and sg_utils packages:
http://www.torque.net/sg/sg3_utils.html . A copy of the "sg3_utils.html"
file is in the "doc" subdirectory.

In the Linux 2.4 kernel series these utilities need to use the SCSI generic
(sg) driver to access SCSI devices. The name of this package (i.e.
sg3_utils) refers to version 3 of the SCSI generic (sg) driver which
was introduced at the beginning of the 2.4 linux kernel series. Significantly
this added a new SCSI command interface structure (i.e. struct sg_io_hdr)
that is more flexible than the older "sg_header" structure found in the
sg driver in the 2.2 and earlier linux kernel series. The sg_io_hdr structure
is also more flexible than the awkward (and limiting) interface to the
SCSI_IOCTL_SEND_COMMAND ioctl supported by the linux SCSI mid level.
The version 3 sg driver also added the SG_IO ioctl that is synchronous
(i.e. it issues the requested SCSI command and waits for the response
(or a timeout) before the ioctl returns to the user space program that
invoked it). The SG_IO ioctl is now supported in other parts of the linux
kernel in the 2.6 series.

Utilities that wish to use the asynchronous SCSI command interface
(i.e. via a write() read() sequence) or issue special "commands"
(e.g. bus and device resets) still need to use the linux sg driver. Note
that various drivers (e.g. cdrom/sr) have different open() flag and
permissions policies that the user may need to take into account.

If you have problems or questions about them please contact the author.
Documentation for the linux sg device driver can be found at:
http://www.torque.net/sg/p/sg_v3_ho.html .
This is written in DocBook and the original xml can be found in the 
same directory with the ".xml" extension. Postscript and
pdf renderings are also in that directory.
Older documentation for the sg version 3 driver can be found at:
http://www.torque.net/sg/p/scsi_generic_v3.txt .

All utilities are either "GPL"-ed or have a FreeBSD license. The
intention is that users may incorporate all or part of the code in their
work as they please. Attribution is encouraged. Please check the code as
other contributors (apart from the author) may also have copyright
notices. For a list of contributors see the CREDITS file.

The "sg_lib.[hc]" files contain ASCII text corresponding to most
of the SCSI commands, errors and warning conditions. The "sg_cmds.[hc]"
files contains code to invoke common SCSI commands and associated
response processing. Both are guided by recent drafts at www.t10.org
which is the home site of SCSI (draft) standards. Linux specific
code has been removed from sg_lib.[hc] and sg_cmds.[hc] . The SCSI
common command implementations in sg_cmds.c access the OS pass through
mechanism via the function based interface defined in sg_pt.h . The
linux specific implementation of that pass through is found in the
sg_pt_linux.c file. Since almost all of the utilities use these files,
a shared library called "libsgutils.so" is built. Use of this library
cuts down the size of the binary distributions of sg3_utils
significantly. The "Makefile" provided in the main directory builds
libsgutils.so and then builds each utility to use that shared library.
Alternatively there is a script called "make_no_lib.sh" that will build
utilities without depending on libsgutils.so . For example
"./make_no_lib.sh sg_dd" builds a version of the sg_dd utility that
does not depend on libsgutils.so . See the INSTALL file for more
information.

All the utilities in the main directory have "man" pages. There is
also a sg3_utils (8) man page that summarizes common facilities
including exit statuses. Additional information (including a version
number) can be found towards the top of each ".c" file corresponding
to each utility.

The sg driver in Linux can be seen as having 3 distinct versions:

   v1   lk < 2.2.6     sg_header based relatively unchanged since 1992
   v2   lk >= 2.2.6    enhanced sg_header interface structure [1999/4/16]
   v3   lk >= 2.4      additional sg_io_hdr interface structure [2001/1/4]
   v3   lk >= 2.6      same interface as found in lk 2.4 [2004/1/??]

This package is targeted as "v3" only. Another package called "sg_utils"
is targeted at "v2" and to a lesser extent "v1". The "sg_utils"
package has a subset of the utilities found in this package.

Some sg ioctls (notably SG_IO) are defined for many block devices
in lk 2.6 . In practice this means all SCSI block devices, ATAPI block devices
(mainly CD and DVD players) but _not_ ATA disks (currently). SATA disks that
use the libata kernel library accept SCSI commands and thus are supported.
Support for the SG_IO as been added to the scsi tape driver (st) in lk 2.6.6
(and support for osst and ch (medium changer) may soon follow).

No utilities in the main directory use the sg driver's older "sg_header"
interface; instead they use the newer "sg_io_hdr" interface. The "sg_io_hdr"
interface can be accessed two ways:
  - using the SG_IO ioctl [for synchronous access]
  - using a write()/read() sequence that convey instances of "sg_io_hdr"

All utilities in the main directory that issue SCSI commands use the SG_IO
ioctl except for sgp_dd.  This is due to the asynchronous nature of sgp_dd.
Note that sgp_dd does _not_ support the "blk_sgio" switch found in sg_dd.
This is important since block devices often identify themselves
(programmatically) as sg devices in lk 2.6 and it would cause serious damage
to do a write() of the sg driver's "sg_io_hdr" meta data (i.e. disk
corruption).

Warning
=======
Many devices use SCSI command sets over transport protocols not normally
associated with SCSI (as defined at http://www.t10.org ). Some of these
devices react poorly (e.g. lock up) when sent SCSI commands that they
don't support. Even sending a supported SCSI command with a field set
to an unexpected value can cause problems.

For example, all "SCSI" devices must support the INQUIRY command which
the SCSI-2 standard says should request a 36 byte response. However later
SCSI standards (e.g. SPC-2) have increased that length but some SCSI
devices lock up when they receive a request for anything other than
a 36 byte response.

Any well implemented "SCSI" device should react sensibly when a utility in
sg3_utils sends a SCSI command that it doesn't support. Unfortunately this
cannot be guaranteed.


Utilities
=========
Here is list in alphabetical order of utilities found in the main directory
of the sg3_utils directory:
  - sginfo, sgm_dd, sgp_dd, sg_dd, sg_emc_trespass, sg_get_config,
    sg_format, sg_ident, sg_inq, sg_logs, sg_luns, sg_map, sg_map26,
    sg_modes, sg_opcodes, sg_persist, sg_prevent, sg_rbuf, sg_rdac, sg_read,
    sg_readcap, sg_read_long, sg_reassign, sg_request, sg_reset, sg_rmsn,
    sg_rtpg, sg_scan, sg_senddiag, sg_ses, sg_start, sg_sync,
    sg_test_rwbuff, sg_turs, sg_verify, sg_vpd, sg_write_long, sg_wr_mode

These utilities and the libsgutils.so library which they depend on are built
by the Makefile in the main directory. This Makefile does not invoke the
Makefile's in the subdirectories. Each utility in the main directory has a
"man" page in section 8 (system administration commands). Binary
distributions of the sg3_utils package (e.g. "rpm" and debian packages)
typically contains the shared library, the utilities in the main directory,
their associated man pages and some documentation files (e.g. README, INSTALL,
CREDITS, COPYING and COVERAGE). See the INSTALL file for instructions about
building and installing from a "tarball".

Man pages can be read (without building and installing the package) by
going to the main directory and executing something like this:
 $ man ./sg_dd.8

To see which SCSI commands (and ATA commands) are used by these
utilities see the COVERAGE file.

Here is a list in alphabetical order of utilities found in the archive
subdirectory:
  - isosize, scsi_devfs_scan, sg_debug, sgq_dd
Some of these utilities have man pages.

Here is a list in alphabetical order of utilities found in the examples
subdirectory:
  - scsi_inquiry, sg_simple1, sg_simple2, sg_simple3, sg_simple4,
    sg_simple5 and sg_simple16.c

Also in that subdirectory is a script to test sg_persist, an example
data file for sg_persist (called "transport_ids.txt") and an example
data file for sg_reassign (called "reassign_addr.txt"). There are also
two scripts for 'sg_senddiag -pf -raw=-' that will put some SAS disk
phys into a compliant jitter tolerance pattern (CJTPAT). 

The "utils" subdirectory contains source and a Makefile to build
"hxascdmp" which accepts binary data from stdin (or a file on the
command line) and outputs an ASCII-HEX and ASCII representation of
it. It is similar to the Unix od command. There is also code to
sg_chk_asc.c which checks a given text file (typically a copy of
http://www.t10.org/lists/asc-num.txt ) and checks it against the
asc/ascq test strings held in sg_lib.c .

The "doc" subdirectory contains copies of web pages relevant to
the sg3_utils package. Currently it contains:
  - sg_dd.html : describes the sg_dd utility and compares it with the
                 Unix dd command
  - sg_io.html : describes the SG_IO ioctl used by sg3_utils. Looks
                 at differences between the lk 2.4 and 2.6 series;
                 including its use with block devices
  - sg3_utils.html : a description of the sg3_utils package contents
  - tools.html : a list of SCSI and storage tools with a summary and url
    

Notes for utilities without man pages
=====================================
The "scsi_inquiry" program shows the use of the SCSI_IOCTL_SEND_COMMAND
ioctl to send a SCSI INQUIRY command. That ioctl() is supported by the 
SCSI sub system mid level and so is common to all sd, sr, st and sg devices. 
This program has been placed in the "examples" subdirectory.

"sgq_dd" is yet another variant of dd found in the archive directory.
From the user's point of view it is very similar to sgp_dd but uses a
non-blocking state machine rather then POSIX threads for parallelism.

"sg_debug" is effectively defunct now. The user can instead do:
$ cat /proc/scsi/sg/debug . This command has been placed in the
archive directory.

"sg_simple1" and "sg_simple2" are simple example programs demonstrating
calls to the SCSI INQUIRY and TEST UNIT READY commands. They only differ
in their error processing: sg_simple1 uses sg_lib.[hc] for error
processing while sg_simple2 does its own more primitive checks.

"sg_simple3" tests out user space scatter gather added to the version 3
sg driver.

"sg_simple4" shows the INQUIRY command using mmap-ed IO to obtain its
response buffer.

"sg_simple5" also sends and INQUIRY and TEST UNIT READY commands. It
uses the generic pass through mechanism based on sg_pt.h . It will
currently build in linux and FreeBSD (with "make -f Makefile.freebsd").
It has extensive error checking code.

"sg_simple16" attempts to send a 16 byte SCSI command, READ_16, to the
scsi device. This is only supported for lk >= 2.4.15 and for adapter
drivers that indicate that they have 16 byte CDB capability (otherwise
DID_ABORT will appear in the host_status).

"sg_sat_identify" attempts to push an ATA IDENTIFY DEVICE or
IDENTIFY pACKET DEVICE command through the SAT-defined ATA PASS
THROUGH (16) SCSI command. If successful, the 256 word (512 byte)
response is output.

"sg_sat_chk_power" attempts to push an ATA CHECK POWER MODE command
through the SAT-defined ATA PASS THROUGH (16) SCSI command. That
ATA command needs to read the "FIS" registers after the command is
completed which involves using the ATA Status Return (sense data)
descriptor (as defined in SAT).

"sg_sat_smart_rd_data" attempts to push an ATA SMART/READ DATA command
through the SAT-defined ATA PASS THROUGH (16) SCSI command. If
successful, the 256 word (512 byte) response is output.


Command line processing
=======================
These utilities can be divided into 3 groups when their handling of
command line arguments is considered:
  - ad hoc, typically in a short form only, sometimes longer (e.g.
    "sg_logs -pcb /dev/sdc")
  - inspired by the dd Unix command (e.g. sg_dd, sgm_dd, sgp_dd, sg_read)
  - recent utilities use "getopt_long" (see "man getopt_long")
    type command lines. These have short form (starting with "-")
    and corresponding longer form (starting with "--") options.

The recent utilities that use "getopt_long" are, in alphabetical
order:
  - sg_format sg_get_config sg_ident sg_luns sg_map26 sg_persist
    sg_prevent sg_read_long sg_reassign sg_requests sg_rmsn sg_rtpg
    sg_ses sg_sync sg_test_rwbuf sg_verify sg_vpd sg_write_long sg_wr_mode


Linux header file problems
==========================
In linux some utilities include 2 important header files:
#include <scsi/sg.h>
#include <scsi/scsi.h>
These files are typically found in the directory /usr/include/scsi which
is maintained by the GNU glibc team. Unfortunately these GNU supplied
headers may not be (functionally) the same as those found in the kernel
source:
/usr/src/linux/include/scsi/sg.h   and
/usr/src/linux/include/scsi/scsi.h

If glibc and the kernel on a machine are of the same vintage then it is
probably sufficient to use the simple includes listed at the start of this
section. Another technique that used to work was to rely on
/usr/include/linux being a symbolic link to /usr/src/linux/include/linux . 
That caused the following:
#include <linux/../scsi/sg.h> 
#include <linux/../scsi/scsi.h>
to find the kernel supplied header files. However recent versions of
glibc have removed this symlink. Hence this technique is no longer
recommended.

The include file path issues are now all addressed in one file called
"sg_linux_inc.h". Please read that file. This fetching of the scsi.h and
sg.h header files has not been a problem in the latter lk 2.4 and lk 2.6
series.

scsirastools
============
This package found at http://scsirastools.sourceforge.net contains utilities
which overlap with the functionality offered in sg3_utils. Utilities of note
in scsirastools are:
  - sgdskfl: for loading firmware into SCSI disks
  - sgmode: get and set mode pages
  - sgdefects: list primary and grown defect lists
  - sgdiag to perform format and other test functions

plscsi
======
This is a utility which can issue arbitrary SCSI commands (i.e. supplied by
user). It is available for Linux and other operating systems.
See http://members.aol.com/plscsi .

Doug Gilbert
6th July 2006