LONE-TAR MAN Pages (Chapter 3)

NAME
SYNOPSIS
DESCRIPTION
OPTIONS
Backup Related Function Letters
Listing/Verify Related Function Letters
Restore Related Function Letters
Both Backup and Restore Function Modifiers
Modifiers used for Backup Only
Modifiers used for Restore Only
DOUBLE BUFFERING
DIAGNOSTICS
ERROR RETURN CODES
COMPRESSION PERFORMANCE
ERROR RECOVERY
Error Recovery During Backup
Error Recovery During Restore
SPECIAL FEATURES
Backup Summary
Restore Summary
Volume Switching
Background Operation
Signal Processing
Screen Input and Output
Use of Wild-cards
Excluding Files From Being Compressed
Specifying Dates and Times
ENVIRONMENT VARIABLES
VOL_SWITCH_CMD - Command to Switch Tape Volumes
REMOTE_PUT - Network Command for Backing Up
REMOTE_GET - Network Command for Restore or Verify
FILES
REMOTE TAPE SUPPORT
VIRTUAL FILE SUPPORT
BIT-LEVEL-VERIFICATION
COMPATIBILITY
Limitations
Internal Implementation Notes

3.1 NAME

LONE-TAR - Tape or Floppy Archiver.

3.2 SYNOPSIS

lone-tar -{MIcCrPtTxNU} [vVfbehnkslLpmGEFZDdR0-99AaSw] [tape] [block size] [compression limit] [floppy/tape size] [-zKEYWORD] file1 file2...

3.3 DESCRIPTION

LONE-TAR saves and restores files on magnetic tape, floppy disk, or add-on hard disk. LONE-TAR's actions are controlled by a key argument. The key is a string of characters containing one or more function letters and possibly one or more function modifiers. Other arguments to the command are file or directory names specifying which files are to be dumped or restored. A directory name refers to the files and to the recursive subdirectories of that directory.
LONE-TAR permits a file to extend across media boundaries. LONE-TAR can archive and restore an entire filesystem. All files and directories are restored. This includes directories containing no files, special character and block device files, named pipe files, linked files, and symbolically linked files. LONE-TAR can handle dual drives of differing storage capacity in a sequential manner with automatic switching between drives. Additionally, LONE-TAR has built in error recovery capability should the backup media develop a fault. The function portion of the key argument is specified by one of the following letters:

3.4.1 Backup Related Function Letters

c Creates a new floppy/tape. Writing begins at the beginning of the floppy/tape rather than the last file. All previous data is erased. If no files are specified, then the previous data is not erased. If the device can seek (i.e. floppy), then data is compressed if the file size is over 15 blocks. Executable files are not compressed unless the Z option is included as a modifier. Files that are already compressed with the pack or compress utility (end in '.z' or '.Z' respectively) are not compressed. Files that end in '.zoo', '.arc' or '.zip' are also not compressed.
C Identical to c above except that compression is turned off.
M This is the Master backup option. LONE-TAR does a complete backup of the filesystem including mounted file systems. This backup should be done from the root directory. The date of the backup is put in the file ./etc/Master_backup and is available for viewing. An example is:

cd /
lone-tar Mvfb /dev/tape 120 .

If there is no ./etc directory present, LONE-TAR puts the date in the file ./Master_backup. The date is updated only if the Master backup completes successfully and is not interrupted. During the Master backup, LONE-TAR places an unenforced lock on each file as it is backed up. The lock prevents writing but allows reading. See the information on the locking option modifiers below for more information. The M option automatically invokes the D modifier to back up directory permissions, the R modifier to use unenforced file lockin, and the l modifier to report unresolved links. If the device is seeking (floppy disk or magneto optical disk), data is LONE-TAR compressed unless the C option is used. LONE-TAR determines if a device is seeking by looking it up in the Device Library.
I This is the Incremental backup option. All files modified since the last Master backup are backed up. LONE-TAR traverses the entire filesystem including mounted filesystems and looks for any files that have been modified since the last Master backup.
The date of the last Master backup is taken from the last modification time of the file ./etc/Master_backup. Do not edit or modify this file. This option should be invoked from the root directory. Example:

cd /
lone-tar Ivfb /dev/tape 120

As in the Master backup, the Incremental backup invokes the D modifier to back up directory permissions and the R modifier to use unenforced file locking.
P Pack-regardless option. This option forces compression of files to a non-seeking device such as a tape. Use this option when you want to compress files to a tape. This is useful when regular tape backups (without compression) start to use more than one tape volume. The P option implies the c option. The k or s (volume size) options must be specified along with the P so that the volume size is known in advance. For example,

lone-tar PZvfbk /dev/tape 120 60000 /

will back up the entire filesystem compressing all files over 15 blocks to a 60 Megabyte tape. The P option supports dual drives with differing capacities. This option uses a combination of memory and temporary free space on the hard disk which is referred to as the Virtual Pipe. The default is to use the /tmp directory. Optionally, the environment variable TMPDIR can be set to the name of an alternative temporary directory. In general, this should reside on the filesystem with the most free space.
r Writes the named files to the end of the floppy disk. The device must be a seeking device such as a floppy or magneto optical drive for this option to be effective. It does not work on most tape drives.

3.4.2 Listing/Verify Related Function Letters

t Lists the names of all files on the floppy/tape. If used without the v modifier, only the filenames are listed, otherwise the filenames as well as permissions and modification times are listed. If the device is seeking, then a rapid listing occurs. An internal default to a block factor of 1 is used in this case. The following lists all files on the tape device /dev/rct0.

lone-tar tvfbk /dev/rct0 120 60000

If you want to see if a few particular files reside on the floppy/tape, you can put the filenames you are interested in on the command-line. If those files are present on the floppy/tape they are listed. The wild-card characters '*' and '?' can be used. If wild-cards are used, they are enclosed in either single or double quotation marks. Only those files with names matching those on the command-line are listed. The following command lists all files ending in ".dat".

lone-tar tvfbk /dev/rct0 120 60000 "*.dat''

T True verification of all files on floppy/tape. LONE-TAR carefully reads the entire media to make sure that the files are intact and there are no bad blocks. This is referred to as a Level 1 verification. The checksums of all header blocks are checked. This does not compare the data on the floppy/tape to that on the hard disk. If any filename arguments are given (wild-cards are allowed), only those files matching the given filename argument are displayed on the screen. At the completion, you are notified whether or not the verification was successful.
TT This is Level 2 verification (also referred to as Bit-level-verification). It encompasses a Level 1 verification with the addition of a Bit-level comparison of the file on the archive media to that on the filesystem hard disk. The command:

lone-tar TTvfbk /dev/rct0 120 60000

performs a Level 2 verification on the media of the tape device with a block factor of 120 and tape capacity of 60000 Kilobytes.
Each filename is listed and if it compares exactly using Bit-level-verification, the symbol <V> is shown on the line following the filename. Any changes such as size changes, date changes, and byte level changes are shown by the symbols <!Size>, <!Date>, and <!Byte> respectively. Bit-level-verification is totally integrated into the product and can verify compressed files, virtual files, compressed virtual files and files split across media. Compressed files are bit-level verified by expanding them on-the-fly and comparing the output to the hard disk file. Virtual files are re-virtualized and then compared to the hard disk file. Level 2 verification produces a summary when completed. The summary includes the number of files that were different and the number of files that were not checked because of inadequate permissions or because they no longer exist. The details of Bit-level-verification are recorded in a separate file referred to as the CHANGEFILE. This defaults to /tmp/ChangedFiles but can be changed using environment variables.

For more details, please refer to the section entitled BIT-LEVEL-VERIFICATION

3.4.3 Restore Related Function Letters

x Extracts (restores) the named files from the floppy/tape. If the named file is a directory present on the archive, then the entire directory is recursively extracted. The list of filenames can include the wild-card characters '*' and '?'. These should again be put within quotation marks if used. If no file arguments are given, the entire content of the floppy/tape is extracted. If multiple versions of the same file are on the tape, the last version overwrites all preceding versions. If a file is in compressed format, it is decompressed in a streaming fashion directly to the hard disk without any intermediate files. This prevents disk fragmentation.
N Non-Destructive Restore. This is the same as the x option except, if a file is already present, it is not overwritten. This is useful for upgrading to new versions of operating systems.
It is also useful if a set of files has been inadvertently wiped out but you are not exactly sure which files have been affected. A Non-Destructive Restore brings back any file not present, but does not affect files already present on the hard drive. Keep in mind that if you are restoring a Master and Incremental backup using the Non-Destructive restore feature, you must restore the Incremental backup first, then the Master backup. This is the opposite of what you would do if you weren't using the Non-Destructive restore feature. Note that device files in /dev are always non-destructively restored.
U Update restore option. All files on the hard disk that are older than those on the backup floppy/tape are updated. This ensures that only the newest copy of a file is restored. If a file to be restored does not exist on the hard disk it is created.

3.4.4 Both Backup and Restore Function Modifiers

The following characters modify the above key function letters:

TARDIR=/tmp
export TARDIR

In the above example, the catalog files go into the /tmp directory.
For user convenience in keeping up with archives, the file LAST_{prefix} is automatically linked to the last catalog file made of its type. Thus, the file LAST_Master is always the catalog of the last Master backup made. Likewise, LAST_Increm contains the catalog of the last Incremental backup. After a period of time, the catalog directory can get rather cluttered. This is cleaned up by removing all files except those starting with `LAST_`.
The environment variable TARFILE is used to set the specific file name you want the catalog file to be. For example,

TARFILE=special23
export TARFILE

assigns the catalog file to /usr/lib/ltar/BACKUPS.DIR/special23 if the TARDIR is not set. If TARDIR is set to /tmp as above, then this would be /tmp/special23.
If the user wants to keep a copy of just the last catalog files (LAST_{prefix} files), the special keyword ``NONE'' can be used with the TARFILE variable. For example,

TARFILE=NONE
export TARFILE

means that no catalog file is to be made except for LAST_{prefix}. This file is placed in the directory specified by the environment variable TARDIR, or the default directory if this variable is not set.
By using an absolute pathname with TARFILE, the TARDIR variable is ignored and no links are made to LAST_{prefix}. For example, the following sets the catalog file to /tmp/CATALOG with no links:

TARFILE=/tmp/CATALOG
export TARFILE

If the TARFILE variable contains a `/`, then it is assumed to be a full or relative pathname, and this is used for the catalog file and TARDIR is ignored. There is no other link made to the catalog file in this case.
b Causes LONE-TAR to use the next argument as the blocking factor for floppy/tape records. The default is dependent on the version of XENIX/UNIX. For improved performance, it is recommended that one use a block factor of 15 or 18 for floppy disks and as large as possible for tapes. The maximum is implementation specific and usually dependent on the operating system and available memory. Typically, this is 120 to 2048 blocks.
f Causes LONE-TAR to use the next argument as the name of the archive rather than the default of /dev/lonetar. When this option is used, LONE-TAR assumes the device is non-seeking. If the name of the file is dash (-), LONE-TAR writes to the standard output or reads from standard input. Thus, you can move hierarchies with the command:

cd fromdir; lone-tar cf - . | (cd todir; lone-tar xf -)

k LONE-TAR reads the next argument as the size of the volume in kilobytes. This size is needed for splitting very large files into extents across volumes. When used with the v option, the screen output includes the room left on the current volume as:

VOL=XXXX

find / -group XX -exec rm -i {};

n The device is not a magnetic tape and thus can seek. Unless the P option is used, compression only works with seeking backup devices. In general, any time the f modifier is used, the device is considered to be non-seeking and compression is turned off. When the n modifier is used along with the f modifier, it indicates the device is actually capable of seeking. The cnf option string is used to force compression on seeking devices (floppy disk, optical disks). The tnf option string is used to force a rapid listing of the archive.
F File option. LONE-TAR gets a list of filenames to use for archiving or restoring from the filename specified in the next argument. Any number of files can be in the list, each on a separate line. This option works for both backing up and restoring files. Additionally, if the name of the file is a dash (-), the standard input is used to obtain a list of files to back up. For example, the following command backs up all source code files (any file ending in ``.c'') that have changed within the last seven days:

find / -name ``*.c'' -mtime -7 -print | lone-tar -CvF -

E Exclude option to exclude files or directories from being archived or restored. This option letter can be repeated to exclude up to one-hundred twenty-eight each of files or directories. For example:

lone-tar cv -E /usr -E /dev . (exclude 2 directories)
lone-tar cvEE /usr /dev . (equivalent to above)
lone-tar cv -E/dev -E/usr/adm/messages . (exclude one directory and one file)

When used during a backup it does not matter if the spelling of the excluded file has a leading '/' or './'.
The wild-card characters '*' and '?' can be used for filename pattern matching. Up to twenty of these patterns can be used. Any wild-card expressions should be quoted with either single or double quotes. The following command excludes all index files (those ending in '.idx') from a Master backup:

lone-tar MV -E '*.idx' .

The E option using wild-cards also applies to directories. For example, if you want to exclude all the "Wastebasket" directories, you would use the following command:

lone-tar MV -E '*/Wastebasket' .

The E option can be used to exclude all read-only filesystems using:

lone-tar cvE Readmounts .

Additionally, the E option can be used to exclude all other mounted filesystems regardless of mount points as follows:

lone-tar cvE Allmounts /

This avoids having to unmount other filesystems and allows others to use the system while backup of one filesystem is in progress.
Furthermore, the E option can be used to exclude all NFS mounted filesystems as follows:

lone-tar cvE Netmounts .

This excludes all the NFS mounted filesystems. You can tell in advance which filesystems are excluded by looking at the output of the mount command. Those filesystems that start with a node name followed by a colon ":", followed by a directory, are the ones excluded.
X This is the eXclude list file. Rather than using a long series of E modifiers on the command-line, it is easier to put the files in a list one per line. For example, if the name of the list is excl.lst, you invoke LONE-TAR as follows:

lone-tar cvX /u/excl.lst

Like the E modifier, you can use wild-cards in the list, but you cannot use the keywords: Readmounts, Netmounts, Allmounts since these could represent actual filenames you wish to exclude.
0-99 These numbers refer to an entry in the Device Library of available devices. The library can have up to 100 entries, and specifies the various parameters associated with a floppy or tape backup device. For example, to do a Master backup to device 9 in the library use:

lone-tar Mv9 .

The file used for the Device Library is /etc/default/tar. The Device Library is an ascii file that can be edited with a simple editor. Sample lines in the Device Library look like:

archive0=/dev/null 120 150000 y # Null device archive1=/dev/rfd048ds9 18 360 n # Floppy archive9=/dev/rStp0 120 5250000 n # Scsi Tape archive13=/dev/rct0 120 60000 y # Tape

lone-tar Cv -zDATE=2/4/96-18:30 .

This option is used for restoring files based on the date the file was last modified. For example, if you are restoring from a Master backup but only wanted to restore files modified on or after November 5, 1995 at midnight, the following command is used:

lone-tar xv9 -zDATE=11/5/95 .

-zSWAP
This byte swaps all data. If this option is used when backing up data, the data is byte swapped just before it is written to the backup device. On a restore, the data is byte swapped just after being read from the backup device. This is useful for transferring data between differing computer systems.

3.4.5 Modifiers used for Backup Only

S Speed option. This option initiates double buffering using shared memory, semaphores and dual processes. In many instances, this significantly speeds up the backup process. Five shared memory buffers, each the size of the blocking factor (specified with the b modifier) are created to facilitate data transfer. See the section entitled DOUBLE BUFFERING on page 58 for more details. DOUBLE BUFFERING on page 58 for more details.
L Specifies the limit (default of 15 blocks) used for compressing files. Any file larger than this limit is compressed.
Z This makes compression include executable files. Unless this modifier is used, files with the execute bit set are not compressed. This option implies c.
l Tells LONE-TAR to notify you if it cannot resolve all the links to the files being backed up. This is the default for Master backups. A list of unresolved linked files is printed at the end of the backup.
e Extent alone option. This keeps all files archived so they do not extend across volumes. If there is not enough room on the present volume for a given file, LONE-TAR prompts for a new volume. It is required that this be used with the k or s option so that LONE-TAR knows exactly how large the volume is.
h Archive the contents of the symbolically linked named files. The command ``lone-tar cv'' archives the symbolic linkage information only, while the command ``lone-tar chv'' archives the actual contents.
D Directory option. This ensures that each directory entry itself, including access permissions, group and user id, is backed up. This information is backed up following all the files within the directory. When restoring, the directory is restored to its original access permissions, group and user id, and modification time. This option is automatically invoked when the M or I options are used. Unless the D option is used, the specific owner, group, and access modes of a directory are not backed up. Restoring such a backup causes each directory to be set to the owner d and group id of the individual doing the restore.
For releases after March 1996, this option is always on. You can turn it off with the -zNODIR option. This is necessary if data is transferred to a system with the standard tar that cannot interpret directory permissions.
d Directory backup depth. This requires an argument specifying the directory depth level to be used when backing up. A level of 0 means only files in the current directory are backed up. No subdirectories are backed up. A level of 1 means only the first level of subdirectories are backed up. This option is automatically set to 0 when names are being piped in from another program using the F option:

find / -newer /tmp/newfile -print | lone-tar CvF -

This prevents directories from being backed up multiple times.
R Unenforced READ LOCK. An attempt is made to READ LOCK every file. A READ LOCK means the file cannot be modified by any other program while the lock is in place. However, this lock is not enforced and if it cannot be obtained, the file is backed up anyway. This option is automatically assumed during the Master backup and Incremental backups.
Under UNIX V (which uses advisory locking), if a lock cannot be obtained, a warning message stating that the file was in use is printed as follows:

WARNING: File was in use during backup

WARNING: File was locked (System ENFORCED) will retry later...
lone-tar: (2nd lock attempt failed) File was NOT backed up because lock was ENFORCED!
lone-tar: (2nd lock attempt failed) File was backed up WITHOUT a lock!
lone-tar: (2nd lock attempt failed) File was NOT backed up because lock was ENFORCED!
^^^^^--> LOCKED by another program!! Waiting...
Resuming backup...

lone-tar MV9RRR .

a Access times unchanged. This option makes sure that the file access times of any of the files backed up or bit-level verified are not changed. Normally the file access time is updated to the last time a file is read or backed up. Unless this option is used, the access time of every file on the system is set to the time the file was last backed up. If this option is used, then the file access times truly reflect the last time the file was needed by someone. Some disk optimizers rely on file access times to re-organize the files.
? This prints the version number of LONE-TAR and a help screen.
-zRESTART=filename This is the restart option to a backup. This is useful when a backup of a large amount of data has failed and you do not want to perform the entire backup again. The proper use of this command involves inserting the restart option within the original command that failed. You must insert the -zRESTART=filename option in the original command just before the list of files to back up. The filename represents the first file with which to restart the backup. For example, if the original command was:

lone-tar MV9 .

and this failed on the third tape because of a bad tape, and the file it was trying to back up when the failure occurred was ./u2/appl/ins.dat, to restart the backup the following command is given:

lone-tar MV9 -zRESTART=./u2/appl/ins.dat .

Please note that the trailing dot is still needed just as it was in the original command.
-zDEV=raw_devices
This is the raw device backup option. A raw character device is backed up from the start to the end. This is useful for large database applications that use raw partitions to store the data. The device specified must be a character device. If a block device is specified by mistake, it results in an error message. Compression works for raw devices just as it does for other files. To avoid compression, make the raw device executable or put it in the COMP_EXCL environment variable.
Any number of raw devices can be specified by using a comma to separate one name from the other. There should be no spaces either before or after each comma. For example, the following command performs a Master backup and backs up three large raw device data partitions.

lone-tar MV9 -zDEV=/dev/rhd10,/dev/ru2,/dev/ru3 .

The raw character devices are backed up after all the data on the main filesystem has been backed up. An alternative way of specifying the command-line is to enclose the raw device list in quotation marks separated by spaces:

lone-tar MV9 -zDEV="/dev/rhd10 /dev/ru2 /dev/ru3" .

If you perform a listing of files on the tape, the raw character devices display as regular files with a large size. The size reflects the true size of the partition. This way the raw partition can also be restored by the regular TAR utility.
You can set the environment variable RAW_SCRIPT to the name of a program you want to run before each raw partition is backed up. This allows you to shutdown databases or other programs that may be accessing the partition. The script file is called twice for each raw partition, once just before the backup is done and once after the backup of the raw partition is complete. To distinguish the two, the first argument is either -begin or -end, so the script knows under which of the two cases it is called. The second argument is the name of the raw partition. The script should return with a zero exit code. If the return exit code is 100, LONE-TAR won't backup the partition.
Using the above example, and assuming RAW_SCRIPT is set to db_startstop, as follows:

RAW_SCRIPT=db_startstop
export RAW_SCRIPT

the following actions/commands take place:

db_startstop -begin /dev/rhd10 Raw partition /dev/rhd10 is backed up db_startstop -end /dev/rhd10 db_startstop -begin /dev/ru2 Raw partition /dev/ru2 is backed up db_startstop -end /dev/ru2 db_startstop -begin /dev/ru3 Raw partition /dev/ru3 is backed up db_startstop -end /dev/ru3

3.4.6 Modifiers used for Restore Only

A Absolute pathname strip option. This strips the leading slash '/' from the pathnames being listed, verified, or restored. This allows files to be restored relative to the current directory. This works also for linked, and symbolically linked files. When using this option, you should specify the filenames on the command-line without a leading slash '/' for proper matching to occur. If an archive is from a DOS system, the leading drive letter, colon combination is also stripped (e.g. C: or D:).
h restores the contents of a symbolic link, if the link exists on the system. This applies to versions after 1 August 1997. See the release notes for further details.
m LONE-TAR restores the last modification time of a file to the time and date of the restore rather than the actual file last modification time.
p Extract using original permissions. It is possible that a regular user may be unable to extract files because of the permission associated with the files or directories being extracted.
w Wait for confirmation before restoring a file. The user is prompted for a y or n answer on each and every file.
Nw This is the Non-destructive restore with interactive renaming. It is somewhat similar to the w modifier except it is only interactive when a disk file is about to be overwritten by a file on the tape. In this case, you choose to overwrite the file, skip over the file, or to rename the file. For example, if a file called patient.dat exists on your hard disk and you issue the following command:

lone-tar xv9 -Nw

you see the following output:

patient.dat, 7 blocks ^^^^^^^--> About to overwrite file!!
Size on hard disk : 3532 bytes (Last modified: 07/26/96 at 02:20) Size on tape : 3532 bytes (Last modified: 07/26/96 at 02:14)
Please choose: 'o' - overwrite the file * 's' - skip over the file and do not restore it 'r' - rename the file and restore it under the new name Enter choice:

lone-tar xv9 -zWHERE=/tmp ./etc/inittab

When this is done, the file you are looking for is /tmp/etc/inittab. If a file on the tape is listed in absolute pathname format (e.g. with a leading slash) as in /etc/inittab, it is still put into the requested directory (e.g. /tmp/etc/inittab). The -zWHERE option internally invokes the A option to strip off the leading pathname slash.
-zNOCNVT
No conversion of DOS files. This feature is used during a restore only and prevents files that have come from a DOS machine from being converted to the UNIX format while being restored. Normally, if the first 512 characters of a DOS file are printable ascii characters, the carriage returns are stripped out while being restored. When this occurs, the message "Dos textmode conversion" displays. When the -zNOCNVT modifier is used, the file is restored in its native DOS format. No conversion of any kind takes place.
-zDATEBEFORE=mm/dd/yy-hh:mm
Restores files that have been modified on or before the given date. This option requires an argument that specifies the exact date and time to be used. To restore all files on the tape modified on or before February 4, 1996 at 2:30 AM the following command is used (assuming tape device is device 9):

lone-tar xv9 -zDATEBEFORE=2/4/96-02:30

-zFLAT
This is the flat file restore. The directory part of the specified files is stripped, and just the filenames are restored. The filenames are restored to the current directory. If the
-zWHERE=dirpath option is used, the files are restored to the specified directory path. You must specify the pathname (directory name and filename) of the files you want to restore. This pathname can either be absolute or relative depending on how the pathnames appear on the tape listing. If you just specify the filename part only, no files are restored. Otherwise, files overwrite on top of each other (e.g. README).
For example, if you want to restore files on the tape in ./usr/u/meddata ending in "*.dat" to a new directory /u2/meddata, the following command is used:

lone-tar xv9 -zFLAT -zWHERE=/u2/meddata "./usr/u/meddata/*.dat"

The double quotation marks are necessary to prevent the shell from expanding the wild-card before it is evaluated by LONE-TAR.
-zTRIM14
Trims the length of restored files to 14 characters. This is useful on systems that support only 14 character filenames. Actually, the number 14 in -zTRIM14 can be replaced with any number greater than 7 allowing files to be trimmed to the desired length.
-zALL_TAPE
This causes LONE-TAR to search the entire tape for files to restore rather than quitting when it has found the first copy of each file. This modifier allows restore of files with multiple instances on the tape.

3.5 DOUBLE BUFFERING

The S (Speed) modifier initiates double buffering. This uses a complex scheme of shared memory buffers, semaphores and dualcantly speed up the backup. While one LONE-TAR process is busy compressing and packaging the data from the hard disk files into the shared memory buffers, the other LONE-TAR process is busy taking the data from the shared memory buffers and transferring it to the tape drive. Because shared memory is used, there is no wasted time spent copying the data in memory. Semaphores are used to coordinate the two processes as well as the shared memory buffers.
The size of each shared memory buffer is the same as that specified in the blocking factor (b modifier). Therefore the optimum size of the shared memory buffer array can be determined for your particular computer by varying the blocking factor and noting the backup speed displayed in the backup summary.
The S (Speed) option is fully integrated into the product so that all features can be used including compression to tape (P option), multiple volumes, splitting files (regular or compressed) across multiple volumes and even the use of dual sequential tape devices.
In general, the speed improvements are more impressive when compression is turned off and large blocking factors of at least 120 blocks or more are used. The speed increases are most significant for computers with multiple CPU's or those that do not already implement double buffering in the low level kernel tape device driver. In most cases, the double buffering implementation keeps your tape drive streaming during the entire backup.

3.6 DIAGNOSTICS

The diagnostics are very thorough with over 100 diagnostic, warning, or error messages. The user is notified if LONE-TAR cannot obtain enough memory for read/write buffers as well as compression buffers. Command diagnostics are available to remind the user of the main options and what LONE-TAR notifies the user if the file size has changed while it is being backed up and automatically adjusts for the new length while backing up. If the volume size is set and the end of the media occurs before the expected size given by the k factor, LONE-TAR notifies with the message "UNEXPECTED: NO MORE SPACE". Usually, LONE-TAR can recover from this and simply prompts for a new volume and continue backing up data. In any case, LONE-TAR suggests the correct volume size to use next time. If a linked file is restored and the file to which it is linked is not present on the hard disk, LONE-TAR notifies the user of what file to restore first. This makes the task of restoring linked files easier and helps resolve link dependencies.

3.7 ERROR RETURN CODES

LONE-TAR returns a diagnostic code indicative of what went wrong during the backup or restore process. A return code of 0 means the backup/verify/restore was successful. The following list shows the basic diagnostic return codes and their respective meaning:
0 - complete success 1 - error in command usage 2 - miscellaneous error, not otherwise defined below 3 - error reading from the floppy/tape device 4 - error writing to the floppy/tape device 5 - error opening or accessing a file or device 6 - error while reading a file from the hard disk 7 - error while writing a file to the hard disk 8 - Not enough memory available 9 - read/write error in the Virtual Pipe (P option) 10 - the header block of the file to be restored is bad 11 - interrupted 12 - error seeking on floppy/tape device 13 - error while verifying data 14 - byte level compare error during LEVEL 2 verification 15 - incomplete backup or restore 16 - error involving semaphore 17 - error involving shared memory 18 - error involving double buffering and split processes

It is important to distinguish these LONE-TAR diagnostic return codes from the UNIXsystem error codes. The lone-tar binary itself reports UNIX system error codes on the screen or in the catalog file. For example, ``lone-tar: Tape Write ERROR 5 occurred because I/O error'' is a common error message. The error 5 reported is a UNIX system error code that means there was an input/output error. The interpretation of the UNIX system error codes is found in the file /usr/include/sys/errno.h.

When you use ltmenu or receive mail from LONE-TAR .cron, the error reported is the return exit code of LONE-TAR. This LONE-TAR diagnostic return code, if not zero, represents a broad error category and its meaning is specific to LONE-TAR alone. Please see Table 22, "LONE-TAR Diagnostic Return Codes,'' for a more comprehensive summary of the LONE-TAR diagnostic return codes as well as the UNIX system error codes and the differences between them.

You can run the grep command on the following words in the catalog files to find out what the error was during a backup: WARNING, lone-tar:, Failed, is not.

3.8 COMPRESSION PERFORMANCE

Ascii text files: Usually 40-70% compression. Database files: Usually 40-85% compression (best for larger files). Object/Compiled programs: Usually 12-30% compression.

3.9 ERROR RECOVERY

3.9.1 Error Recovery During Backup

LONE-TAR has several levels of error recovery. If the hard disk is failing and one must do a backup before replacing the drive, most backup programs quit on the first bad file encountered. LONE-TAR switches into salvage mode and asks if it should continue. If so, LONE-TAR replaces the unobtainable data blocks with blocks of lower-case x's. In these cases, it is advisable to use the smallest block factor possible, since the entire block factor of data is considered unobtainable and replaced with lower-case x's. At the end of the backup, the number of incompletely backed up files are reported. If these errors occur when LONE-TAR is running as a background task, the yes answer is assumed and as much data as possible is backed up.

3.9.2 Error Recovery During Restore

LONE-TAR has two levels of error recovery on restoring data. The first is a hard error encountered while reading the media (bad sector or bad section of tape), and the second is a corrupt header block. Normally, there should be a header block of data preceding each valid file. This block of data contains information such as the filename, file size, permissions, and the date the file was last modified. If either of these two error situations occur, the user is prompted with the message:

Do you wish to attempt ERROR RECOVERY? (y/n):

lone-tar: directory not in proper format File header (name, size, date) is unobtainable This means the header information about the file has been corrupted or is not in the TAR format
Do you wish to attempt ERROR RECOVERY? (y/n):

lone-tar Pvbffkk 120 /dev/tape /dev/tape2 60000 40000 /

backs up the entire filesystem using dual tape drives, each with a different capacity tape. The dual drive feature requires that the devices use the same blocking factor. LONE-TAR supports dual drive specification using the numbers 0-99 to represent the devices. For example:

lone-tar MV7 -8 .

completes a Master backup of all files using device entry 7 then automatically switches to device entry 8 when device 7 fills up. The dual drive feature does not strictly enforce volume number checking on restore.

3.10 SPECIAL FEATURES

3.10.1 Backup Summary

LONE-TAR prints a summary of the backup process when the v or V option modifiers are invoked. This summary includes the date, the number of files backed up, the total size of all files backed up, the time taken to back up (not including volume switching), and backup speed. Additionally, the amount of room left on the current volume is printed.
Most importantly, the number of files that were not backed up is reported. Additionally, any files that were incompletely backed up (due to media hard disk errors) are also reported.

3.10.2 Restore Summary

This summary reports the number of files successfully restored. In addition, the number of files not successfully restored, as well as any files incompletely restored, are reported. Files incompletely restored include compressed files that could not be fully decompressed, virtual files that could not be fully revirtualized, regular files in which a medium error was encountered and only part of the data was recovered, cross-device linked files that could only be made using symbolic links, and regular links used in place of unsupported symbolic links.

3.10.3 Volume Switching

LONE-TAR does volume number checking on listing or restoring files. If the wrong volume is inserted, the user is notified and asked to insert the correct volume. LONE-TAR allows the listing or restoration to start with any volume of the set. For the special case in which one volume of a set has been lost, volume checking can be overridden by repeatedly inserting the wrong volume.
If bad media is inserted when switching volumes during a backup, LONE-TAR allows you to either try again, format the media (if applicable), or exit the program. A simple menu guides these choices. This is useful, for example, if you accidentally insert a write protected tape after several volumes have already been backed up. When formatting the media, LONE-TAR prompts for the actual command needed to format the media. After successful format, the backup resumes.
Sometimes you may wish to quit the backup when a volume prompt is encountered. In these cases, type q to quit.

3.10.4 Background Operation

LONE-TAR is aware when it is operating as a background process. This is determined upon starting and a message is displayed as follows:

Running as a background task

PLEASE REMOVE VOLUME: 1 INSERT VOLUME: 2 AND TYPE "y" WHEN READY! __ (Put your reply in "/dev/ltar_listen" ) (eg. echo y >/dev/ltar_listen )

echo "y" >/dev/ltar_listen

3.10.5 Signal Processing

LONE-TAR responds to the QUIT signal by immediate termination with a core dump. There is no special processing for this signal. LONE-TAR responds to the INTR signal (Interrupt - sent when the <DEL> key is hit) with a pop-up menu giving the user the choice of immediate termination, termination after the present file is complete, or returning to what it was doing before being interrupted. LONE-TAR responds to a kill command with the same pop-up menu as mentioned above. If the kill command is given to LONE-TAR when running in background, it immediately terminates.

3.10.6 Screen Input and Output

Under normal circumstances, LONE-TAR writes to standard error and receives input from standard input.
If standard input is already being used to pipe filenames or data to LONE-TAR, then the device /dev/tty is used for obtaining input from the user. Copyright information is always sent to standard error.
If the program is run as a background task from the Bourne shell or the Korn shell with job control turned off, the screen output is sent to standard output. Screen input is obtained from /dev/ltar_listen, in these cases, and the prompt for a response is followed by a short message telling you to put the reply in /dev/ltar_listen. This has been described previously in the section on Background Operation. If the program is run from cron or from an at job, the output is to the user's mail and the input is obtained from /dev/ltar_listen.
If the program is run from the Korn shell (with job control enabled) as a background task, the screen output is to the user's terminal and the input is to the user's terminal. The program is suspended by the Korn shell when it tries to read from the terminal (and in some cases when it tries to writThis is normal behavior for the Korn shell in this mode. In these cases, you need to use the job control features of the Korn shell to move LONE-TAR into the foreground, answer any questions, and then optionally you can move the program into the background.

3.10.7 Use of Wild-cards

The wild-card characters '?' and '*' can be used for pattern matching whenever a filename argument is used, in most, but not all cases. There are some limitations when using wild-cards to back up files. The '?' wild-card represents any single ascii character. This can include an alphabet character, a digit or a punctuation mark. The '*' wild-card represents any number of ascii characters. The '*' can be used at the beginning of a pattern, or at the end of a pattern, but not in the middle. Table 16 shows acceptable and unacceptable uses of the '*' wild-card. Table 16 shows acceptable and unacceptable uses of the '*' wild-card.
If an '*' appears in the middle of text, all the characters following the '*' are ignored.
The wild-cards can be used for pattern matching when specifying files to list, restore, exclude from a backup or restore, or exclude from compression. Wild-card expressions should be enclosed in either single or double quotation marks. This is so the shell won't expand the wild-card characters before they are passed to LONE-TAR. The following shows the proper use of a command to restore all files ending in '.wk3' (lotus files), but exclude those files in directories that represent figures from the 1990's.

lone-tar xV9 -E "199?/*" "*.wk3"

The exception to this rule requiring quoting is when specifying files to back up. In this case, the wild-cards should not be quoted because these need to be expanded by the shell before being passed to LONE-TAR. If the wild-cards are quoted, they are interpreted literally.
If you need to use wild-card pattern matching to back up select files, it is best to use the find command with a pipe to LONE-TAR. For example, if you want to back up all files ending in a '.c', the way to do this is as follows:

find . -name "*.c" -print | lone-tar CvF -

This searches the entire hard drive for files ending in a '.c' and backs them up.

3.10.8 Excluding Files From Being Compressed

Files can be excluded from compression in any of the following three ways: 1) Using the executable bit 2) Matching for a suffix 3) Pattern matching of file/directory names using wild-cards
All executable files (those with the execute permission bit set) are excluded from compression unless the Z modifier is used. Additionally, files can be excluded based on the suffix. By default the following suffixes are excluded:

Z .z .zoo .arc .zip .gz

An additional fourteen more suffixes can be excluded by setting the environment variable SUFFIXES. For example, to exclude files ending in either '.zzz' or '.lrc', the following command is used before invoking LONE-TAR:

SUFFIXES=".zzz .lrc"
export SUFFIXES

All suffixes that are excluded must start with a period.
Specific filenames or directories can be excluded by using the COMP_EXCL environment variable. For example, if you have many directories containing graphic images that are already compressed and the directory name always ends in either '.ict_dir' or '.pixl_dir' all these can be excluded using the following command before invoking LONE-TAR:

COMP_EXCL="*.pick_dir *.pixl_dir"

Up to twenty different combinations of wild-card combinations or specific file or directory names can be specified in this environment variable. Each one must be separated from the other with either a space or a tab.

3.10.9 Specifying Dates and Times

Dates and times can be specified using two different formats. Either of the formats can be used with the -zDATE or -zDATEBEFORE option modifiers. The first format involves the traditional MM/DD/YY format with an optional military time. This format is comfortable to most users residing in the United States. The complete specification is:

-zDATE=MM/DD/YY[-HH:MIN]

-zDATE=MMddhhmm[YY]

3.11 ENVIRONMENT VARIABLES

Table 19 shows all the possible environmental variables that are used to help specify or modify the actions of LONE-TAR.

3.11.1 VOL_SWITCH_CMD - Command to Switch Tape Volumes

This allows you to customize what happens when you are to switch tape volumes. This specifies the actual program that is run when the end of the tape volume has been reached. The program that is specified is usually one that has been written on-site and is specific for your computer and tape drives. Instead of prompting you to switch volumes, the program you specify runs instead. This is useful for large robotic backup devices that need special instructions. When this command returns it is assumed that the tape volume has been changed and that the backup or restore can resume. There is no volume prompt message or intervention required when this command is used. It is assumed that the command specified takes care of whatever operator prompting is required.

3.11.2 REMOTE_PUT - Network Command for Backing Up

This specifies the command to use to make contact with a remote tape device across the network. It can take two arguments that are represented by '%s'. These are replaced by the hostname and the tape device to which you are sending data. For example, if the following command is used:

REMOTE_PUT="rcmd %s dd of=%s obs=120 ibs=8k 2>/dev/null"
export REMOTE_PUT
lone-tar Cvf galaxy:/dev/rmt1 /etc

The actual pipe to connect to the tape device is:

rcmd galaxy dd of=/dev/rmt1 obs=120 ibs=8k 2>/dev/null

This allows for flexibility in specifying network connections.

3.11.3 REMOTE_GET - Network Command for Restore or Verify

This is similar to REMOTE_PUT above except that it is used as the actual command to obtain data from a network site for doing a restore or verify.

3.12 FILES

3.13 REMOTE TAPE SUPPORT

The only requirements for network backup are a TCP/IP network and proper access permissions to the tape drive on the host machine. It is not necessary to have an NFS network package.
LONE-TAR allows back up of any node on a network to a host machine and tape drive. LONE-TAR executes using the CPU on the node to be backed up and establishes a connection to the tape drive on the host. LONE-TAR behaves as usual except, instead of sending data to the local tape drive, the data is sent through a network connection to the remote tape drive. When compression options are turned on, the data on the local node is compressed and sent across the network, which reduces network traffic. Multiple nodes can be backed up to the same tape drive.
The following example illustrates a network backup. Let us say you want to perform a Master backup of your workstation and put the data on a tape drive on the host machine called galaxy. The name of the tape drive on the host machine galaxy is called /dev/rmt0. Suppose this tape drive has a 2 GB capacity. The command to do this from your workstation is:

lone-tar MVfbk galaxy:/dev/rmt0 20 2000000 .

LONE-TAR automatically makes the network connection to the tape drive /dev/rmt0 on the machine galaxy and performs the backup.
If there are several nodes to back up, you use the no rewind device /dev/nrmt0. In this way, each node on the network is backed up to a separate dataset of the tape. A simple shell script run from the host machine galaxy to back up the nodes called alpha, beta and delta is:

for node in alpha beta delta
do
rcmd $node lone-tar MVfbk galaxy:/dev/nrmt0 20 2000000 .
done

This makes each dataset on the tape contain the backup from each respective node. The first dataset contains the data from machine alpha, the second dataset contains the data from machine node beta and so forth.
Please note the block size on the remote tape is the same as that specified on the command-line. When dealing with tape drives using a variable block size mode (4mm and 8mm drives), you must use the same exact block size to restore the tape. Otherwise, you receive no data from the tape drive.

3.14 VIRTUAL FILE SUPPORT

Virtual files are files whose size is much greater than the actual amount of data contained in the file. This is because the data not accounted for is null data, e.g. binary zeros. Since the null data does not actually take up any real space on the filesystem, the file appears to contain more data than it actually does. The places in the file where the null data occurs will be referred to as black holes. The ability of a file to contain black holes of null data, yet not consume any additional space on the filesystem is a special feature of UNIX. It is not available under DOS.
If a virtual file is backed up without any special attention, all of the null data is read from the file and put on the tape. This is inefficient and wastes tape capacity if there is a lot of null data in the file. Furthermore, when the file is restored, the null data is also restored causing the file to consume much more disk space than it originally did. After the restore, the file is no longer a virtual file, but just an ordinary file with a lot of null data in it. Therefore it is wise to mark virtual files for special consideration before backing these up, so a restore puts the virtual file back just as it was and preserves the black holes.
Virtual files can be designated for special consideration using the environment variable VIRTUAL_LIST. This variable refers to the absolute pathname of a file that contains a list of all the virtual files that you wish to have backed up. The list should contain either the absolute or relative pathname of the virtual file, with one pathname per line.
Each file in this list receives special attention. The special attention consists of finding all the black holes and marking their location in the virtual file. Then this location information as well as the actual data in the virtual file is backed up. If compression is turned on, the actual data (but not the black hole location information) is compressed.
On restore of this virtual file, the black hole location information is first read into memory. Then the data is read into memory and restored being careful to put the black holes back exactly where they were in the original virtual file. This process is referred to as revirtualizing. If the virtual file is also compressed, the data is uncompressed on the fly and revirtualized in one streaming process so that no intermediate files are created.
It is suggested that the list of the virtual files be gathered and placed in the file /usr/lib/lone-tar/Virtual.lst. Then the environment variable VIRTUAL_LIST is set to this name.
A good way of finding the virtual files is by using the fsck command. If it reports the following:

fsck /dev/root

/dev/root
** Phase 1 - Check Blocks and Sizes
POSSIBLE FILE SIZE ERROR I=3042
POSSIBLE FILE SIZE ERROR I=8238
POSSIBLE FILE SIZE ERROR I=8596

ncheck -i 3042 8238 8596 /dev/root

3.15 BIT-LEVEL-VERIFICATION

This feature can be used with any TAR format archive. It compares files on the tape against those on the hard drive. The speed is about the same as that of the original backup. It is suggested that this be performed immediately after a backup. It is always done as a separate command. Thus, to back up a set of files and then verify them takes two separate LONE-TAR commands. If you put the letters TT in with the backup option letters, they are ignored.
Currently, Bit-level-verification is an all or none feature. In other words, you cannot specify particular files to compare. You can specify files to exclude, however. This can be done by using the E option or the TCNOCHECK environment variable.
When comparing files, LONE-TAR first looks to see if there are any differences in the dates or sizes of the files. If the file has been modified after the backup was performed it is flagged with the symbol <!Date> to signify that is has been modified. If a file has changed size, it is flagged with the symbol <!Size>. Files that have been modified are not considered errors. A file that has the same or earlier date and same size is bit-level compared against its counterpart on the hard disk. Any difference is flagged with the symbol &l;!Byte> and is considered a serious error. If there is a perfect match for every byte in the file, the symbol <V> is printed beside the filename so that the user is aware that there is a perfect comparison and verification. Any files that have been removed from the hard drive since the backup or are otherwise inaccessible are ignored. These are flagged with the symbol <!Not checked>. The CHANGEFILE contains the reason that any file was not checked.
For example, the output:

./tmp/Exaa00437, 2 blocks <!Not checked> ./etc/termcap, 13 blocks (compressed) <V> ./etc/group, 2 blocks <V> ./etc/passwd, 2 blocks <!Date> <!Size> ./usr/adm/messages, 12 blocks (compressed) <!Byte>

TCNOCHECK=/etc/NOCHECK; export TCNOCHECK

LONE-TAR would look in the file /etc/NOCHECK for a list of files to exclude from Bit-level-verification. This variable is only active during a Level 2 verification. If the file /etc/NOCHECK contained a single name ./etc/wtmp, then this file is excluded from Bit-level-verification. Since this file changes every time someone logs into the system, it is often different from the file on the tape. Most systems have a standard set of files that change on a minute to minute basis, and these are the files to put in this list.
The list should be structured so that there is a single file on each line. If your backup was done with a leading './' then the name of each file in the list should likewise start with a leading './'. If the list includes a name of a directory, the entire directory and all subdirectories are excluded from the verification. This list can contain wild-card patterns.
When a file is excluded, you see the message:

./etc/wtmp, 5 blocks <Excluded>

CHANGEDIR=/usr/lib/data; export CHANGEDIR

the detailed summary of files that change is kept in the file /usr/lib/data/ChangedFiles. If you then set the variable, CHANGEFILE as follows:

CHANGEFILE=doggy; export CHANGEFILE

the detailed summary of files that change are kept in the file /usr/lib/data/doggy.
If CHANGEFILE is set but CHANGEDIR is not set, then the file is kept in the /tmp directory. If the variable CHANGEFILE is set to the keyword "NONE", then no record of the changed files is kept. The output that would have been kept, goes to /dev/null.

3.16 COMPATIBILITY

LONE-TAR can read floppy/tape media created by any version of tar. LONE-TAR's error recovery capabilities can be used on any single or multi-volume tar archive. LONE-TAR's unique restore capabilities can be used on any standard tar archive. Most versions of tar on UNIX machines can read floppy/ tapes produced by LONE-TAR even across volumes. However, files compressed by LONE-TAR remain compressed. A file with a pathname greater than 100 characters in length cannot be properly restored by the UNIX tar program. Only LONE-TAR can properly restore this file.
LONE-TAR is compatible with DOS-TARTM a similar product under MS-DOS. All floppies (360K, 640K, 720K, 1.2Mb, 1.44Mb) are totally compatible and interchangeable. DOS-TARTM does automatic filename, pathname, and file conversion from UNIX to MSDOS. LONE-TAR does automatic filename, pathname, and file conversion from MSDOS to UNIX. DOS-TARTM can read any version of tar and transfer files to the MSDOS filesystem.

3.16.1 Limitations

The total length of the path and filename is limited to 170 characters. If a path exceeds this a warning message is printed, and the backup resumes. If a file is shortened while being archived and the device is non-seeking, LONE-TAR fills the difference between the new and old size with nulls. If the device is able to seek, LONE-TAR restamps the header block and automatically adjusts the size to the new size. To extract special character/block device files, LONE-TAR must be invoked by the super user. LONE-TAR does not support the u or w options.
The maximum capacity supported for a seeking device (floppy, magneto-optical) is two Gigabytes on platforms with a 32 bit CPU. The maximum supported size of a virtual file is two Gigabytes on platforms with a 32 bit CPU and 512 Megabytes on those with a 16 bit CPU. There is no limit to the number of files that can be backed up.

3.16.2 Internal Implementation Notes

LONE-TAR can handle an almost unlimited number of links or symbolic links during backup. This is due to a very space efficient method of storing these in memory. The ulimit (maximum file size) is checked and increased if need be. During data decompression there are no intermediate files. A compressed file is expanded in a streaming fashion as it is read from the archive media to the hard disk. This ensures that there is no disk fragmentation during the restore process. Data compression also occurs in a streaming fashion, with no intermediate files. The data compression and decompression functions are built directly into LONE-TAR with no reliance on external programs. Compressed installation archives created by SCO TAR are recognized and decompressed.

Main Navigation

Demo Downloads

Technical Articles

First to Market!

Watch instructional videos from:
- Smart phones
- YouTube.com
- Right here on Cactus.com