|
NuLib2 Manual |
|
| NuLib2 v3.0.0 Manual - By Andy McFadden - Last revised 2015/01/09Table of Contents
IntroductionNuLib2 is a command-line archive utility, along the lines of "PKZIP". It allows you to perform many common operations on NuFX archives, such as those created by the Apple II "ShrinkIt" utility, as well as Binary II archives. Files with extensions "SHK", "SDK", "BXY", "BSE", "SEA", "BNY", and "BQY" are handled. You can add, delete, extract, test, and list files in a NuFX archive. Compressed disk images can be extracted to files, and vice-versa, making it a handy utility to have when using an Apple II emulator. NuLib2 is the successor to NuLib, which did many of the same things. NuLib2 is not meant to set a new standard or fight for supremacy on the PC desktop; rather, it is intended to help people working with Apple IIs and Apple II emulators. All compression algorithms specified in the NuFX specification are fully supported. NuLib2 has a number of features not found in NuLib:
All features of the original NuLib are supported, except for a couple of really obscure ones. NuLib2 is built on top of NufxLib, a NuFX archive manipulation library.
Command OverviewCommands are specified like this:
There are seven commands: add files, list files (two variations), extract files (two variations), delete files, and verify archive integrity. There are ten modifiers, discussed in later sections. If you run nulib2 without any arguments, you will be presented with an identification banner and a command summary. The identification banner describes the current version of the NuLib2 application, and the version of NufxLib upon which it was built. The latest version of NuLib2 can always be found at http://www.nulib.com/. If you want a more detailed command summary, use:
If you wish to specify modifiers, you may lump them together with the command, like this:
or specify each independently, like this:
If you want to give your archive a name that starts with a hyphen, you will
have to specify it as a full or partial path, e.g. "
Some commands require a list of filenames. These must be listed after the archive name, e.g.:
The names given will be compared with records in a case-insensitive fashion, so asking NuLib2 to extract "foo" or "FOO" would match on "foo", "FOO", and "fOo". The commands are explained next.
Listing Archive Contents (-v, -t)The contents of NuFX archives are listed in a format similar to what ProDOS 8 ShrinkIt 2.x displays. If you use the command:
you will see output similar to this: archive.shk Created:22-Aug-90 15:33 Mod:22-Aug-90 15:33 Recs: 4 Name Type Auxtyp Archived Fmat Size Un-Length ----------------------------------------------------------------------------- BUG.REPORTS TXT $0003 22-Aug-90 15:33 lz2 32% 3089 FINDER.DATA FND $0000 22-Aug-90 15:33 unc 100% 458 +HELLO BAS $0801 22-Aug-90 15:33 lz2 86% 605 GIF.SYS16 S16+ $0000 22-Aug-90 15:33 lz2 31% 39165 ----------------------------------------------------------------------------- Uncomp: 4152 Comp: 1988 %of orig: 47%For each file, the display includes the filename, ProDOS file type, ProDOS "aux" type, the date and time when the file was added to the archive, the compression format, a compression ratio percentage, and the length of the file before it was compressed. The first line has some information about the archive, and the last line has a summary of file sizes and compression performance. A "+" in the leftmost column, in front of the filename, indicates that the file is "locked". NuLib2 considers a file to be locked when it has the ProDOS write, rename, and delete permissions disabled, but still has read permission enabled. All other files are considered to be unlocked. A "+" is shown next to the file type if the file is "extended", meaning it has a resource fork. A "-" next to the file type indicates that it's an empty file stored improperly due to a bug in GSHK. The listing for a disk image is similar: prodisk.shk Created:21-Apr-90 15:11 Mod:21-Apr-90 15:11 Recs: 1 Name Type Auxtyp Archived Fmat Size Un-Length ----------------------------------------------------------------------------- TEST Disk 140k 21-Apr-90 15:11 lz1 37% 143360 ----------------------------------------------------------------------------- Uncomp: 143360 Comp: 53051 %of orig: 37%The file type is displayed as "Disk", the "aux" type is the size of the disk image in KBytes, and -- if the image is for a ProDOS disk -- the filename is the disk volume name. The "Fmat" column will be one of "lz1", "lz2", or "unc", indicating ShrinkIt LZW/1, LZW/2, or uncompressed. Files added to archives by NuLib2 always use LZW/2 or, if compression fails, are stored uncompressed. LZW/1 is used by P8 ShrinkIt and the original NuLib. The filename field will show the full name, including all subdirectories leading up to the file. If there isn't enough room to display the entire filename, it will be truncated on the left, replaced with two dots. The following example has five files from the "gno:user:man:man2" directory, one of which was too long to fit in the display: Name Type Auxtyp Archived Fmat Size Un-Length ----------------------------------------------------------------------------- gno:usr:man:man2:sigblock.2 GWP $8010 01-Dec-97 00:07 lz2 58% 2667 gno:usr:man:man2:signal.2 GWP $8010 01-Dec-97 00:07 lz2 56% 7036 gno:usr:man:man2:sigpause.2 GWP $8010 01-Dec-97 00:07 lz2 61% 2358 ..usr:man:man2:sigsetmask.2 GWP $8010 01-Dec-97 00:07 lz2 56% 3046 gno:usr:man:man2:wait.2 GWP $8010 01-Dec-97 00:07 lz2 51% 6577 -----------------------------------------------------------------------------You can also list the contents of an archive like this:
The 't' command generates a simple list of filenames: BUG.REPORTS FINDER.DATA HELLO GIF.SYS16The filenames are never truncated or embellished, which makes this command useful when you're searching for a specific file.
Creating Archives and Adding Files (-a)You can add files to a new or existing archive with the "-a" command. If the archive you specify does not exist, a new one will be created. All files will be added to the end of the archive. If the name of the file being added matches the name of a file already present in the archive, you will be allowed to replace the existing file or skip adding the new file:
If you respond with "y", the existing file will be deleted when the archive is updated. If you say "n", the new file will be skipped. Entering "A" or "N" will cause NuLib2 to automatically enter "y" or "n" respectively on any future conflicts. (You may also hit 'q' here to abort the operation and quit.) Files in subdirectories will be added with whatever path separator is appropriate for the current system. On a UNIX-like system that would be '/', while under Win32 it's '\'. GS/OS follows the Mac OS convention and uses ':'. There are a number of modifiers that can be used with this command.
While adding files, NuLib2 displays the name of the file as it will appear in the archive, along with a completion percentage: DONE compressing BASIC.System DONE storing BOOT.GSOS DONE compressing BOOTU3 DONE storing COMM/FINDER.DATA DONE compressing COMM/ProTERM/PT3.SYSTEM DONE compressing COMM/ProTERM/PT3 DONE compressing COMM/ProTERM/PT3.GLOBAL DONE storing COMM/ProTERM/PT3.WELCOME 15% compressing COMM/ProTERM/PT3.CODE0 Files stored without compression will say "stored", while files compressed with LZW/2 will say "compressing". NuLib2 tries to mimic GS/ShrinkIt as closely as possible, so files shorter than 512 bytes are never compressed, and files that don't get smaller are stored uncompressed.
Extracting Files (-x, -p)Files can be extracted with the "-x" and "-p" commands. You may specify a list of files to extract, or specify none and extract all files. A simple example that extracts all files from an archive:
If you wanted to extract a file called "fubar" and a file called "ack:splat", you would use:
The directory hierarchy is always preserved unless "-j" is set. In the above example, the files would be extracted as "fubar" in the current directory and "splat" in a subdirectory called "ack" (assuming that the file was archived on a system where ':' separates directory names). Using the "-r" flag (described below), you could extract all of the files in the "ack" subdirectory, like this:
The "-r" flag does a prefix string match, meaning it just compares the first part of the name in the archive against the name you specify. So if you had entered:
above, it would have extracted "acknowledge" and "acksent" as well as "ack:foo" and "ack:bar". Specifying the filesystem separation character (usually '/' or ':') allows you to grab just the directory you want. If you try to extract a file with the same name as an existing file, you will be prompted for instructions:
You can choose (from left to right) to overwrite the existing file, skip the extraction of this file, overwrite all existing files, never overwrite an existing file, or rename the current file. If you elect to rename, you will be prompted for a new name for the file. (You may also hit 'q' here to quit immediately.) The modifiers usable with the "-x" command are:
While extracting files, NuLib2 displays progress information: DONE expanding BASIC.System DONE extracting BOOT.GSOS DONE expanding BOOTU3 DONE extracting COMM/FINDER.DATA DONE expanding COMM/ProTERM/PT3.SYSTEM DONE expanding COMM/ProTERM/PT3 DONE expanding COMM/ProTERM/PT3.GLOBAL DONE extracting COMM/ProTERM/PT3.WELCOME 90% expanding COMM/ProTERM/PT3.CODE0The filenames shown are the names as they are being written to disk. For example, if you extracted files with "-e" (preserve types) and "-l" (convert end-of-line character) set:
You would see something like: DONE expanding BASIC.System#ff2000 DONE extracting BOOT.GSOS#fc0801 DONE expanding BOOTU3#060800 DONE extracting COMM/FINDER.DATA#c90000 DONE expanding COMM/ProTERM/PT3.SYSTEM#ff2000 DONE expanding COMM/ProTERM/PT3#ff2000 DONE expanding + COMM/ProTERM/PT3.GLOBAL#040000 DONE extracting+ COMM/ProTERM/PT3.WELCOME#040000 90% expanding COMM/ProTERM/PT3.CODE0#060000The "+" sign indicates that an EOL conversion was performed on the file.
The "-p" command extracts the files to a pipe. The contents of the extracted files are written to stdout. The normal status messages are suppressed. Only the "-r" and "-l" modifiers can be used with "-p". This command is useful for examining the contents of individual files. For example, the command:
will dump the contents of "document.txt", with text automatically converted, and pipe the results into "more". If you specify more than one file, they will be sent to the output one after the other.
Verifying Archive Integrity (-i)If you want to verify that an archive is undamaged, use this command. NuLib2 does the same processing that it does when extracting files from the archive, but no output is produced. Every CRC (Cyclic Redundancy Check - a checksum computed when the data was added to the archive) is tested. The progress information shown when testing an archive looks like this: DONE verifying BASIC.System DONE verifying BOOT.GSOS DONE verifying BOOTU3 DONE verifying COMM/FINDER.DATA DONE verifying COMM/ProTERM/PT3.SYSTEM DONE verifying COMM/ProTERM/PT3 DONE verifying COMM/ProTERM/PT3.GLOBAL DONE verifying COMM/ProTERM/PT3.WELCOME 90% verifying COMM/ProTERM/PT3.CODE0 If a problem is found, you will see a message like this one: Found a bad CRC in BASIC.System Archive may be damaged, continue anyway? [y]es, [n]o: If you choose 'y', NuLib2 will pick up where it left off, and continue processing the archive. If you choose 'n', NuLib2 stops immediately. If you don't specify a list of files:
then NuLib2 will test every file in the archive. If you use a list:
it will only test the files you told it to. You may use the "-r" modifier when specifying files to test, in the same fashion as when extracting files.
Deleting Files (-d)This command deletes entire records from an archive. A list of filenames must be specified. If you manage to delete all of the files, the archive itself will be removed. An example of this command:
A progress report will be displayed during processing, e.g. Deleting foo Deleting bar Deleting ack/splat The "-r" modifier may be used to select entire subdirectories. See the notes in the section on extraction, above.
Disk ImagesNuLib2 doesn't try to write floppy disk images onto a floppy the way ShrinkIt does. Instead, it extracts disk images as ProDOS-ordered files, suitable for use with your favorite Apple II emulator. When adding disk images, NuLib2 assumes that the file is in ProDOS block order. If you want the archive to work correctly with ShrinkIt on an Apple II, don't add DOS-ordered disk image files. The NufxLib library comes with a sample program called "imgconv" that can convert between 2IMG (".2mg") images and NuFX (".shk") archives. It is able to convert DOS-ordered .2mg files to ProDOS order before adding them to the NuFX archive. NuLib2 will only accept files that are a multiple of 512 bytes. If you insist on trying to add odd-sized files as disk images, NuLib2 will print a warning and add it as an ordinary file.
Binary II ArchivesNuLib2 supports Binary II (.BNY, .BQY) archives in a more or less transparent fashion. For all operations except adding and deleting files you can specify a .BNY file in place of a .SHK and get more or less the same results. A few modifier flags aren't currently supported for Binary II archives. You can't specify EOL conversion with "-l", freshen or update with "-f" or "-u", or extract comments (there are no such things) with "-c". You can, however, specify filenames using "-r" for partial matches, use "-e" and "-ee" to extend filenames, and use "-j" to truncate paths. The default behavior is to refuse to overwrite existing files. NuLib2 currently just stops if it encounters a file that already exists. You can use the "-s" flag to tell NuLib2 to overwrite any existing files. It will not, however, overwrite directories with files or files with directories. Files that were compressed with "Squeeze" compression (via BLU v2.27 or SQ3) will be automatically un-squeezed. If the filenames end in ".QQ", the extension will be stripped. Programs like ShrinkIt and BLU use the filename embedded within the SQ format as the output name when extracting, but NuLib2 ignores that because it tends to leave you with a filename you weren't expecting. In some cases you will need to explicitly tell NuLib2 that a file is a Binary
II archive with the "-b" flag. If you want to strip the Binary
II header off of a .BXY file, you will need to use "
MiscellaneousFiles that were somehow archived without a filename will be referred to as "UNKNOWN". Rumor has it some older versions of ShrinkIt omitted the file name for DOS 3.3 disk images. On Win32 and UNIX-like systems, the ProDOS access permissions are not preserved precisely. ProDOS has read, write, rename, and delete permission, as well as "backup needed" and "invisible" flags. If NuLib2 believes the file is locked, it will disable write permission on the file. When adding files, "locked" files will have read and "backup needed" enabled and the other flags disabled, while "unlocked" flags will have all the flags except "invisible" enabled. In other words, the basic concepts of "locked" and "unlocked" are preserved, but the full set of flags is not. The NuFX specification forbids filenames that start with the filename separator character, i.e. you can't put "/foo/bar" in an archive. If you specify a full pathname, the leading '/' will be dropped. On Win32 and UNIX-like systems, a leading "./" in the filename is redundant and will be stripped. If ".." is used as a path component, the pathname will be reduced to just the filename. When making changes to an archive, a new archive is constructed in a temporary file ("nulibtmpXXXXX") in the current directory. When everything completes successfully, the temp file is renamed over the original archive. The exception to this is that newly-created archives are written in place. If NuLib2 crashes or is killed with signal while modifying an archive, the temp file may be left lying around. Filenames with invalid characters will be converted to something palatable for the current system (e.g. "/" is set to "_" on UNIX-like systems). If file type preservation is enabled, the character will be preserved exactly (e.g. '/' becomes "%2f"). The character set used for filenames in ShrinkIt archives is Mac OS Roman; this is the character set used on the IIgs and old Macintoshes (and, consequently, on HFS disks). The Linux and Mac OS X versions of NuLib2 will automatically convert between Mac OS Roman and Unicode when adding, extracting, and listing files, using UTF-8 encoding. NuFX archives store three dates with every file: creation, modification, and when it was archived. On systems that don't have creation dates, the modification date will be substituted. There are certain filenames you can't use on a Windows "FAT" filesystem, such as "AUX" and "PRN". Neither Windows nor Linux's vfat driver will allow it. Standard utilities like WinZip fail with a mysterious error message. As a workaround, the Win32 version of NuLib2 will consistently prefix all MS-DOS device entries with '_', so "AUX" and "aux.foo.txt" will be extracted as "_AUX" and "_aux.foo.txt". If filetype preservation is enabled, the name used will be "%00AUX" (the %00 is removed when the file is added with -e, thus preserving the original name). This handling does not apply to versions built for other systems, so attempting to extract a file named "AUX" onto a FAT filesystem under Linux will cause NuLib2 to fail. No attempt is made to extract and preserve comments, even in "preserve" mode. This is probably a bug. Archive files that start with junk -- such as a vestigal MacBinary header or HTTP headers -- appear occasionally on FTP sites. NuLib2 will search through the first 1024 bytes of the file to find the actual archive start. Silly benchmark of the day: creating a 14MB archive containing the contents of my hard drive took about 40 minutes on an accelerated IIgs. NuLib2 accomplished the same feat in about six seconds on a 500MHz Pentium-III running Linux.
AcknowledgmentsNuLib2 would not have been possible without all the lessons learned from NuLib. Andy Nicholas, Kent Dickey, Frank Petroski, Robert B. Hess, Bruce Kahn, and Devin Reade contributed code to NuLib's development. My original plan for preserving ProDOS file types was, in retrospect, mighty screwed up. My thanks to Bill North for setting me straight. Eric Shepherd spent a bunch of time messing around with early versions of NuLib2 on BeOS, and helped me get all the configuration stuff in order.Devin Reade built it on several different platforms, and made a repository for binary distributions. This document is Copyright © 2000-2015 by Andy McFadden. All Rights Reserved. The latest version can be found on the NuLib web site at http://www.nulib.com/. |