|This article was originally written in 1997 - virtually at the dawn of consumer CD recording. It is, at you might expect, a little dated. However, there are still some excellent points made here and anyone writing to CDs or DVDs should take heed of the cautions here.|
Dana J. Parker
Originally published in EMedia Professional
April, 1997, pp. 71-81
Copyright © 1997, Dana Parker All rights reserved.
This copy has been placed by InfinaDyne with permission from the author
Today, CD-Recordable has truly come into its own in terms of reliability and ease of use, but caveats still apply.
Several products emerged that were designed to address just this issue, including Spira's Moniker, which promised drive-letter access to CD recorders, making writing to CD-R as transparent a process as writing to a hard drive or floppy. More recently, products and technologies offering so-called incremental or packet writing schemes, such as JVC/Gutenberg Systems/Smart Storage's CD-R Extensions, Sony's CD-RFS, and most recently Adaptec's DirectCD have promised to eliminate the traditional woes of CD recording by circumventing the dangers associated with writing virtual images on the fly. But the risks remain for "serious" CD-R users, who are likely to use their CD-R systems for multiple purposes. Packet writing may prove to be a boon for internal backups and archiving, but discs intended for distribution and as input for replication still need to meet the old standards, and the old standards come complete with the risk of the old buffer underrun.
Many of those who have purchased CD-R systems for personal or business use are finding that recording a CD is not as simple a process as recording a floppy. For instance, some of these CD-R novices, after installing a CD recorder, may try to record a backup copy of their fragmented hard drive, complete with non-ISO 9660 filenames, several other programs running in the background, and a screen saver activated. Such would-be CD recorders soon find themselves the proud creators of their first-ever gold "coaster," the industry's popular term for unusable CD-R discs. And many will wonder why nobody ever told them that at most levels, CD recording remains not quite a plug-and-play proposition.
Just as experienced chefs don't have to follow a recipe, experienced users of any technology become so inured to its little idiosyncrasies that they make allowances for them without thinking. To someone who takes it for granted that a severely fragmented hard drive can sabotage the most basic CD recording session, the wisdom of running a defragmentation utility and turning off the screen saver before recording can seem as self-evident as the necessity of stirring constantly to prevent the formation of lumps in gravy. The result of this familiarization is the tendency of the experienced user to do certain things so much by rote that these very important, basic techniques are rarely mentioned, much less stressed in the instructions given CD-R users. And even when good instructions are supplied, there is a tendency on the part of new users to place less importance on such instructions, even when supplied, if no adequate reason for their advisability is given.
Every experienced CD-R user develops his or her own habits in recording CDs. Some of these habits are acquired the hard way, as a result of disastrous experiences. Others are cultivated to provide "insurance" against potential disaster and are holdovers from CD-R's early days, when recording a 650MB coaster wasted an hour and an $80 piece of media.
Today, CD-Recordable has truly come into its own in terms of reliability and ease of use, but caveats still apply. Even with the advent of packet writing, whose proponents say it will make buffer underrun history, the risks remain. Because CD-R discs intended for distribution or to be used as input for mass replication will enjoy greater cross-drive and replicated compatibility from being recorded the "old-fashioned" way, packet writing will not replace disc-at-once or even multisession recording anytime soon.
Ask 20 experienced CD-R users about their favorite tips for successful recording--given persisting risks and potential stumbling blocks--and you'll get 20 different answers, but each user's list will include, in one form or another, seven basic rules, as follows:
Successful CD-R users know that ignoring these rules does not necessarily guarantee failure. But given the vagaries of fortune and the ever-narrow margin for error, why not aspire to perfect CD-R practice, if only to be on the safe side?
One well-known area of inflexibility for CD recorders is their inability to tolerate delays or interruptions in the stream of data channeled to them from the CD-R software's specified source. Data must be supplied in a constant flow because the data recorded on a CD-R disc cannot be picked up from the source and laid down on the target platter byte for byte in its original configuration. The process of moving data from a magnetic medium to CD-R requires that the data be converted, interleaved, error-corrected, and rendered redundant, and all of these operations must take place between the time the data leaves the host system and the moment the actual laser pulse creates optical marks on the disc.
Every CD recorder includes a buffer, which is a kind of staging area for data to be written to disc; if for some reason the buffer becomes empty because data is leaving it faster than data is entering it, a buffer underrun will result. If the buffer becomes empty and leaves the drive with nothing to be recorded to the disc, the interleave ramps down and the recorder loses its train of thought, so to speak, and produces a bad disc.
There are two basic ways to prevent this problem: make sure your recorder has the biggest buffer money can buy and the hardware supports, or do everything you can to make sure that the hard drive streams data to the recorder in a fast, steady, and uninterrupted manner. The best approach is to do both, but after the purchase is made, the only thing to do is to concentrate on the second strategy. Buffer underrun is the most common single point of failure in CD recording, and thus it is only wise and proper that five of the seven rules of safe and sound CD-R use are strategies for avoiding buffer underruns.
Data is recorded on a CD-R disc in a continuous spiral track. Not only are sectors laid down end to end, but the ISO 9660 format requires files to be laid down in a series of uninterrupted, contiguous, single-extent sectors. The source for most CD-R discs, however, is a magnetic hard disk, which allows files to be fragmented and stored on the disk in many, sometimes widely divergent, locations. No matter how fast the hard drive, forcing the read head to scavenge numerous locations on the hard drive for pieces of large files causes unnecessary delays. Successful CD-R users defragment their drives religiously.
"I have gotten into the habit of running ScanDisk and defragmenting my hard drive once or twice a week," says Deirdré Straughan of Adaptec, proprietors of Easy-CD Pro, CD-Creator, and DirectCD CD-Recordable software. "Regular defragging has eliminated the recurring buffer underruns which seem to have been related to some bad sectors on my hard drive."
Jerry Warner of CD-R service bureau CD Solutions also testifies to the efficacy of frequent defragmentation when copying data from a hard drive onto CD-R. "I always run Norton Utilities' Speed Disk before I burn a disc because it makes things easier on my equipment," Warner says. "When you record a disc using the Macintosh, the physical structure on the resulting CD will match that of the hard drive. If you don't run Speed Disk first, your CD-ROM drive will spend a lot of time changing speeds as it moves around the scattered CD you've created looking for file fragments." Warner adds, "A disc written from a defragmented hard drive will also allow you to maximize the storage potential on the CD because you won't have any 'holes' in your data."
Some CD-R power users recommend reserving a partition on a hard drive for the sole purpose of staging a disc image, rather than recording "on the fly" from a virtual image scattered over a large hard drive.
Dave Grimes, a San Diego, California-based independent CD-R consultant, is an enthusiastic proponent of performance-optimizing hard drive partitioning. "Most high-density hard drives these days are zoned into seven or more zones, and RPM being constant, throughput increases or decreases as expected," Grimes says. "So we see drive specs like 4 to 7MB/sec average transfer rates."
Grimes says the first step for users is to understand the extent to which fragmentation can stymie a write session if a multizoned hard drive is partitioned into a single file system and the data clusters are scattered willy-nilly on the hard disk platter. Grimes says he solves the problem by partitioning the 2GB hard drive he uses in part for staging CD images into multiple file systems, "not only to improve the cluster factor--which has always been a weak part of hard drive file systems--but also to tailor the performance of a set of files based on usage."
Grimes says his hard drive contains four 256MB file systems, all located near the inside of the disc, and one 1GB file system which he says is "deliberately placed at the outer cylinders where RPM gets you more data. Long sequential transfers [such as those used in writing to CD-R] get a boost from this arrangement," Grimes says. "The placement of the whopper partition is ideal for my CD-R work area."
Paul Crowley of CD Productions, a Buffalo Grove, Illinois-based CD-R service bureau, approaches the fragmentation problem from a different angle. "Most of my work is transferring files to CDs," Crowley says. "The usual source is tape, either DAT or QIC. I load the files to be put onto CD into a clean partition on my drive. This approach eliminates most of the problems with fragmentation and the like."
When writing to CD-R, users of CD-Recordable software are typically given two choices for ordering the data selected for writing: creating a real image or a virtual one. Creating a real image favors recording success by easing the burden on CD recording hardware and software. While a virtual image--which consists of a look-up table of addresses of the files to be written to CD-R--saves hard drive real estate, it increases the possibility of a buffer underrun by adding one more chore to the process of moving data from magnetic disk to CD media.
A real image exists on the hard drive in ISO format, so that the recording process only has to interleave, add redundancy, and calculate error correction. Cross Interleaved Reed Solomon Code (CIRC), which is the most basic level of CD error correction, employs two methods to detect and correct errors, redundancy, and interleaving. CIRC uses about 25 percent data redundancy. The data is laid out in a way that will allow errors to be corrected even though the data is not 100 percent redundant; it uses a parity checking algorithm to reproduce data that is unreadable. In addition, the data is interleaved and distributed over a large physical disc area.
A virtual image forces the recorder and software to look up the file's location, access it, translate data in the operating system's native file format into the ISO 9660 file format, and add redundancy, error correction, and interleaving in the same time frame.
UCLA engineer Ami Fischman is among those CD-R users who align themselves with the "real image" approach. "When I create a real ISO image first, I almost never get buffer underruns, although burning from a virtual image that may be scattered throughout a hard drive yields about a 15 percent failure rate, which is unacceptable at $8 per disc." Fischman suggests staging the data to be recorded in a virtual image from a live file system on a magneto-optical drive, such as the MaxOptix unit she uses, whose 650MB per side matches the size of a "74-minute" CD-R disc. "This method seems to work better than using a live system off the hard drive," she says, "because the operating system isn't performing any other operations with the M-O drive during the burn. I've only had one failure using the M-O, after about 12 CDs, and that could have been easily attributed to human error."
CD Solutions' Warner takes a different approach. "I very seldom build an image file first," Warner says. "I never build a real image on the Macintosh, but I will on the PC if I notice there are a lot of small files or if it takes a long time to build the virtual image." When a virtual image requires a long time to build, it means that there are a lot of lookups and recordings of file addresses going on--too many, perhaps, to risk using a virtual image. A complicated directory structure with many small files puts a heavy load on the system that already faces the burden of ISO 9660 file formatting.
Once upon a time, nobody recorded CD-R discs without doing a write test every time. When a full disc required an hour or more to record, and each piece of media represented an $80 investment, doing a write test, even if the test itself took a full hour, was just something you did. Now, write tests can be run at 2X or 4X speeds, which proportionately reduces the time required to test a full disc's worth of data and reduces it further for smaller sessions on a multisession disc. Still, new users ask, "Why do a write test? I did a speed test, and my system checks out as sufficient for 2X recording."
The answer is that while a speed test checks the ability of your hardware--hard drive, SCSI bus, and CPU--to deliver data fast enough, it does not take into account the structure of the data to be sent across. Even a real image on a partitioned, defragmented hard drive may be bogged down by many small files, generally, or too many files in a directory, or too many deeply nested, complicated directory structures. These factors not only slow the transfer rate across the SCSI bus, they can add overhead in the form of file and directory entries in the ISO 9660 format, bulking up your dataset, which can even result in the dataset being--when prepared for writing to CD--too large to fit on CD-R media. Such "annoyances" are undetected by speed checks--you can't know if they're going to be a hindrance until you try to send them across the SCSI bus to the recorder.
"I often make multiple copies of a single set of files, so I don't do a test write for every set," says CD Productions' Crowley. "However, I usually do a test write for the first CD of a set, just to make sure. This sometimes reveals problems that aren't obvious, such as the dataset being too large to fit on the CD."
Adaptec's Straughan also suggests testing the waters extensively before committing data to CD. "When writing data to CD-R with Adaptec's Easy-CD Pro, I always do a complete write test," Straughan says. "Because my system is only marginally optimized for CD recording, I never know when some tiny factor will cause a 2X write to fail. Adding a write test to the process takes longer in the short run, but it also saves on blown discs."
Can a user's desktop system ever be perfectly optimized, stabilized, and considered reliable for CD recording?
However, if you cannot refrain from upgrading your operating system, resetting your cache, bumping up or down your available RAM, installing a new sound or video playback card, connecting a digital camera, or adding a new hard drive or other external storage device, you can make an effort to recall, when something goes wrong, what you have changed since the last time the process worked. You can also record--i.e., write down--the configuration that seems to afford the most consistent success rate. That means that you can always go back to what worked when your latest system tweak results in an unpredictable and unwanted outcome.
Albert Monastirsky of C & I, a Buenos Aires, Argentina-based service bureau and distributor of CD-R hardware, describes a Windows 3.x system ideally configured for CD-R as one optimized for lower memory, and using a scanned and defragmented hard drive. Under Windows 95, CD-R performs better, he says, if the system is started without CONFIG.SYS and AUTOEXEC.BAT. The best course of action, he says, is to "use a good premastering software and not change the system configuration just before proceeding to record without first running whatever performance test is available in the premastering software."
CD-ROM Productions' Crowley also suggests erring on the side of caution. "I heartily recommend overkill as far as the system configuration is concerned," he says, "at least with respect to what I have heard many people using. I have all bus mastering SCSI controllers and a Wide SCSI hard drive. The CD recorder is also on a separate SCSI adapter from the hard disk. The benefit to this is a transfer rate around 1MB/sec from the partition I record from," he says. Using this configuration, he says that
he has never had a buffer underrun problem.
Adaptec's Straughan is less bullish on the prospects for system optimization where CD-R is concerned. "My suspicion is that no system is truly stable, even if you can resist the temptation to add, subtract, and tweak once it's up and running," Straughan says, arguing that things deteriorate and change on their own. "The hard drive is a big factor in CD recording, and that, of course, is constantly changing, if only in terms of the data it contains. I have a CD-ROM drive as well as a CD recorder on my SCSI bus, and I recall one occasion when I realized that having a disc in the CD-ROM drive while I was recording was causing random SCSI errors. So now I make sure there's nothing in the CD-ROM drive, unless I'm copying from CD to CD," Straughan says.
Windows users have become accustomed to doing their computer work on systems that can keep several applications open at one time. Two keystrokes take you from a database to a word processor to a CD-ROM reference application. If no activity takes place on the keyboard for a certain period, a screen saver pops up automatically.
All of these concurrent applications take their toll on the available system resources. While in the process of recording a disc, your word processor may elect to perform the autosave you scheduled for every 20 minutes. Your screen saver, noticing no keyboard input for the last five minutes, suddenly
kicks in with a starfield simulation. Your phone rings, and the fax software you set to auto receive picks it up and displays an image of the incoming fax. Each of these processes requires accessing the hard drive, right in the middle of what was supposed to be a fast, steady, uninterrupted stream of data flowing from your hard drive to the disc in your CD recorder. When you record to CD-R, the recording software should be the only application running. CD-R power users have become accustomed to looking at recording status displays instead of starfields when there's a disc in the oven.
“When I launch Easy-CD Pro," Adaptec's Straughan says of the company's software tool, "I shut down every other application. A few people have told me they get away with recording in the background while doing other things, but I wouldn't risk it with my current system."
CD-ROM Productions' Crowley also dismantles background operations for CD recording. "I turn off the screen saver and terminate everything else on the computer before starting the actual burn. I think my configuration can handle some activity, but I'd rather not chance it."
Al Foster of DataDisc, a CD-R reseller and service provider, says there are other resident programs that run in a system's background that, while unapparent to users, may impede the recording process. "Turn off the virus checker if is memory resident," Foster advises. The extra time that the virus software takes in the background, Foster says, can sometimes interrupt the flow of data to the CD recorder and thus cause problems that may lead to buffer underrun. "Visually check to see that the virus software didn't leave behind its validation files. If it did create some of these files, I remove them."
Contrary to what you might expect, it's possible to record an unreadable disc without ever running into a buffer underrun, and users are just as susceptible to turning a piece of media into a coaster after recording as before or during the process. Power users don't take it for granted that the disc just recorded is done and done right; they know that the time to determine the reliability of the recorded disc is right away, while the source is unchanged and the system retains the settings and configuration that may have caused the problem.
After the burn is the time to label discs, too, and to label them correctly. For instance, many a new CD-R user has inadvertently ruined a perfectly good disc by using a ballpoint pen, which etches a human-readable "label" into the machine-readable data layer, thereby destroying the data layer and leaving the CD-R media with a legibly labeled but unreadable disc. Labeling while discs are still warm from the recorder is an excellent habit to acquire. Waiting may not ruin the disc but the sooner a disc is labeled, the less likely it is to be lost. If labeling does damage the disc, doing it right away is a good idea for the same reasons cited for testing a disc immediately; if the disc is ruined, you can make another right away.
In fact, most CD-R service bureaus that deal with dozens or hundreds of discs a day make it a point to label discs before the burn, and they don't consider the process of recording a success until the disc has been tested. "We print the label on the disc before the disc is burned to cover for any damage such as a scratch that might happen during the printing process," says CD Solutions' Warner. "By printing first, we're able to use the error correction abilities of the premastering software to correct any damage that might have occurred." Warner also emphasizes the importance of testing discs created for their viability as premastering sources. At CD Solutions, they run Clover Systems' Disc Detective CD testing utility on the CDs after the burn to check for defects that may emerge in the recording that can't be seen or that the premastering software fails to catch. "We let the CD-R disc cool down first from the burn just in case something changes during that stage," Warner says. "I don't know that it makes any difference, but by letting it cool down first it removes one variable from the problem list."
Warner says poorly structured or error-riddled discs, whether plagued by "gaps" in the data or scratches or ink specks on the polycarbonate, have caused CD readers to drop to data transfer rate levels that are below the company's standards while being tested in Disc Detective. In such cases, Warner says, "The disc may be readable but performance wouldn't be acceptable at that point on the disc. Having run many of the discs sent to us by our customers for mass replication, it looks like that habit should be picked up by a few more people."
Not all CD-R power users always place the same importance on all strategies for beating buffer underrun and other aspects of ensuring CD recording success, and not all of them practice each one for every recording session. But users would do well to develop a sound set of CD recording habits all the same, if only because the practice is insurance against the inevitable mistakes that result from operator error.
Because CDs were not designed for data, much less for recording random access data on the desktop, the ability to record CDs comes with a price that recent usage-easing innovations like packet writing only partially reduce. While packet writing does make CD-R a more user-friendly, floppy-like, goof-proof backup- and internal-use medium, there will always be applications in which CD-R users want to make a disc that looks and acts like a compact disc. For the latter, the price of compatibility with existing systems is doing everything possible to avoid making a disc that looks and acts like a gold-plated coffee coaster.
If you adopt these habits as part of your CD recording ritual--along with rubbing that rabbit's foot and reciting your mantra--things will almost never go wrong. And if they do, you can justifiably blame the equipment.
691 South Milpitas Boulevard,
Milpitas, CA 95035;
408/945-8600; Fax 408/957-6666;
C & I
Av. Medrano 681, 7o piso, Apartment C, (1179)
Buenos Aires, Argentina;
+54/865-6669; Fax +54/865-4572
418 S Diamond Drive
Long Grove, IA 52756
CD Solutions, Inc.
6 East Monument Street, P.O. Box 536,
Pleasant Hill, OH 45359;
937/676-2376, 800/860-2376; Fax 937/676-2478;
Dana J. Parker is a Denver, Colorado-based independent consultant and writer and regular columnist for Standard Deviations. She is also a Contributing Editor for EMedia Professional. She is the co-author of CD-ROM Professional's CD-Recordable Handbook (Pemberton Press, 1996).
Comments? Email Dana Parker at firstname.lastname@example.org