The first thing to mention is that there is tons of “imaging” software out there, but there is a HUGE difference between these and a data recovery imaging tool. Many of these software work great such as Acronis TrueImage, but as soon as you put a weak or failing hard drive into the mix; the software shows it’s flaws. This software is not designed with failing hard drives in mind; it is designed as a solution for easy duplication, and backup solution.

So lets get down to it. What makes a data recovery imaging tool different from all of that software? It is a combination of specialized hardware and highly advanced features that come together to place these tools into a completely different league then normal software imaging solutions.

Lets talk hardware:

So you may be wondering what kind of “special” hardware is needed to insure that a data recovery imaging tool is up to the job.

The first thing is that this hardware has to be able to support the full array of ATA commands, as well as any custom manufacturer commands that may be implemented. While a standard ATX Motherboard may be able to handle most of the ATA commands; there are many undocumented commands that are not “required” by the ATA standard that the manufactures use for their own purposes. These commands are completely different between each manufacturer as each manufacturer designs their firmware in a different fashion.

The Second is that when working with failing hard drives; you may run into drives that have circuit board issues that will cause a short in your system. A standard motherboard and/or PSU will see this and it’s circuit protection will trip; in this scenario you will likely not even be able to power up your system with the drive plugged in let alone run any kind of imaging operation, or at the very least run diagnostics on the drive.

Data recovery imaging equipment has specialized power regulation and monitoring systems that allow you to apply power to drives, and monitor the currents running through them. If there is a short on the circuit board this hardware won’t just shut down; instead it will likely notify you of the short and shut down the drive promptly to protect the drive from any further damage. At the same time the hardware is highly resistant to any damage from short circuits, etc so you do not have to worry about this hardware sustaining any damage from damaged PCB.

Write Protection is another hardware feature that is especially important for forensic examiners. Every device a forensic examiner plugs a drive in HAS to be write protected, even if the device shouldn’t be making any writes to the master drive it is a safety precaution that is required by forensic examiners; for this reason it is nearly standard on all data recovery imaging tools.

There are other hardware features that are good to have, but the main purpose of the customized hardware is to facilitate the features mentioned above. The real power lies in the software that ties into the hardware (in some cases the hardware may have firmware embedded into it that acts as the software, and makes the unit standalone)

What kind of software features set a data recovery imaging tool above your normal software imager?

Sector Timeouts: The ability to customize how long you wait for a sector to be read. This feature is useful because you may want to quickly avoid any problem areas; in this case you will set your timeout very low. If you want to really make sure that heads cannot read this sector, then you would set the timeout really high.

Sector Skipping: Skipping is a useful feature to attempt to navigate around problem areas. All good data recovery tools will have the ability to skip AT LEAST 1 sector. The most advanced tools should be fully customizable, and you will be able to change this skip parameter on the fly.

Retry: This feature can be great if used properly. Most normal software imaging tools have this set to 99, which is terrible. This means it will sit on 1 sector and try 99 times to read it. This is a huge waste of time, and can potentially cause even more damage to the drive. If you are trying to copy a weak or failing drive you want to get as much data, as quickly as possible because the drive is liable to fail at ANY time.

The best way to utilize a retry is “Multi pass imaging”

Multi pass imaging: Multi Pass Imaging is one of the most powerful features a data recovery imaging tool can have. Only the best of the best tools have this feature. The idea is that you can customize “passes” where each pass has different parameters. You would likely want to start with a quick pass; quickly skipping over bad areas and getting all of the easy to get sectors. Once the first pass is completed the imager starts over from the last sector is wasn’t able to read on the first pass. Here is the catch: The imager remembers all of the sectors is already copied. This way you are not wasting your time going over sectors you have already gotten, and you can try more in-depth settings on the sectors that are giving you trouble.

Reverse Imaging: Reverse imaging is useful in many cases. Some drive seem to copy more reliably when copied in reverse. This may be because reverse imaging often disables any drive caching, and also moves at a slower pace. Reverse imaging is also useful if you suspect there is media damage somewhere towards the front or middle of the disk; by imaging in reverse you attempt to avoid this damaged area in hopes that the heads will live a little longer while you try to extract the data from the drive.

Power Cycle: Sometimes a Drive may hang completely when faced with a certain error, or bad sector. In this case it is critical that you are able to cycle the drive, and be able to continue exactly where you left off.

Reset: A similar concept to power cycling, but this involves just resetting the drive without re powering it.

UDMA Speed adjustment: If a drive is unstable it may run better at a slower speed. Being able to change the max UDMA speed may help the drive run more reliably during the imaging operation; sacrificing speed.

PIO mode: PIO mode avoids UDMA transfer all together in favor of the older PIO mode. This mode only runs at ~2MB/s but is useful if you are working with a VERY unstable drive, or area of a drive.

Ignore ECC: All sectors have ECC (Error Correction Code). Sometimes a sector may be reported as UNC, even though there is some readable data within the sector (maybe 25% of the sector data is corrupt, but 75% is not. With this feature you ignore the error correction code telling you the sector is bad, and read what you can out of the sector anyways. This feature is useful off the basis of: SOMETHING is better then NOTHING.

Disable Read-ahead: This is another feature that was added to hard drives that is useful during normal operation, but when working with an unstable disk may cause more instability. Basically, when a you request a certain sector/s from a drive it will actually read ahead of what was requested and cache it, predicting that you will likely be requesting those sectors as well.

Data-Only Copy: This feature is self explanatory; if possible you want to try to image only the sectors with actual data on them. If your patient drive is a 1TB, the chances of it being 100% full are not very high. In most cases the drive is going to be no more then 50% full. When copying data in a recovery scenario, every single move you make counts, so wasting time imaging blank sectors does not make any sense. Data only copying requires that the tools knows what information it needs to retrieve to know where all of the data is; such as a FAT table, or MFT, etc. Usually only certain file systems are supported. The common ones that will be supported are NTFS, FAT, and HFS. Ext support is becoming more common as well.

Image by selective heads: This is the big one. Sometimes a drive may identify itself, and seem like it is working normally, but one or two of the heads are practically dead, or VERY weak. In this case you will likely need to perform a head transplant, but you would like to get all of the data you can off of the drive first. This is where this feature comes in. You can actually tell your imaging tool to image only data from a certain head. By doing this you can image the data from the heads that are working well, and then perform you head transplant and then image the data from the heads that you did not image before.

So, those are the most important features of a data recovery imaging tool. You will find that there are even more features that can be useful in certain situations, but the ones listed are the ones that you really want to have if you are going to be getting an imager for data recovery use.

Unfortunately, it is always met with the same answer from myself: “There really is no safe hard drive, all drives from all manufacturers are susceptible to failure.”

While there may be a “better brand at this moment” even the manufacturer that is seemingly having less failures is still going to have some failures. You can’t change the fact that hard drives are mechanical devices with moving parts, that are mass produced. There is really no way around it; there are going to be defective units; just like there are defective automobiles, and defective video game consoles, etc. There will always be defective hard drives.

So what can you do to protect yourself and your valuable data? Make a plan.

You should always have a plan, and follow that plan as closely as possible. Here are some tips to avoiding a scenario that requires recovery:

• Keep a good backup, in a seperate, unattached location from your workstation/computer

This comes in the form of an external HDD, DVDs, a home server, etc.

A key note to this is: Don’t store your data ONLY on your external, this is a common mistake that will cost you dearly in the future.                               Your internal HDD in your computer has space, use it. Contrary to popular belief, the HDD will not get slower as you fill it up, as long                              as you keep it De-fragmented.

• Be careful with your notebook

Many people carry their notebooks around while they are still powered on. This can be dangerous as there is a HDD in that                                                 notebook spinning, with the heads possibly unloaded onto the platters (Even if you are not doing anything, windows may have                                         requested the HDD to access some service data, etc).

I would recommend getting into the habit of not using standby if you are going to carry your laptop around, but instead do a full                                    shutdown. It may be a slight pain, but not nearly as big of a pain as having to have your data recovered. Most data recovery                                              companies will make the recovery process as painless as possible, but there will always be a loss of time (if the data lost was work                                    related) and time = money for many businesses.

• Be careful when purchasing an External Drive

A big problem is external Hard Drives. Most of the drives made by manufactures are placed in enclosures with sub-par ventilation,                                and furthermore manufacturers use specific drives in these that are “Not as good as OEM drives”.

For an external I would recommend purchasing a internal OEM drive, and then picking out an enclosure yourself. Get one with a                                    ”active cooling” (a fan) if possible. By doing this you also gain the freedom to choose the interface types you would like.

Finally, the “best” solution with an external drive is you pick up a enclosure that supports 2(or more) drives, and will allow you to                                  configure the drives into a “Mirrored Array” aka “RAID 1″. This means that you only have the storage capacity of 1 drive, but all the                                data written to the main drive will be written to the second drive as well, so in the case that the main drive dies the second will still                                be functional. This is not full proof, as if you accidentally delete a file from the main, it will mimic this on the second drive as well.                                  Another thing that this does not protect from is shock damage to the enclosure such as a drop.

By following these tips you will significantly reduce the chances of needing recovery. As we begin to rely more and more on digital data it is becoming more critical that we insure that this data is stored safely and securely.

I wanted to make it known that I have not fallen off of the face of the Earth; It has been a busy few months and here at the office we have all kinds of interesting projects going on; also our data recovery business has ramped up lately, and I am busier then ever.

Anyways, to get down to the topic of this post. I wanted to give my “predictions” for the storage industry, and for data recovery in 2010.

4K Sectors:
To start; Hard drives are not going anywhere for a while. Western Digital has initiated the move to 4K sectors
Nice article here: WD “Advanced Format”

Not only is this good for the HDD industry as this transition will improve the efficiency of ECC code, and also allow for higher capacity drives, but it is also good for computing in general as it will force users to finally make the jump to a new OS as Windows XP cannot use 4K sector drives without many issues.

From a Data Recovery perspective I am not exactly looking forward to dealing with these new drives. Whenever a major change is made like this; there is bound to be growing pains involved, and these growing pains are likely to come in the form of faulty firmware (bad translators, etc) and an array of other fun failures.

SSD creeps closer to World Domination:

As I mentioned; HDD aren’t going anywhere SOON. This doesn’t mean that the successor isn’t mustering it’s force. The current latest and greatest is the new controller by “Sandforce”:
Sandforce SSD Controller

This controller is a significant improvement over any other SSD controller on the market right now. The bar has been raised and now all the other controller designers are having to step up their game to keep playing.

The next big bang will come this year, with the introduction to Intels 25nm NAND technology.
Intel 25nm NAND technology

This will bring SSD prices down to a unbelievably affordable level. This will be the first real dagger in the back of the HDD industry.

Data Recovery Outlook:

Data Recovery is as strong as ever. More people are adopting digital storage, and unfortunately this is increasing the chances that someone will forgo a backup solution, and lose valuable data because of it. That is why we exist; we are the second line of redundancy; the one you never want to have to deal with, but are glad it is there when you need it.

My predictions are pretty much more of the same, with a little new flavor thrown in the mix somewhere. Seagate drives will continue to fail in the same ways they have been for the past two years, and WD drives will have some new problems show up.

SSD will definitely make a showing, but many DR firms are not yet fully equipped to handle SSD. Also, users are typically using SSD for OS/application, and using HDD as the primary storage for their important data. This is a trend that I believe will continue for a number of years to come.

The most interesting aspect of recovery I feel is going to start coming from the mobile sector. I feel that we may start running into cell phone recoveries as people have begun using smartphones as a true alternative to a notebook. Of course most of this will use the same methodology as SSD recovery as phones simply use flash memory to store data. I think the most interesting side of cell phone recovery will be the logical side; where we are going to have to learn about these mobile OS’s in-depth. (Android, Windows Mobile 6, 7, etc)

So that is it for now; more later. Sooner then later this time. That is a promise.

Furthermore, the “repair” is not so much of a repair. You could consider this repair to be the equivalent of a tourniquet. The hard drive is like an arm that has been badly wounded and is bleeding profusely. The tourniquet is the “repair”, which temporarily relieves the underlying issue, but the fact is it’s only a temporary solution to a escalating problem. With hard drives, there is no hospital that can stop them from degrading; we can only hide the degradation temporarily.

So what is really going on when a drive is “repaired”? Reallocation.

A hard drive is constantly performing built-in maintenance routines and when the drive recognizes an unstable sector, the hard drive realizes this, marks the sector bad, then puts it in a reallocation queue.

What is this “reallocation”?
It’s a simple principle: the detected bad sector gets put into a list that says “do not use this sector anymore, because it can’t hold data reliably”. This list is known as the “G-list” or “Grown Defects List”.

Once the bad sector has been removed and put into the G-List, a new, reserved sector is used in its place. This reserved sector is “mapped” as the same LBA (Logical Block Address) as the old, dead sector, but it is in a different physical location on the disk.

This can cause issues once many sectors begin failing because the heads will have to change position to read the new sectors that are located in the reserved areas. This will cause delays in access times and will eventually be noticeable to the end user.

All hard drives contain these reserved sectors and the OS is unable to see or use them. Only the drive knows of their existence and their location on the disk.

There is even more to this story though, as there are two types of bad sectors. There are “physical bad sectors” and “logical bad sectors”.

A physical bad sector is a sector that for some reason can no longer to be magnetically manipulated and used to store data. These are the types of errors that will be mapped to the G-List.

A logical bad sector is a sector that is able to be magnetically manipulated, but for some reason was incorrectly manipulated (maybe power was pulled during a write to that sector or the heads are failing/weak and wrote incorrectly to the sector, etc).

These logical bad sectors can be regenerated (usually by writing fresh, correct data to them) and then reused. But, sometimes logical bad sectors can be incorrectly written to the G-List. This often happens when the heads of a hard drive are failing and are reading good sectors with UNC errors because the read element of the heads is malfunctioning. If this happens, the drive will attempt to queue these sectors for reallocation and, if successful, good sectors can end up in the G-List of the drive. Luckily, if you are in need of your data being recovered, such an issue is reversible. Most data recovery technicians can easily access the G-list of a drive that is held within it’s “service area” and remove false entries, or even clear the list completely.

The last thing that I would like to mention is refurbishing. Refurbishing drives takes a somewhat similar approach as these normal repair programs do: they remap bad sectors. The difference is that good refurb companies have reverse engineered hard drives so that they have the manufacturer-unique commands that tell the drive to reallocate bad sectors to a different list. This list is called the “Primary Defects List” or “P-List”. Sometimes it may be called the “Push-Down Defects List”.

The P-list is typically only written once in a disk’s lifetime. This is typically done at the manufacturing site when the drive is assembled and the servo code is written to the platter surface. The P-list handles reallocation much differently than just replacing the bad sector with a reserved sector. It is a more complicated process that will be discussed in more details in a future post, but, in short, it eliminates the performance issues mentioned that occur when many sectors are replaced with reserved sectors.

I hope this has enlightened some about hard drive repair. In conclusion, there is no real “repair” for a drive. Even the refurbishing technique is a temporary solution. The fact is that when a few sectors degrade magnetically, the degradation usually begins to spread to nearby sectors fairly quickly, and the drive will soon become unusable.

Lets start off with a picture:

The end of the arm