Skip to content

Latest commit

 

History

History
346 lines (211 loc) · 10.9 KB

hard-drive-recovery.adoc

File metadata and controls

346 lines (211 loc) · 10.9 KB

Hard drive recovery

Table of Contents

Over the years, I developed an interest in recovering data from hard-drives, USB sticks, SSDs and filesystems. These are my notes related to that. Note that all this comes with NO WARRANTIES OR GUARANTEES. Be careful, always work on a copy never on the original medium (where this makes sense).

It all started with: I used to be a big fan of Spinrite but it’s just too slow now. Looking for advice, I found these tips:

Recovery

  • Copy everything you can while it works with ddrescue

  • Run one or several badblocks passes with -n (non-destructive read/write)

There is even an option that will run subsequent passes until no new bad block or sector is found.

  • Run a filesystem checking tool and have it check for badblocks again (or use a logfile of your last check)

See this answer to know how to run badblocks

Read test and keep log

On a 512 bytes sector-sized disk

badblocks -b 512 -c 256 -o backblocks-readtest.log -s -v /dev/sdc

I think -c value could have been higher.

After consider using -n for a non-destructive read-write mode.

ddrescue operation

First pass:

ddrescue -v --sector-size=4096 --sparse --no-scrape /dev/... file mapfile

Second pass:

ddrescue -v --sector-size=4096 --sparse --idirect --retry-passes=3 /dev/... file mapfile
-b // --sector-size

make sure to get this right! With: fdisk -l <device>

-n // --no-scrape

Skip the scraping phase. Avoids spending a lot of time trying to rescue the most difficult parts of the file. Useful for a first quick pass.

--idirect

Use direct I/O for input. Recommended by ddrescue tutorial. -b must be the good value otherwise reads will fail.

Remember to use a log viewer like ddrescuelog if you want a fancy progress monitor.

Special case: ddrescue over the network

I used sshfs + ddrescue with a reasonable 10MiB/s rate using a 100Mbps network and slow machines.

  • I didn’t use ddrescue’s --sparse option because I was afraid it wouldn’t work over a FUSE filesystem

  • sshfs option -o big_writes

So basically:

sshfs <host>:<directory> <mountpoint> -o big_writes

Further optimization is possible. I haven’t tried other sshfs options.

Special case: cloning with ddrescue

ddrescue -v -f --no-scrape /dev/[baddrive] /dev/[gooddrive] /root/recovery.log

Using ddrescue to restore

ddrescue -f source.img /dev/[newdrive] restore.logfile

Hardware issues

If BIOS doesn’t detect the drive or the motor doesn’t spin, it doesn’t look good. Rubber hammer freezer trick.

If BIOS does detect the drive but all reads result in errors that are logged in syslog and ddrescue generates only errors then finding an exact PCB of the drive and swaping it might work. It worked for me once.

If BIOS detects the drive but the OS has trouble reading it. Try Direct I/O (-d) and smaller cluster size (ex: -c 16).

SATA Reset

Sometimes the kernel or the drive will be put in a bad state that will result in all subsequent reads yielding errors. A reset of the drive might fix it. It worked for me once and significantly improved the reliability of the drive at recovering the remaining of the data.

Replace X with the problematic drive letter:

echo 1 > /sys/block/sdX/device/delete

Find host to initiate re-scan (locally it’s host0):

echo "- - -" > /sys/class/scsi_host/host0/scan

Monitor dmesg or your syslog while doing so.

Look at the rescued data

Attempt to mount drive in Windows

Use ImDisk Toolkit or OSFMount. Click on mapped drive letter to try to run chkdsk.exe on it.

From ext2/3/4 filesystem

Paragos Software ExtFS for Windows (there’s a free version). However note that GPT partitionned hard drives won’t work on Windows XP.

Attempt to mount drive in Linux

Find offset where partition starts.

parted <image>
GNU Parted 2.3
Using picked.img
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) unit
Unit?  [compact]? B
(parted) print

If parted doesn’t work, use fdisk or gdisk (GPT partition tables) and multiply start sector with the sector size (usually 512 or 4096)

Mount using offset of partition you are interested in:

mount -o ro,loop,offset=XXXX <image> <mountpoint>

On some distros loop files are not automatically created. Follow instructions from search for partition section. Then replace <image> with /dev/loop0.

Damaged filesystem

Did ddrescue routine documented above. Only a few blocks damaged but at the beginning of the disk so the ext4 partition was unmountable.

Found many interesting things here. However nothing worked. All superblocks would fail even backup ones which is weird since they probably happened in disk area that were not considered damaged by ddrescue.

Testdisk would fail on the real device but it would work on the ddrescue image. I recovered some critical files here. Incredible.

Attempting to mount the image cleanly with (offset found with technique above):

mount -o ro,loop,offset=XXX <image> <mountpoint>

Results in a complain that filesystem requires a repair and is read-only. Loop mount to access underlying partition device files (documented below):

losetup -f --show --partscan <image>

Then use testdisk to find alternate superblocks. Under advanced filesystem options. Run the command it instructs you to run:

fsck.ext4 -b <superblock> -B <blocksize> /dev/loop0p1

Then the filesystem became mountable. Yay!

Search for partitions and filesystems

Simple

Mount image as loopback device and check with cfdisk:

losetup -f --show --partscan <image>

If partition table is corrupt or not present you can omit --partscan.

Will output created loopback device then:

cfdisk <lo-device>

When done:

losetup -d <lo-device>

Advanced

testdisk /log <image>.dd

Quick search then deep search.

Testing a disk

Figure out block size

lsblk -o NAME,PHY-SeC

Run badblocks in destructive read-write mode (way faster than non-destructive)

badblocks -b 512 -c 256 -w -s -v <dev>

Reusing bad drives

Make sure that the filesystem is aware of the bad blocks on it. See badblocks on how to do that.

Filesystem specific tools and info

HFS / HFS+

In order for Linux to mount this filesystem, a volume header (first sector) and alternate volume header (last sector of partition minus 1024) must be present. When attempting to recover a disk, the end of the drive was corrupted and the alternate volume header couldn’t be found.

hfsprescue (precompiled binaries available!) can do an excellent job at recovery. Better than photorec. Be sure to use a recent version.

010 editor template available to parse the volume header and HFS' metadata

fatx (Xbox, Xbox360)

Tools

  • fatxplorer (Windows-only, some features need a license)

  • fatx (no longer compiles on arch)

Xbox 360 specific

Information on Xbox hardware disks seems to be used in the cryptographic protection of the disks' content. What is used is tied with low-level information on the drive itself that is hard to replace without firmware-level tampering knowledge. This means that it is easier to transfer save games using a filesystem-aware tool rather than dd an entire Xbox360 drive into a new one.

That said, there is a lot of crypto involved in there so corruption can occur easily. The crypto seems to be signed by keys that can include console keys, Xbox LIVE keys, etc. Most of the stuff I was interested with (savegames to recover) was signed by my console keys luckily. Since I wasn’t swapping consoles only disks, it seemed like I could do it.

So here’s what worked (tried many things with fatx but they all failed):

  • ddrescue of the dying drive

  • restored image in working drive

  • Using fatxplorer in a Windows VM with USB passthrough, I extracted content I wanted to keep

  • Bought new Xbox drive

  • First, got it working with my console, imported profile from Xbox LIVE, etc.

  • Then with fatxplorer, I restored the files I backed up earlier

Side note: There are some tutorials and files out there to bring back to life the original Xbox backwards compatibility even on a 3rd party drive.

Create an interoperable filesystem

Nowadays it’s complicated to get an interoperable filesystem that works across Windows, Linux and Mac OS X. These instructions are for a Linux system trying to create such a filesystem.

Exfat with a specific GPT table works.

Here are the commands:

gdisk /dev/sdX
Command (? for help): o
This option deletes all partitions and creates a new protective MBR.
Proceed? (Y/N): Y

Partition type is 0700 (Microsoft basic data)

Command (? for help): n
Partition number (1-128, default 1):
...
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): 0700
Changed type of partition to 'Microsoft basic data'

Write partition, then format with exfat:

mkfs.exfat /dev/sdX1

Reformated with another filesystem on top

Chances are that filesystem metadata from the original fileysystem is still readable.

Partition table changes on block devices

Remember to run partprobe when you change partition tables to have Linux rescan the tables and expose partition block devices. For example:

partprobe /dev/sdb

Will create /dev/sdb1 if it exists after a disk restore.

Compression of disk images

High compression can be achieved on mostly empty filesystems. lz4 is a very fast algorithm.

Other options are available too but I’m not sure how portable these are yet.