Setting up a hard disk USB 2.0 enclosure for backup under Linux

For years, I have been using an IDE Travan tape for backup. I detailed the advantages of removable media like tape and how to use it under Linux in another article.

However, the choice of tape drives and tape cartridges for a home office environemnt are rather limited. The media is expensive, and the drives are also expensive.

My Seagate IDE Travan drive is capable of backing up 10GB, and 20GB with compression. This was satisfactory for years, until I got a 4 megapixel camera, and disk usage started to go up. I had to split the backup into two tapes, one for the pictures directory, and another for everything else.

Finally, I realized I should go for a more contemprary solution, and settled on an external hard disk and enclosure.

So, I got two 250GB Western Digital Caviar drives, for C$70 each, and a USB 2.0 enclosure for $26. Works well, and much more breathing room.

Here is a detailed tutorial on how to set this up under Linux. I use Ubuntu 64-bit for the server, but the commands should work on other distros as well.

The enclosure

The enclosure I bought is an MIDTE MDT-U3510 3.5" USB 2.0 enclosure.

Install the hardware

Carefully, put disk in enclosure after connecting the power connector and the IDE cable. Connect to a power outlet and press the On button.

Connect to PC

Connect the USB cable to the PC.

Check the dmesg command output for something like the snippet below. Note that the drive model, size, and capacity will be detected. Also, note that the device is detected as a SCSI drive since the kernel uses emulation.

[529675.334975] usb 4-6: new high speed USB device using ehci_hcd and address 2
[529675.463351] Initializing USB Mass Storage driver...
[529675.464369] scsi2 : SCSI emulation for USB Mass Storage devices
[529675.464665] usbcore: registered new driver usb-storage
[529675.464668] USB Mass Storage support registered.
[529675.464678] usb-storage: device found at 2
[529675.464680] usb-storage: waiting for device to settle before scanning
[529678.262770] Vendor: WDC WD25 Model: 00JB-57REA0 Rev: 20.0
[529678.262780] Type: Direct-Access ANSI SCSI revision: 00
[529678.263947] SCSI device sdb: 488397167 512-byte hdwr sectors (250059 MB)
[529678.263951] sdb: assuming drive cache: write through
[529678.270676] SCSI device sdb: 488397167 512-byte hdwr sectors (250059 MB)
[529678.270679] sdb: assuming drive cache: write through
[529678.276789] sdb: unknown partition table
[529678.284660] sd 2:0:0:0: Attached scsi disk sdb
[529678.284701] sd 2:0:0:0: Attached scsi generic sg1 type 0
[529678.285398] usb-storage: device scan complete

Also, check the USB connections using the info in /proc, or a script like usbtree.

 # usbtree
/:  Bus 04.Port 1: Dev 1, Class=root_hub, Driver=ehci_hcd/6p, 480M
    |__ Port 6: Dev 2, If 0, Class=stor., Driver=usb-storage, 480M
/:  Bus 03.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=uhci_hcd/2p, 12M

This tells us that this is 480Mbps.

Partition the disk

Now, we need to partition the disk. Use the following command to do so:

cfdisk /dev/sdb

The following messages appear in dmesg's output:

[530183.089507] SCSI device sdb: 488397167 512-byte hdwr sectors (250059 MB)
[530183.089511] sdb: assuming drive cache: write through
[530183.094050] sdb: sdb1

Which shows that the kernel detected the new partition.

Create the file system

After partitioning, we need to create a file system on that drive.

This can be achieved using the command:

mkfs -t ext3 /dev/sdb1

Mount the disk

Now, we mount the disk using the following command:

mkdir /mnt/extdisk

mount /dev/sdb1 /mnt/extdisk

We also see the following messages in dmesg

[530921.342377] kjournald starting. Commit interval 5 seconds
[530921.343051] EXT3 FS on sdb1, internal journal
[530921.343056] EXT3-fs: mounted filesystem with ordered data mode.

Test the setup

Now we can use the disk just like another other disk/directory on the system.

Using these commands:

cd /
time find . -mount -depth | cpio -ovH crc -O /mnt/extdisk/backup.cpio

The output indicates that we backed up 22GB in 36 minutes, which comes to 10430815 bytes per second (10 MB/s).

We can improve this by using a larger block size for cpio, such as:

cd /
time find . -mount -depth | cpio -ovH crc -C65536 -O /mnt/extdisk/backup.cpio

This gives us 22GB in 20 minutes, which means 18794421 bytes per second (18 MB/s).

Backup strategy 

The backup scheme I use keeps a backup for every day for the last week. This is done by putting the day of week in the file name, so on Monday, the last Monday backup is overwritten.

You may notice the time increasing every day slightly. This may be due to disk fragmentation as the old backup is deleted and a new one is written. 

Other options

Another option is to use the dump command, which backs up entire file systems, and can be used for incremental backups (i.e. only the changes not the entire filesystem). Here is a how to on dump for backup.

Variances in Enclosures

I have noticed significant variations between enclosures. I use two enclosures, with two disks. The disks and the enclosures are the same manufacturer and same model. At one point, one enclosure would take 7 hours while the other would take only 42 minutes, and this is on the same disk, same power supply and same cable. Linux reports the same info for lsusb -v and same firmware and model version for the disk. The only difference was the enclosure itself, even after I created a new file system on the disk to rule out fragmentation. 

I ended up exchanging the slow enclosure for a new one, and the new one proved faster, as well as much quieter than the two old ones.

Conclusion

Disk enclosures are a good way to do backup. It is hard to match tape as a passive medium that is not as sensitive as hard disks. It is also not very helpful for off site backups or rotating backups.

However, with the capacity of hard disks going up and tape unable to catch up at reasonable prices, there is little choice today.

Performance is definitely better than tape as well.