Linux: Tape backup for a home network

Sponsored Links

Linux: Tape backup for a home network

Introduction

In another article, I have discussed why tapes are still the most efficient form of backup. In this article, I detail how I used tapes in a Linux based home network for backup purposes, using a Seagate/Certance STT20000A IDE ATAPI Internal Travan TR-5 Tape drive. There are also several resources listed.

Read below for more ...

Background and Historical Notes

I have been backing up my personal stuff on tape for about 15 years. I used to make copies of my home directory (on UNIX, at work) to a QIC tape that had a whopping 60 MB capacity, running on an NCR TOWER (Motorola 680x0 CPU). Later, I would use 150 MB QIC tapes on the same architecture.

I then migrated to the same drives but on NCR System 3000 (Intel x86 CPU), and had to do byte swapping conversion on the data:

dd if=/dev/rmt/c0s0 conv=swab | cpio -icvmuld

Since QIC drives were expensive, this was out of my reach at home. But there were plenty of those at work, and my stuff was mainly on my machine at work. This high cost factor alone contributed to my being delayed in running UNIX/Linux at home for years.

When I started using Linux at home, I bought an HP Colorado 2.5GB/5GB Travan IDE tape drive. It worked well and I backed up stuff on it when my hard drive was just 2GB. I used several tapes, and backed up every week or two, normally overnight.

The tape drive was recognized as an IDE Tape, so the device was /dev/ht0, so in the dmesg output, or in /var/log/messages you should see something like:

ide-tape: hdd <-> ht0, 600KBps, 14*26kB buffer, 2600kB pipeline, 310ms tDSC, DMA

A Real Life Example

In 2000, my home was broken into and the PC stolen! Those tapes came in handy with all my data on them. I only lost about a month's worth of data.

I bought a used Seagate 10GB/20GB Travan IDE tape drive off someone selling it on eBay for about 100$US, including several used tapes. I restored my home directory and everything was good to go. If I had bought another hard disk and copied stuff to it, all my data would have been gone forever! Remember that next time you say "buy another disk!"

All the computers on the home network share the server's drive using either Samba (for Windows clients) or NFS (for Linux clients). Each user is responsible for making sure that their files end up on the server, otherwise, they are not backed up.

I use a set of 6 tapes for weekly backup and 3 for monthly backups, all using cpio. The monthly ones are kept offsite, at a friend/co-worker's house. Even with removable hard disks, you can't justify the cost of 9 hard disks!

The Linux cron facility is used to start the backup in the early hours of the weekend. I even have another cron job the day before to check that the tape is inserted in the drive and that it has been rewind to the beginning of tape. The entire process is unattended, except for an email telling me the backup is done, and to change the tape, or to remind me of the monthly backup.

My backup is about 7 GB now for the home server, which contains my kids homework, my files, digital pictures, CVS repository, ..etc. So there still room on the tapes.

Configuration

In order to configure it with Linux, you need to specify the kernel argument hdx=ide-scsi (where x is the letter of the IDE position for that drive). You can specifiy this argument in the lilo or grub configuration file.

You will know that the tape is correctly installed and that it is recognized by the Linux kernel, when you run the dmesg command or view the /var/log/messages log, and see the following:

First a message that the IDE driver has recognized the drive.

hdd: Seagate STT20000A, ATAPI TAPE drive

Then, you should see the SCSI over IDE layer loaded, and it should recognize the tape as well:

SCSI subsystem initialized
scsi0 : SCSI host adapter emulation for IDE ATAPI devices
Vendor: Seagate Model: STT20000A Rev: 8.25
Type: Sequential-Access ANSI SCSI revision: 02

Finally, you should see the SCSI Tape driver loaded:

st: Version 20040122, fixed bufsize 32768, s/g segs 256
Attached scsi tape st0 at scsi0, channel 0, id 0, lun 0
st0: try direct i/o: yes, max page reachable by HBA 163838

The tape device is now known as /dev/st0 to the tape programs.

You can also use /dev/nst0 if you do not want the tape to rewind on close. This is useful if you want to be creative and "append" backups on a tape.

Note: You may see the following messages in the output of dmesg or in /var/log/messages. Do not be alarmed. There is no ill effect from there, and they are harmless:

Jul 14 12:53:36 zzzz kernel: st0: Error with sense data: Current st0: sense key Illegal Request
Jul 14 12:53:36 zzzz kernel: Additional sense: Invalid command operation code

Backup and Restore Commands

You can then use the following commands:

To get the tape status, do the following:

mt -f /dev/st0 status
SCSI 2 tape drive:
File number=0, block number=0, partition=0.
Tape block size 512 bytes. Density code 0x47 (TR-5).
Soft error count since last status=0
General status bits on (41010000):
BOT ONLINE IM_REP_EN

Note the status bits at the end of the output. These are useful, but are often cryptic. Here is an explanation of the most important ones:

Status Bit Description
BOT The tape is positioned at the beginning of the first file.
EOT A tape operation has reached the physical End Of Tape.
EOF The tape is positioned just after a filemark.
WR_PROT The tape (or drive) is write-protected. For some drives this can also mean that the drive does not support writing on the current medium type.
ONLINE The drive has a tape in place and ready for operation.
DR_OPEN Door is open. Depending on the type of drive, this usually means that the drive does not have a tape in place.
IM_REP_EN Immediate report mode. This bit is set if there are no guarantees that the data has been physically written to the tape when the write call returns. It is set to zero only when the driver does not buffer data and the drive is set not to buffer data.
SM The tape is currently positioned at a setmark. DDS specific.
EOD The tape is positioned at the end of recorded data. DDS specific.
D_6250
D_1600
D_800
This "generic" status information reports the current density setting for 9-track 1/2 inch tape drives only.

If you do not have a tape cartridge inserted in the tape drive, you will see:

DR_OPEN IM_REP_EN

If you have a tape cartridge in the tape drive, but the write protect tab is on, you will see:

BOT WR_PROT ONLINE IM_REP_EN

In order to rewind the tape, do the following:

# Rewind the tape
mt -f /dev/st0 rewind

For actual backup, I do not use dump since it only dumps one filesystem, and does not cross the boundary. Therefore I will need two tapes to backup my system, which is kind of a waste. I could actually make the backups back to bak, but this complicates things if I am looking to read info from the tape, specially in the panic of a recovery situation. It is best to backup everything using cpio, with relative pathnames, and then restore whatever you need to wherever you need, even if partitions are different.

Therefore, I prefer to use cpio. Even tar is a bit primitive compared to cpio, since it has many nice features, such as preserving the modification time.

To backup, use the following command:

# Backup a directory using cpio crc header format
cd /home
find . -print | cpio -ovH crc -O /dev/st0

The -H crc option specifies that the backup will be in a portable format that can be read on other UNIX machines, and not only Linux.

If you are doing a full system backup, then you have to exclude some directories, such as /proc, /dev and /mnt.

To list the tape contents, you can do:

cpio -itv -I /dev/st0

To restore the tape contents, you can do:

cpio -icvmuld -I /dev/st0

Check the appropriate manual pages for the mt and cpio commands for more details (links below in the Resources section). You may also want to check the dd manual page as well, since you can do some neat tricks with it.

So, I will stay with tape for the forseable future, until optical technology catches up with disk size increases.

Resources and Links

Tape backup

Dear Khalid,

Thanks very much for your clear explanation of the tape backup procedure. I happened to have a SCSI STT20000 tape drive and running now SUSE Linux Enterprise Server 9. During my Novell 5 time I used Backup Exec, but this works great also. Your input on Internet was very much appreciated! Lost quite a bit of data recently; this will prevent a second time.

Thanks for a great resource

Thanks for a great resource for tape backup.

I would like to ask a question or advice.

I have a Debian PC and a Windows PC Xp and I would like to back up files on both, via a LAN, to a PC running Debian Sarge as a server which has a Seagate IDE tape drive installed in it.

I have looked at Amanda, Bacula and Flexbackup but after reading your how to's I think I am closer to a solution.

If I was to put the files needed to be backed up into a folder onto to the server and then have them backed up to the tape drive.

Do you have any tips you could share.

I also really enjoyed your family history section

another option for backup automation

Thanks for the great article! I just thought i'd add here that I use flexbackup ( http://www.edwinh.org/flexbackup ) for a relatively simple backup routine...

find . -depth -print | cpio

find . -depth -print | cpio -ovH crc -O /dev/st0

Use the -depth flag with find.

A better choice for creating the backup is to add the -depth flag to find. This allows read-only directories to be restored. (A directory is written to tape after its contents. So, the contents of a directory are restored and then the directory permissions are restored).

best regards,
chris

2007Sep30Sun09h08m56sPEST-4EDT and still ticking . . .

I strongly suspect that tape is still the best way to go. At least it is in my case, but then not everybody treats their home/SOHO operations like a professional software development facility.

Also, I don't believe in retiring equipment just because there's a sale at the discount market; and that goes for being coerced to upgrade by software vendors. If it ain't broke, don't fix it and if it does as it should, don't upgrade/replace it.

If you follow the latest trends, you hardly have time to configure everything before somebody is telling you that you need new equipment. Thank God they're not upgrading hammers and screwdrivers at every heartbeat -- we'd never get anything done.

Thanks, Khalid, I hope that your wisdom survives, with you and with all who have heard you.

Ernie

Incremental backup

Hi Khalid,
Thanks for the solution you have given here, but I would also want to know that how we can take incremental backup and how we can append the data by using a tape drive.
Since i am using a LTO3 tape drive and i have to perform daily backup.
It will be great if you will let me know to perform incremental backup and how to append data in the tape drive.
Since I have tried using "tar" command and it was not working.
Reply on my mail ID if possible.

Thanks

hello

I am using to backup Linux a software called Dmailer Backup Software , their software is free and their online storage service is also free but only up to 3gb .
But anyway what i like at it because its easier to use than other softwares and also much faster.