Digital Preservation Q&A - Recent questions tagged hardware

Do you have any LTFS Tape Library Recommendations?

Tue, 11 Dec 2018 14:44:05 +0000

Hello all,

My institution currently uses LTO tape as part of our backup strategy for both business and collections data. We are both scaling up the number of tapes we create and are seeking to switch to the LTFS standard (https://en.wikipedia.org/wiki/Linear_Tape_File_System) for our collections data. We are looking for an LTFS-compliant tape library to help us increase our capacity, and I'm hoping that some of you might be able to share some details about what you're doing...

If you use tape for backups, do you use LTO? If you use LTO, do you use LTFS or a different standard/format?
What kind of tape drive/library set up do you have?
What software do you use to manage your hardware?

Best way to read SCSI tape drive on a modern PC?

Wed, 06 Jun 2018 12:00:51 +0000

We recently aquired a number of DLT-IV and DDS-1 tapes, and we'd like to recover the data stored on them. We already have readers for both tape formats; however these are both SCSI devices, and since modern PCs don't have SCSI connectors we cannot hook them up directly. After a bit of Googling I came up with a few options myself:

Buy a SCSI to USB adapter cable (used, as these are not produced anymore). However, I came across some reports that this is not a good option for tape drives, as tape transport commands (rewind, fas-forward) won't work. See e.g. here
Buy a writeblocker that has a SCSI connection. I'd expect that this would have the same limitations as option 1 above (also, these days no-one seems to be making SCSI writeblockers anymore)
Buy a used SCSI adapter card and build that into the machine that is used for imaging (but are these cards even compatible with modern desktops?).
Do the imaging using an old desktop that already has a SCSI card (or is compatible with it). For various reasons I'd rather not go this route unless absolutely necessary ...

As I don't want to reinvent the wheel, I'm curious if anyone with experience reading SCSI-connected tape drives could give me some recommendations on the best way to proceed with this. (BTW the workstation we'll be using for data extraction runs Linux.)

Incomplete ISO image after imaging CD-ROM - how to prevent and detect this?

Thu, 03 Sep 2015 10:33:26 +0000

While running some tests creating CD-ROM ISO images with ddrescue, I ended up with ISO images that were incomplete in some cases (last ~50 MB of image file missing), even though ddrescue’s log file didn’t report any errors. Below the results I got from 4 attempts at imaging the same CD-ROM on the same PC (note that some of the ddrescue options I used are slightly different, but this appears to be unrelated to my issue). For this I used 2 different external DVD readers:

Reader A - modern Samsung USB device;
Reader B - old SATA (internal) device, refurbished to USB.

Attempt 1 - reader A

Command line:

ddrescue -b 2048 -r4 -v /dev/sr0 windows_98_upgrade_nl.iso windows_98_upgrade_nl.log

This resulted in a 601.7 MB ISO image. Here are the contents of the log file:

# Rescue Logfile. Created by GNU ddrescue version 1.17
# Command line: ddrescue -b 2048 -r4 -v /dev/sr0 windows_98_upgrade_nl.iso windows_98_upgrade_nl.log
# current_pos  current_status
0x23DC0000     +
#      pos        size  status
0x00000000  0x23DCB000  +

I.e. the log file indicates the CD was imaged without problems. MD5 checksum is 82603be06a8142aad1dfaa9e1279371f

Attempt 2 - reader B

Command line:

ddrescue -d -n -b 2048 /dev/sr0 windows_98_upgrade.iso windows_98_upgrade.log

Again this resulted in a 601.7 MB ISO image, again with no indication of read errors in the ddrescue log. MD5 checksum was (again) 82603be06a8142aad1dfaa9e1279371f.

Then by chance I discovered some text files in the image file weren’t readable, so I did a third try, now again with reader A.

Attempt 3 - reader A

Command line:

ddrescue -d -n -b 2048 /dev/sr0 windows_98_upgrade_test.iso windows_98_upgrade_test.log

This resulted in a 660.9 MB ISO file. Again no errors in the ddrescue log; MD5 checksum is 24f0f746d0817121253c6b1242d4246e. After mounting the image, the text files that were problematic in the earlier images were normally readable.

Attempt 4 - reader B

Command line:

ddrescue -d -n -b 2048 /dev/sr0 windows_98_upgrade_refurbished_onemoretry.iso windows_98_upgrade_refurbished_onemoretry.log

Result was identical to result of attempt 3!

So summarising, 2 runs of ddrescue (using 2 different USB readers) resulted in exactly the same error, whereas the remaining 2 runs (again using 2 different readers) completed fine. So what’s going on here!?

Md5sum directly on physical CD

As a first step I computed the MD5 checksum directly on the phyical disc, using:

md5sum /dev/sr0

I repeated this 4 times, using both readers A and B, plugging them into different USB slots. In each case the result was 24f0f746d0817121253c6b1242d4246e, which is identical to the hash I got for the ISO in attempts 3 and 4 (confirming these images are correct).

Comparison of ISO images in hex editor

I also did a comparison of the intact and faulty ISOs in a hex editor. This revealed that in the faulty images a block of about 59 MB of data is missing at the end of the file. I double checked this by copying the block of missing data to a separate file (missingblock.dat), after which I appended it to one of the faulty files using:

cat windows_98_upgrade_nl.iso missingblock.dat > isorepaired.iso

Then check:

md5sum isorepaired.iso

Result:

24f0f746d0817121253c6b1242d4246e

Which corresponds to the value of the intact image.

But why is this happening in the first place?!

The really important question is why this is happening in the first place, and if there’s any way to avoid it? The thread below on the ddrescue mailing list describes a somewhat similar (but not quite the same) issue:

https://lists.gnu.org/archive/html/bug-ddrescue/2014-02/msg00003.html

Note the following quote from the response by ddrescue’s main author. He suggests that the problem might an issue with a USB port, adding:

Ddrescue can’t know if the data are really good or if the hardware is lying about it.

If correct, this would apply to other imaging tools as well. Based on my results, I’m curious if other people may have run into similar issues. More importantly: how does one even detect errors like these? Of course it is always possible to run a checksum on the physical medium and then compare it to the ISO checksum, but this takes ages. A more quick and dirty approach would be to compare the size of each created image against the size of input medium. E.g. to get the size of a CD-ROM I can use something like this:

lsblk /dev/sr0 -n -b

Result:

sr0   11:0    1 660850688  0 rom

(Third column is size of CD in bytes).

To get the size of the ISO image:

du -b windows_98_upgrade_refurbished_onemoretry.iso

Result:

660850688   windows_98_upgrade_refurbished_onemoretry.iso

This does not guarantee the image is correct, but it will detect missing blocks of data.

I also ran some cursory checks with isovfy and isoinfo, but the output of those tools turned out to be identical for both faulty and intact images, so they’re probably not very helpful for this sort of error.

I’m curious how other people/memory institutions are dealing with this. Any thoughts / suggestions are welcome!

Addition

On Twitter Alexander Duryee rightly pointed out that a CD-Rom's Primary Volume Descriptor contains a field with the size of the disk (this is also where lsblk gets this value). So one would assume that ddrescue would check against this number. Apparently it doesn't do this, so I think I'll reprt this as a bug. (Note that such a check doesn't guarantee the copied data are identical to the source disc.)

Does anybody have experience with a CRU USB Media Writeblocker?

Tue, 02 Dec 2014 14:12:42 +0000

We recently purchased a CRU USB Media Writeblocker -- we liked the price and the ability to put SD cards directly into the writeblocker. BUT we are unable to get it to read drives (we've tried several drives on several computers -- all running Windows 7) and neither we nor the manufacturer can't figure out what's going on. Either the computer doesn't recognize that anything has been plugged in at all, or it says that there is a problem with the drive and it cannot be read.

Has anybody else experienced something similar, or know of some sort of setting that may be causing the issue? Is there something silly I'm forgetting to do? Any advice/thoughts welcome.

What are best practices for cleaning floppy drive heads?

Wed, 30 Jul 2014 22:40:25 +0000

When working with legacy floppy disks, do you ever clean the drive heads? If so, when? Are there any established best practices for floppy drive maintenance?

What specifications are available for digital preservation and processing workstations?

Thu, 24 Jul 2014 10:59:59 +0000

Is there a place where specifications are available for workstations used in digital processing and digital preservation activities, such as disk imaging or format validation?

What are the specs of workstations people are using for digital preservation activities?

Wed, 23 Jul 2014 13:55:44 +0000

The questioner at #digpres2014 wanted to know detailed specifications of the machines that are being used in digital preservation. Specifically, what dedicated machines in a stand-alone environment have been allowed by various government agencies?

A good answer will have dates, operating systems, processors, memory, CPUs - as much detail as possible!

Is there a matchmaking service to match my legacy media with other institutions' hardware?

Wed, 23 Jul 2014 13:49:24 +0000

The questioner at #digpres2014 wanted to know:

If I get a one-off of a legacy media (such as an eight-inch floppy or nine-track tape), and my institution does not have the hardware to read it, where can I go?

What criteria should I consider when determining the hardware specs for a disk imaging station?

Wed, 23 Jul 2014 13:46:30 +0000

The person who asked the question at #digpres14 was thinking specifically of the tower that powers the disk imaging station rather than all the peripherals. Another way of framing the question is what are the bare minimum specs (RAM, processor, ports)?