NetBSD Problem Report #10826
Received: (qmail 8180 invoked from network); 14 Aug 2000 00:51:46 -0000
Message-Id: <20000814005142.D96DC1FE0A@crasse.smd.ebone.net>
Date: Mon, 14 Aug 2000 00:51:42 +0000 (GMT)
From: smd@ebone.net
Reply-To: smd@ab.use.net
To: gnats-bugs@gnats.netbsd.org
Subject: restore does not DTRT at end of volume on DVD-RAM
X-Send-Pr-Version: 3.95
>Number: 10826
>Category: bin
>Synopsis: restore does not DTRT at end of volume on DVD-RAM
>Confidential: no
>Severity: serious
>Priority: high
>Responsible: bin-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Mon Aug 14 00:52:00 +0000 2000
>Closed-Date:
>Last-Modified: Tue Feb 05 17:24:14 +0000 2002
>Originator: Sean Doran
>Release: CVS 2000 08 13
>Organization:
>Environment:
System: NetBSD crasse.smd.ebone.net 1.5E NetBSD 1.5E (SCREAM) #0: Thu Aug 10 04:03:43 CEST 2000 smd@crasse.smd.ebone.net:/usr/src/sys/arch/i386/compile/SCREAM i386
>Description:
dump to two dvd ram sides
then try to restore everything; at the end of the first volume,
dump chokes
>How-To-Repeat:
dump -b 2 -B 2430000 -f /dev/rcd0d MD5 *.gz
DUMP: Dumping sub files/directories from /scratch2
DUMP: Dumping file/directory MD5
DUMP: Dumping file/directory crasse.root.0.gz
DUMP: Dumping file/directory crasse.u.0.gz
DUMP: Dumping file/directory crasse.usr-X11R6.0.gz
DUMP: Dumping file/directory crasse.usr-local.0.gz
DUMP: Dumping file/directory crasse.usr-pkg.0.gz
DUMP: Dumping file/directory crasse.usr.0.gz
DUMP: Dumping file/directory crasse.var.0.gz
DUMP: Date of this level 0 dump: Mon Aug 14 00:12:28 2000
DUMP: Date of last level 0 dump: the epoch
DUMP: Dumping a subset of /dev/rsd3a (a subset of /scratch2) to /dev/
DUMP: Label: none
DUMP: mapping (Pass I) [regular files]
DUMP: mapping (Pass II) [directories]
DUMP: estimated 3685151 tape blocks on 1.52 tape(s).
DUMP: Volume 1 started at: Mon Aug 14 00:12:28 2000
DUMP: dumping (Pass III) [directories]
DUMP: dumping (Pass IV) [regular files]
DUMP: 5.05% done, finished in 1:34
DUMP: 10.06% done, finished in 1:29
DUMP: 15.03% done, finished in 1:24
DUMP: 20.01% done, finished in 1:19
DUMP: 25.04% done, finished in 1:14
DUMP: 30.07% done, finished in 1:09
DUMP: 35.10% done, finished in 1:04
DUMP: 40.09% done, finished in 0:59
DUMP: 45.12% done, finished in 0:54
DUMP: 50.07% done, finished in 0:49
DUMP: 55.11% done, finished in 0:44
DUMP: 60.10% done, finished in 0:39
DUMP: 65.02% done, finished in 0:34
DUMP: Closing /dev/rcd0d
DUMP: Volume 1 completed at: Mon Aug 14 01:18:25 2000
DUMP: Volume 1 took 1:05:57
DUMP: Volume 1 transfer rate: 614 KB/s
DUMP: Change Volumes: Mount volume #2
DUMP: Is the new volume mounted and ready to go?: ("yes" or "no") yes
DUMP: Volume 2 started at: Mon Aug 14 01:21:03 2000
DUMP: Volume 2 begins with blocks from inode 224643
DUMP: 67.37% done, finished in 0:32
DUMP: 72.40% done, finished in 0:27
DUMP: 77.41% done, finished in 0:22
DUMP: 82.38% done, finished in 0:17
DUMP: 87.29% done, finished in 0:12
DUMP: 92.19% done, finished in 0:07
DUMP: 97.00% done, finished in 0:03
DUMP: 3684787 tape blocks on 2 volumes
DUMP: Volume 2 completed at: Mon Aug 14 01:55:33 2000
DUMP: Volume 2 took 0:34:30
DUMP: Volume 2 transfer rate: 606 KB/s
DUMP: Date of this level 0 dump: Mon Aug 14 00:12:28 2000
DUMP: Date this dump completed: Mon Aug 14 01:55:33 2000
DUMP: Average transfer rate: 610 KB/s
DUMP: Closing /dev/rcd0d
DUMP: DUMP IS DONE
crasse# restore -x -b 2 -v -f /dev/rcd0d
Verify tape and initialize maps
Dump date: Mon Aug 14 00:12:28 2000
Dumped from: the epoch
Level 0 dump of a subset of /scratch2 on crasse.smd.ebone.net:/dev/sd3a
Label: none
Extract directories from tape
Initialize symbol table.
Make node ./BACKUPS
Make node ./BACKUPS/20000813
Extract requested files
You have not read any tapes yet.
Unless you know which volume your file(s) are on you should start
with the last volume and work towards the first.
(Use 1 for the first volume/tape, etc.)
Specify next volume #: 1
extract file ./BACKUPS/20000813/crasse.u.0.gz
extract file ./BACKUPS/20000813/crasse.root.0.gz
extract file ./BACKUPS/20000813/crasse.var.0.gz
Tape read error while restoring ./BACKUPS/20000813/crasse.var.0.gz
continue? [yn] y
Tape read error while restoring ./BACKUPS/20000813/crasse.var.0.gz
continue? [yn]
...
>Fix:
>Release-Note:
>Audit-Trail:
From: woods@weird.com (Greg A. Woods)
To: gnats-bugs@gnats.netbsd.org (NetBSD GNATS submissions and followups),
netbsd-bugs@NetBSD.ORG (NetBSD Bugs and PR posting List)
Cc:
Subject: Re: bin/10826: restore does not DTRT at end of volume on DVD-RAM
Date: Sun, 13 Aug 2000 21:13:43 -0400 (EDT)
[ On Monday, August 14, 2000 at 00:51:42 (+0000), smd@ebone.net wrote: ]
> Subject: bin/10826: restore does not DTRT at end of volume on DVD-RAM
>
> Note: There was a bad value `bin' for the field `>Confidential:'.
> It was set to the default value of `yes'.
oops! ;-)
> dump to two dvd ram sides
> then try to restore everything; at the end of the first volume,
> dump chokes
I can't be sure when I tried this last with other media, but I'm fairly
sure it's a more generic bug -- nothing to do with the type of media or
the device.
--
Greg A. Woods
+1 416 218-0098 VE3TCP <gwoods@acm.org> <robohack!woods>
Planix, Inc. <woods@planix.com>; Secrets of the Weird <woods@weird.com>
From: Hubert Feyrer <feyrer@rfhs8012.fh-regensburg.de>
To: smd@ebone.net
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: bin/10826: restore does not DTRT at end of volume on DVD-RAM
Date: Tue, 15 Aug 2000 04:45:27 +0200 (MET DST)
On Mon, 14 Aug 2000 smd@ebone.net wrote:
> dump -b 2 -B 2430000 -f /dev/rcd0d MD5 *.gz
Out of curiosity, which DVD-RAM is that, and do you happen to know since
when there's code to write to DVD-RAMs?
I happen to have a Panasonic 5.6G DVD-RAM at work that didn't work some
while ago (1.4U?).
Regarding your PR, did you have better success with tar?
Does the size of the archive you write to the DVD make a difference?
E.g. a tar-archive with one small file vs. a tar-archive with several/many
files?
- Hubert
--
NetBSD - because Unix isn't just #include <linux.h>, i386, ILP32, ELF, ...!
From: smd@ebone.net (Sean Doran)
To: hubert.feyrer@informatik.fh-regensburg.de
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: bin/10826: restore does not DTRT at end of volume on DVD-RAM
Date: Tue, 15 Aug 2000 10:53:06 +0200 (CEST)
| Out of curiosity, which DVD-RAM is that, and do you happen to know since
| when there's code to write to DVD-RAMs?
ahc0: target 1 synchronous at 10.0MHz, offset = 0xf
cd0 at scsibus1 target 1 lun 0: <MATSHITA, PD-2 LF-D100, A113> SCSI2 5/cdrom rem
ovable
It's been working "forever", at least since early 1.4 days.
On another box:
ahc1: target 2 synchronous at 10.0MHz, offset = 0xf
scsipi_inqmatch: 2/5/1 <, , >
cd1 at scsibus1 target 2 lun 0: <TOSHIBA, SD-W1101 DVD-RAM, 1033> SCSI2 5/cdrom re
movable
This also works. I've not had any kind of problem writing to /dev/rcdXd
since the days I used to use CD-R.
| Regarding your PR, did you have better success with tar?
Both tar and restore fail in exactly the same way: if I do multivolume,
I lose. If I keep stuff only within the 2.6GB/side, things Just Work,
and check out with MD5.
Worse, I cannot use pax-as-tar or gtar to write out a 2.4GB dump file.
I will raise a separate PR for this RSN.
| Does the size of the archive you write to the DVD make a difference?
| E.g. a tar-archive with one small file vs. a tar-archive with several/many
| files?
Good question. I will check that also.
Sean.
From: Sean Doran <smd@ebone.net>
To: hubert.feyrer@informatik.fh-regensburg.de
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: bin/10826: restore does not DTRT at end of volume on DVD-RAM
Date: 16 Aug 2000 19:43:22 +0200
Hubert Feyrer <feyrer@rfhs8012.fh-regensburg.de> writes:
> Regarding your PR, did you have better success with tar?
> Does the size of the archive you write to the DVD make a difference?
> E.g. a tar-archive with one small file vs. a tar-archive with several/many
> files?
I examined pax rather than tar.
What I've found is that "pax -r" loses at the end of the
DVD-RAM medium, where the kernel reports this:
cd0(ahc0:1:0): Check Condition on CDB: 0x08 12 99 92 04 00
SENSE KEY: Illegal Request
INFO FIELD: 1218960
COMMAND INFO: 83886080 (0x5000000)
ASC/ASCQ: Logical Block Address Out of Range
which I would have expected pax would consider in an
EOM-like way. It kind-of does, asking for a volume
change, but when the volume change happens, pax doesn't
resynchronize, and loses the spilled-over portion of the
last file on the previous volume.
Examination with dd seems to show that all the right bits
are actually on the medium.
Is this the same with any removeable (re)writeable medium?
Sean.
pax -r -v -f /dev/rcd0d './Various Artists - In Dust We Trust - Invisible Compilation 1997'
...
./Various Artists - In Dust We Trust - Invisible Compilation 1997/14 Lick - Hollowed.mp3
./Various Artists - In Dust We Trust - Invisible Compilation 1997/15 Evil Mothers - Orisha (The Stork Mix).mp3
pax: Failed read on archive volume 1 <Invalid argument>
pax: Attempting to recover from an archive read failure.
pax: Failed read on archive volume 1 <Invalid argument>
pax: Attempting to recover from an archive read failure.
pax: Failed read on archive volume 1 <Invalid argument>
pax: Attempting to recover from an archive read failure.
pax: Failed read on archive volume 1 <Invalid argument>
pax: Attempting to recover from an archive read failure.
pax: Failed read on archive volume 1 <Invalid argument>
pax: Attempting to recover from an archive read failure.
pax: Failed read on archive volume 1 <Invalid argument>
pax: Attempting to recover from an archive read failure.
pax: Failed read on archive volume 1 <Invalid argument>
pax: Attempting to recover from an archive read failure.
pax: Failed read on archive volume 1 <Invalid argument>
pax: Attempting to recover from an archive read failure.
pax: Failed read on archive volume 1 <Invalid argument>
pax: Attempting to recover from an archive read failure.
pax: Failed read on archive volume 1 <Invalid argument>
pax: Attempting to recover from an archive read failure.
pax: Failed read on archive volume 1 <Invalid argument>
pax: Attempting to recover from an archive read failure.
pax: Failed read on archive volume 1 <Invalid argument>
pax: Archive read error limit (10) reached
pax: ustar vol 1, 805 files, 2496430080 bytes read, 0 bytes written in 102 secs (24474804 bytes/sec)
ATTENTION! pax archive volume change required.
/dev/rcd0d ready for archive volume: 2
Load the NEXT STORAGE MEDIA (if required)
Type "y" to continue, "." to quit pax, or "s" to switch to new device.
If you cannot change storage media, type "s"
Is the device ready and online? > unknown command, try again
Type "y" to continue, "." to quit pax, or "s" to switch to new device.
If you cannot change storage media, type "s"
Is the device ready and online? > y
pax: Archive I/O error. Trying to recover.
./Various Artists - In Dust We Trust - Invisible Compilation 1997/01 Sheep on Drugs - X-Lover (Critter Mix).mp3
./Various Artists - In Dust We Trust - Invisible Compilation 1997/06 Evil Mothers - You Had Enough (ripped Up).mp3
...
stored versus extracted:
15c15
< MD5 (15 Evil Mothers - Orisha (The Stork Mix).mp3) = 62a5c4131c89253f9b149ef12e98aeea
---
> MD5 (15 Evil Mothers - Orisha (The Stork Mix).mp3) = f061466706348b4e2d89f6800d8a0a17
-r--r--r-- 1 smd wheel 4559868 Oct 23 1999 15 Evil Mothers - Orisha (The Stork Mix).mp3
-r-------- 1 smd wheel 1568768 Oct 23 1999 15 Evil Mothers - Orisha (The Stork Mix).mp3
From: Sean Doran <smd@ebone.net>
To: hubert.feyrer@informatik.fh-regensburg.de
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: bin/10826: restore does not DTRT at end of volume on DVD-RAM
Date: 16 Aug 2000 19:58:09 +0200
Hubert Feyrer <feyrer@rfhs8012.fh-regensburg.de> writes:
> Does the size of the archive you write to the DVD make a difference?
> E.g. a tar-archive with one small file vs. a tar-archive with several/many
> files?
pax / tar/ cpio / dump / pkgsrc/archivers/gcpio / pkgsrc/archivers/gtar
files written to the DVD-RAM such that the entire archive fits on one
volume work perfectly, and pass the MD5 test.
archive files from the above formats that are written to
the DVD-RAM such that multiple volumes are needed, all
have synchronization problems at the start of the 2nd
volume, losing the file that spills over.
I imagine this could be tested with a vnd drive, yes?
1. create empty file
2. vndconfig over it
3. archive into it until the archiver says "switch volumes"
and waits for user input
4. vndunconfig
5. create next empty file
6. goto 2.
following the complementary process when reading.
(incidentally, tar and gtar would not even write a 2.35GByte
file into a tar file, whether argument to "-f" was a
device or an ordinary file or pipe).
: crasse (es) ; ls -l
total 2433184
-rw-r--r-- 1 root wheel 16028682 Aug 13 20:24 crasse.root.0.gz
-rw-r--r-- 1 root wheel 2353193522 Aug 13 21:20 crasse.u.0.gz
-rw-r--r-- 1 root wheel 121929728 Aug 14 02:40 crasse.var.0.gz
: crasse (es) ; /usr/bin/time -l tar -cf - . | tar -tvf -
drwxr-xr-x root/wheel 0 Aug 14 02:38 2000 ./
-rw-r--r-- root/wheel -1941773774 Aug 13 21:20 2000 crasse.u.0.gz
-rw-r--r-- root/wheel 16028682 Aug 13 20:24 2000 crasse.root.0.gz
-rw-r--r-- root/wheel 121929728 Aug 14 02:40 2000 crasse.var.0.gz
9.70 real 0.00 user 0.91 sys
0 maximum resident set size
0 average shared memory size
0 average unshared data size
0 average unshared stack size
84 page reclaims
0 page faults
0 swaps
2640 block input operations
3 block output operations
13473 messages sent
0 messages received
0 signals received
34489 voluntary context switches
21 involuntary context switches
: crasse (es) ; /usr/bin/time -l pax -w . | pax -v
drwxr-xr-x 2 root wheel 0 Aug 14 02:38 .
-rw-r--r-- 1 root wheel 2353193522 Aug 13 21:20 ./crasse.u.0.gz
-rw-r--r-- 1 root wheel 16028682 Aug 13 20:24 ./crasse.root.0.gz
-rw-r--r-- 1 root wheel 121929728 Aug 14 02:40 ./crasse.var.0.gz
251.63 real 0.18 user 18.34 sys
0 maximum resident set size
0 average shared memory size
0 average unshared data size
0 average unshared stack size
95 page reclaims
0 page faults
0 swaps
75267 block input operations
3 block output operations
243277 messages sent
0 messages received
0 signals received
633037 voluntary context switches
311 involuntary context switches
pax: ustar vol 1, 4 files, 2491156480 bytes read, 0 bytes written in 251 secs (9924926 bytes/sec)
/usr/bin/tar loses its brains totally when trying to read
from the pax process above, presumably because the USTAR
header is different?
I suppose this last bit could be a separate PR. -:)
Sean.
From: Hubert Feyrer <feyrer@rfhs8012.fh-regensburg.de>
To: Sean Doran <smd@ebone.net>
Cc: gnats-bugs@gnats.netbsd.org
Subject: Re: bin/10826: restore does not DTRT at end of volume on DVD-RAM
Date: Thu, 17 Aug 2000 01:23:05 +0200 (MET DST)
On 16 Aug 2000, Sean Doran wrote:
> Is this the same with any removeable (re)writeable medium?
Sorry, I cannot tell you (I mainly wondered about support for DVD-?RAM, I
don't know about the internals). I think you should ask again on
tech-kern, pointing at your PR.
- Hubert
--
NetBSD - because Unix isn't just #include <linux.h>, i386, ILP32, ELF, ...!
From: smd@ebone.net (Sean Doran by way of Erik E. Fair)
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/10826: restore does not DTRT at end of volume on DVD-RAM
Date: Thu, 7 Sep 2000 14:08:53 -0700
Also please see bin/10848.
Sean.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.