NetBSD Problem Report #56631

From greywolf@starwolf.com  Sun Jan 16 03:20:17 2022
Return-Path: <greywolf@starwolf.com>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 8F58E1A923A
	for <gnats-bugs@gnats.NetBSD.org>; Sun, 16 Jan 2022 03:20:17 +0000 (UTC)
Message-Id: <20220116032015.D321C10C98@valentina.serenity.starwolf.com>
Date: Sat, 15 Jan 2022 19:20:15 -0800 (PST)
From: greywolf@starwolf.com
Reply-To: greywolf@starwolf.com
To: gnats-bugs@NetBSD.org
Subject: rdump fails to back up to remote file
X-Send-Pr-Version: 3.95

>Number:         56631
>Category:       bin
>Synopsis:       rdump fails to back up to file on remote host
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    bin-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Jan 16 03:25:00 +0000 2022
>Last-Modified:  Sat Jan 22 15:00:03 +0000 2022
>Originator:     Greywolf
>Release:        NetBSD 9.99.93
>Organization:
Meh.
>Environment:
System: NetBSD valentina.serenity.starwolf.com 9.99.93 NetBSD 9.99.93 (VALENTINA) #11: Thu Jan 13 11:45:10 PST 2022 greywolf@valentina.serenity.starwolf.com:/usr/src/sys/arch/amd64/compile/VALENTINA amd64
Architecture: x86_64
Machine: amd64
>Description:
# rdump 0uaf dumphost:/backupdir/_root.0 /
  DUMP: Found /dev/rld0a on / in /etc/fstab
  DUMP: Date of this level 0 dump: Sat Jan 15 18:49:08 2022
  DUMP: Date of last level 0 dump: the epoch
  DUMP: Dumping /dev/rld0a (/) to /backupdir/_root.0 on host dumphost
  DUMP: Label: none
  DUMP: mapping (Pass I) [regular files]
  DUMP: mapping (Pass II) [directories]
  DUMP: estimated 341149 tape blocks.
rdump: Lost connection to remote host.

# export | grep RCMD_CMD
RCMD_CMD
# echo "<${RCMD_CMD}>"
</usr/bin/ssh>
#

>How-To-Repeat:
	rdump 0uaf remote:/path /filesystem
>Fix:

>Audit-Trail:
From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/56631: rdump fails to back up to remote file
Date: Sun, 16 Jan 2022 07:14:57 -0000 (UTC)

 greywolf@starwolf.com writes:

 ># rdump 0uaf dumphost:/backupdir/_root.0 /
 >  DUMP: Found /dev/rld0a on / in /etc/fstab
 >  DUMP: Date of this level 0 dump: Sat Jan 15 18:49:08 2022
 >  DUMP: Date of last level 0 dump: the epoch
 >  DUMP: Dumping /dev/rld0a (/) to /backupdir/_root.0 on host dumphost
 >  DUMP: Label: none
 >  DUMP: mapping (Pass I) [regular files]
 >  DUMP: mapping (Pass II) [directories]
 >  DUMP: estimated 341149 tape blocks.
 >rdump: Lost connection to remote host.


 This can fail for several reasons.

 RCMD_CMD is called as $RCMD_CMD -4 -l $user $host /etc/rmt, so
 that's IPv4 only. The $host is also canonicalized, e.g. a short
 hostname (without fqdn) gets a '.' appended.

 rcmd() uses a single file descriptor (stdin, stdout, stderr are
 all the same), so ssh messages on stderr botch the RMT protocol.

 The command invoked is /etc/rmt which can manage a tape drive.
 While it can write to a file, it doesn't create it.

 I was successful with an .ssh/config stanza of:

 Host mytapehost.*
 	LogLevel Quiet
 	ForwardX11 no
 	ForwardAgent no

 and pre-creating the output file on mytapehost. You can also
 wrap ssh like:

 /bin/sh
 exec ssh -qxa "$@"

 with a similar effect.


 N.B. rcmd(1) is unsuitable for testing as it ignores RCMD_CMD.

From: matthew green <mrg@eterna.com.au>
To: gnats-bugs@netbsd.org
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: re: bin/56631: rdump fails to back up to remote file
Date: Sun, 16 Jan 2022 19:17:34 +1100

 one problem is that the rmt protocol is kinda broken in that
 there's no clear definition of what is passed to open() when
 the remote processes does this.

 unlike local files, dump won't pass O_CREAT to the remote
 connection, so, if the file doesn't already exist it won't
 be used.

 from dump/dumprmt.c:

 rmtopen(const char *tapedevice, int mode, int verbose)
 {
         char buf[256];

         (void)snprintf(buf, sizeof buf, "O%s\n%d\n", tapedevice, mode);
         rmtstate = TS_OPEN;
         return (rmtcall(tapedevice, buf, verbose));
 }

 there are 3 callers:

         while (rmtopen(tape, 0, 0) < 0)
                         if (rmtopen(tape, 0, 0) >= 0) {
         while ((tapefd = (host ? rmtopen(tape, 2, 1) :

 note how they pass "0" or "2" as "int mode".  what do these
 0 and 2 magic numbers mean?  oh let's see in rmt(8):

 from rmt/rmt.c:

                 getstring(device, sizeof(device));
                 getstring(mode, sizeof(mode));
                 DEBUG2("rmtd: O %s %s\n", device, mode);
                 tape = open(device, atoi(mode),
                     S_IRUSR|S_IWUSR|S_IRGRP|S_IWGRP|S_IROTH|S_IWOTH);

 in other words, these values are passed to the remote host
 directly as the mode argument to open().  in netbsd (and most
 bsd i assume still, these days, and linux has the same in the
 generic configs, but i recall there was something sysV that
 changed these values):

 #define O_RDONLY        0x00000000      /* open for reading only */
 #define O_RDWR          0x00000002      /* open for reading and writing */

 but there's no O_CREAT, so remote files are not created.


 .mrg.

From: greywolf@starwolf.com
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/56631: rdump fails to back up to remote file
Date: Fri, 21 Jan 2022 23:39:03 -0800 (PST)

 Hi there!

 On 1/15/22 11:20 PM, Michael van Elst wrote:
 >  This can fail for several reasons.
 >  
 >  RCMD_CMD is called as $RCMD_CMD -4 -l $user $host /etc/rmt, so
 >  that's IPv4 only. The $host is also canonicalized, e.g. a short
 >  hostname (without fqdn) gets a '.' appended.

 Seems a bit archaic; should it not be resolving the host to a FQDN?

 >  rcmd() uses a single file descriptor (stdin, stdout, stderr are
 >  all the same), so ssh messages on stderr botch the RMT protocol.
 >

 Oooff.  Good to know, even if this seems to violate the POLA

 >  The command invoked is /etc/rmt which can manage a tape drive.
 >  While it can write to a file, it doesn't create it.

 Apparently it *can* create it if passed the proper argument; I tested
 this by handing the following to rmt locally:

 O/path/to/file
 514

 [514 = 0x202 = O_CREAT|O_RDWR]

 I'm just not quite sure what to patch to make this work or to make it at
 least an option to pass thru from rdump to rmt.

 >  I was successful with an .ssh/config stanza of:
 >  
 >  Host mytapehost.*
 >  	LogLevel Quiet
 >  	ForwardX11 no
 >  	ForwardAgent no
 >  
 >  and pre-creating the output file on mytapehost. You can also
 >  wrap ssh like:
 >  
 >  /bin/sh
 >  exec ssh -qxa "$@"
 >  
 >  with a similar effect.

 >  N.B. rcmd(1) is unsuitable for testing as it ignores RCMD_CMD.

 Thank you for the heads up; rcmd(1) didn't even register on my radar.

 (The fact that RCMD_CMD is taken as a full literal string with no
 tokenization (i.e. one cannot include arguments in RCMD_CMD) seems
 ALSO a bug, or at least a candidate for enhancement, but that's
 something for another day.)


 				--*greywolf;

From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/56631: rdump fails to back up to remote file
Date: Sat, 22 Jan 2022 09:21:24 -0000 (UTC)

 greywolf@starwolf.com writes:

 > On 1/15/22 11:20 PM, Michael van Elst wrote:
 > >  This can fail for several reasons.
 > >  
 > >  RCMD_CMD is called as $RCMD_CMD -4 -l $user $host /etc/rmt, so
 > >  that's IPv4 only. The $host is also canonicalized, e.g. a short
 > >  hostname (without fqdn) gets a '.' appended.
 > 
 > Seems a bit archaic; should it not be resolving the host to a FQDN?

 I don't see a reason to even canonicalize the hostname, probably
 some compatibility stuff.

 Restricting to IPv4 comes from the API, and that might also be
 some ancient compatibility issue, but probably just a cheap method
 to add IPv6 support.

 rcmd(...) calls rcmd_af(..., AF_INET).

 and rcmd_af() translates AF_INET into the -4 option and AF_INET6
 into the -6 option of rsh.


 > >  rcmd() uses a single file descriptor (stdin, stdout, stderr are
 > >  all the same), so ssh messages on stderr botch the RMT protocol.
 > >
 > 
 > Oooff.  Good to know, even if this seems to violate the POLA

 Yes, but not that uncommon. scp or rsync aren't transparent either
 and you can easily break them by sending output in e.g. .login.
 rsh also, unlike ssh, doesn't necesarily have a stderr channel.


 > I'm just not quite sure what to patch to make this work or to make it at
 > least an option to pass thru from rdump to rmt.

 You need to change multiple things.

 RMT protocol might need some abstraction to the mode argument to be portable.

 rdump needs an option to specify the mode, best some abstracted way,
 maybe also to distinguish between tapes and files.

 /etc/rmt might need to distinguish between files and devices, also
 to filter the mode. It could also try to emulate tape operations
 on and with files.

 rcmd() needs to understand an arbitrary protocol and not massage
 hostnames itself. There are proper IPv6 APIs.


 > (The fact that RCMD_CMD is taken as a full literal string with no
 > tokenization (i.e. one cannot include arguments in RCMD_CMD) seems
 > ALSO a bug, or at least a candidate for enhancement, but that's
 > something for another day.)

 I think that is intentional. The command isn't interpreted by a shell,
 but passed to execlp() directly. This avoids any shell injection issues,
 and for a command regularily run by root, that's better.

 You could think about splitting RCMD_CMD into words and prepend argv[],
 but that's limited. You could create a wrapper script and point RCMD_CMD
 to it.

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: bin/56631: rdump fails to back up to remote file
Date: Sat, 22 Jan 2022 21:56:02 +0700

     Date:        Sat, 22 Jan 2022 09:25:02 +0000 (UTC)
     From:        mlelstv@serpens.de (Michael van Elst)
     Message-ID:  <20220122092502.13A9C1A923B@mollari.NetBSD.org>


   |  RMT protocol might need some abstraction to the mode argument to
   |  be portable.
   |  
   |  rdump needs an option to specify the mode, best some abstracted way,
   |  maybe also to distinguish between tapes and files.
   |  
   |  /etc/rmt might need to distinguish between files and devices, also
   |  to filter the mode. It could also try to emulate tape operations
   |  on and with files.

 rmt should remain for remote tape access.  That is what it is
 designed for, as tapes are (were) special, capacity generally
 quite limited, so needing rewinding and swapping in the middle
 of output ... the same could be said of old removable other
 media (floppies...) but tapes were more special in that it was
 often difficult to detect pending media exhaustion.

 dump needs to deal with all of that, hence the remote tape stuff
 which allows dump to manage changing the "tape" when it calculates
 that the time for that has arrived.

 For writing to remote files one should simply have dump
 write to stdout, then pipe that to netcat (or whatever you
 like for this purpose) rather than attempting to mangle
 the rmt protocol to become a general remote file access
 protocol.

 This PR should be closed.  There is nothing to fix here.

Home	PR Database Search