NetBSD Problem Report #51457

From www@NetBSD.org  Thu Sep  1 23:52:51 2016
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 8178D7A1B7
	for <gnats-bugs@gnats.NetBSD.org>; Thu,  1 Sep 2016 23:52:51 +0000 (UTC)
Message-Id: <20160901235250.194A17A2B2@mollari.NetBSD.org>
Date: Thu,  1 Sep 2016 23:52:50 +0000 (UTC)
From: mfpnb@plass-family.net
Reply-To: mfpnb@plass-family.net
To: gnats-bugs@NetBSD.org
Subject: Incorrect postinstall version-number sort options
X-Send-Pr-Version: www-1.0

>Number:         51457
>Category:       toolchain
>Synopsis:       Incorrect postinstall version-number sort options
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    kre
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Sep 01 23:55:00 +0000 2016
>Closed-Date:    Sat Sep 03 04:07:07 +0000 2016
>Last-Modified:  Sat Sep 03 04:07:07 +0000 2016
>Originator:     Michael Plass
>Release:        7.99.36
>Organization:
>Environment:
NetBSD mipi 7.99.36 NetBSD 7.99.36 (RPI2.201608220040Z) evbarm

>Description:
The command
  postinstall fix obsolete_stand
uses a pipeline that includes the step
  sort -t. -r -k1n -k2n -k3n
intended to put the highest three-part version number on the first
line of the output.

But, for example,

$ printf '7.9.1\n7.9.0\n7.9.10\n7.10.1\n' | sort -t. -r -k1n -k2n -k3n
7.9.10
7.9.1
7.9.0
7.10.1

see http://mail-index.netbsd.org/netbsd-users/2016/05/28/msg018532.html
for an explanation why.

The mis-sorting can cause the wrong collection of kernel modules to
be deleted.

To make it worse, other systems interpret these options
differently. The same command on FreeBSD 11 gives

7.9.0
7.9.1
7.9.10
7.10.1

and on OS X (El Capitan),

7.10.1
7.9.0
7.9.1
7.9.10

The latter agrees with a very old GNU/Linux system.

The portability matters, since the script is invoked on the host system 
during a cross-build of 'release'.



>How-To-Repeat:

>Fix:
A better set of options is suggested by

$ printf '7.9.1\n7.9.0\n7.9.10\n7.10.1\n' | sort -t. -r -n -k1,1 -k2,2 -k3,3
7.10.1
7.9.10
7.9.1
7.9.0

which gives the desired output on all the tested platforms.

It might be appropriate to spell the command $SORT in the postinstall
script, but I have not checked the details.

>Release-Note:

>Audit-Trail:
From: "Robert Elz" <kre@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/51457 CVS commit: src/usr.sbin/postinstall
Date: Fri, 2 Sep 2016 05:38:44 +0000

 Module Name:	src
 Committed By:	kre
 Date:		Fri Sep  2 05:38:44 UTC 2016

 Modified Files:
 	src/usr.sbin/postinstall: postinstall

 Log Message:
 PR toolchain/51457 -- use sort correctly


 To generate a diff of this commit:
 cvs rdiff -u -r1.203 -r1.204 src/usr.sbin/postinstall/postinstall

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

Responsible-Changed-From-To: toolchain-manager->kre
Responsible-Changed-By: kre@NetBSD.org
Responsible-Changed-When: Fri, 02 Sep 2016 05:42:18 +0000
Responsible-Changed-Why:
I am (for now) handling this PR


State-Changed-From-To: open->feedback
State-Changed-By: kre@NetBSD.org
State-Changed-When: Fri, 02 Sep 2016 05:42:18 +0000
State-Changed-Why:
Does it work correcty for you now?
(I did not s/sort/${SORT}/ -- someone else can decide if that is warranted.)


From: Michael Plass <mfpnb@plass-family.net>
To: gnats-bugs@NetBSD.org
Cc: Michael Plass <mfpnb@plass-family.net>
Subject: Re: toolchain/51457 (Incorrect postinstall version-number sort options)
Date: Fri, 2 Sep 2016 07:52:41 -0700

 > On Sep 1, 2016, at 10:42 PM, kre@NetBSD.org wrote:
 > 
 > Does it work correcty for you now?

 No, not for cross builds - On both FreeBSD 11 and OS X:

 $ printf '7.9.1\n7.9.0\n7.9.10\n7.10.1\n' | sort -t. -r -k1,1n -k2,2n -k3,3n
 7.9.0
 7.9.1
 7.9.10
 7.10.1

 This is (I think) because posix says that "If any modifier is attached to a
 field_start or to a field_end, no option shall apply to either."

 http://pubs.opengroup.org/onlinepubs/9699919799/utilities/sort.html

 The NetBSD sort does this differently, but then again the man page doesn't
 mention posix.

 So to use modifiers, it needs to be
   sort -t. -k1,1nr -k2,2nr -k3,3nr
 but is there a reason not to use
   sort -t. -nr -k1,1 -k2,2 -k3,3
 ?

 Thanks,
 - Michael

From: "Robert Elz" <kre@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/51457 CVS commit: src/usr.sbin/postinstall
Date: Fri, 2 Sep 2016 20:25:14 +0000

 Module Name:	src
 Committed By:	kre
 Date:		Fri Sep  2 20:25:14 UTC 2016

 Modified Files:
 	src/usr.sbin/postinstall: postinstall

 Log Message:
 PR toolchain/51457 -- be more posix compat with sort usage, though it
 (currently anyway) makes no difference on NetBSD.


 To generate a diff of this commit:
 cvs rdiff -u -r1.204 -r1.205 src/usr.sbin/postinstall/postinstall

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: toolchain/51457 (Incorrect postinstall version-number sort options)
Date: Sat, 03 Sep 2016 03:32:30 +0700

     Date:        Fri,  2 Sep 2016 14:55:01 +0000 (UTC)
     From:        Michael Plass <mfpnb@plass-family.net>
     Message-ID:  <20160902145501.793C27A286@mollari.NetBSD.org>

   |  This is (I think) because posix says that "If any modifier is attached to a
   |  field_start or to a field_end, no option shall apply to either."

 I actually wondered about that when I made the change, but wasn't sure if
 any implementations actually acted that way.

   |  So to use modifiers, it needs to be
   |    sort -t. -k1,1nr -k2,2nr -k3,3nr

 I started to do it that way, but ...

   |  but is there a reason not to use
   |    sort -t. -nr -k1,1 -k2,2 -k3,3
   |  ?

 Not really I guess, so I changed it to that (before I even got this far
 into reading your message...)   I have a slight preference towards not
 using global opts unless they are (and can rationally) be applied to the
 whole record (so global -r makes sense, as here the reverse sort is wanted
 however the sort gets done - I'd use the r tak on a field selector if just
 that field was to be sorted in descending order) but the records themselves
 are not numeric, just the fields, so global -n just seems "wrong" ... but
 the posix rules make that combination fail, so ...

 OK now?

 kre

From: Michael Plass <mfpnb@plass-family.net>
To: gnats-bugs@NetBSD.org
Cc: Michael Plass <mfpnb@plass-family.net>
Subject: Re: toolchain/51457 (Incorrect postinstall version-number sort options)
Date: Fri, 2 Sep 2016 18:00:52 -0700

 > On Sep 2, 2016, at 1:35 PM, Robert Elz <kre@munnari.OZ.AU> wrote:
 > 
 > OK now?

 Yes - I did a cross-build with a dummy 7.99.35/modules directory,
 and this time it deleted the correct one.

 Pullups probably aren't mandatory, since the problem wouldn't affect
 actual installs until a version number hit two digits, and there's
 an easy workaround for cross-builds. But it is just a one-line change.

 Thanks,
 - Michael

From: Michael Plass <mfpnb@plass-family.net>
To: gnats-bugs@NetBSD.org
Cc: Michael Plass <mfpnb@plass-family.net>
Subject: Re: toolchain/51457 (Incorrect postinstall version-number sort options)
Date: Fri, 2 Sep 2016 18:10:29 -0700

 Oh, and there does not seem to be an nbsort in the toolsdir, so using
 $SORT for the cross builds does not help.

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: toolchain/51457 (Incorrect postinstall version-number sort options)
Date: Sat, 03 Sep 2016 11:03:22 +0700

     Date:        Sat,  3 Sep 2016 01:10:01 +0000 (UTC)
     From:        Michael Plass <mfpnb@plass-family.net>
     Message-ID:  <20160903011001.8ADF97A2C1@mollari.NetBSD.org>

   |  Pullups probably aren't mandatory, since the problem wouldn't affect
   |  actual installs until a version number hit two digits,

 Yes, NetBSD 7 will be dead before NetBSD 10 arrives, and I don't see
 a .10 appearing anywhere in the NetBSD 6 or 7 releases, so I think we
 can just pass on pullups of this (reduce churn.)

 kre

State-Changed-From-To: feedback->closed
State-Changed-By: kre@NetBSD.org
State-Changed-When: Sat, 03 Sep 2016 04:07:07 +0000
State-Changed-Why:
Submitter confirms that problem is fixed


>Unformatted:

Home	PR Database Search