NetBSD Problem Report #51152

From reed@reedmedia.net  Wed May 18 19:51:06 2016
Return-Path: <reed@reedmedia.net>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 9EA297A3D5
	for <gnats-bugs@gnats.NetBSD.org>; Wed, 18 May 2016 19:51:06 +0000 (UTC)
Message-Id: <21316-1463601065@reedmedia.net>
Date: Wed, 18 May 2016 14:51:06 -0500
From: reed@reedmedia.net
Reply-To: reed@reedmedia.net
To: gnats-bugs@gnats.NetBSD.org
Subject: pkg_alternatives shebang to shebang interpreters failure
X-Send-Pr-Version: 3.95

>Number:         51152
>Category:       pkg
>Synopsis:       pkg_alternatives via shebang won't continue and will use previous shell
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          analyzed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed May 18 19:55:00 +0000 2016
>Closed-Date:    
>Last-Modified:  Tue Jul 05 03:15:01 +0000 2016
>Originator:     reed@reedmedia.net
>Release:        NetBSD 6.0_STABLE
>Organization:

>Environment:
Architecture: x86_64
Machine: amd64
>Description:
First saw this on a Darwin Kernel Version 15.0.0 system. OS X something.

I used the packages from Joyent. It includes pkg_alternatives.

I saw that running a script that has #!/opt/pkg/bin/python2 in the top
line resulted in bash errors like:
import: not found

I was able to reproduce with NetBSD and OS X with some simple scripts.
A #! interpreter cannot be another #! interpreter. It will just use
the same old shell and the second #! won't be used.

execve(2) manual on my old NetBSD says: "The interpreter named by
interpreter must not itself be an interpreter file."

And same thing in script(7) manual.

Note it says execution will fail. But it doesn't fail on NetBSD
nor OS X. It just uses the original interpreter.

This was discussed with jperkin on #pkgsrc IRC. Thanks!

My workaround will be to use specific python path and not the 
path to the pkg_alternatives wrapper.
>How-To-Repeat:
Use pkg_alternatives with a python2 (not python2.7 specific).
>Fix:
Rewrite pkg_alternatives to be a compiled binary instead
of using a sh interpreter.


>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: pkg-manager->jmmv
Responsible-Changed-By: hauke@NetBSD.org
Responsible-Changed-When: Thu, 19 May 2016 07:00:32 +0000
Responsible-Changed-Why:
Over to maintainer.


From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/51152: pkg_alternatives shebang to shebang interpreters
 failure
Date: Sun, 22 May 2016 04:59:45 +0000

 On Wed, May 18, 2016 at 07:55:00PM +0000, reed@reedmedia.net wrote:
  > Rewrite pkg_alternatives to be a compiled binary instead
  > of using a sh interpreter.

 ...why is it anything other than just a symlink?

 -- 
 David A. Holland
 dholland@netbsd.org

Responsible-Changed-From-To: jmmv->pkg-manager
Responsible-Changed-By: jmmv@NetBSD.org
Responsible-Changed-When: Sun, 03 Jul 2016 02:41:13 +0000
Responsible-Changed-Why:
Sorry. I won't be rewriting pkg_alternatives so I cannot keep this bug.


State-Changed-From-To: open->analyzed
State-Changed-By: jmmv@NetBSD.org
State-Changed-When: Sun, 03 Jul 2016 02:41:13 +0000
State-Changed-Why:
Yup... This was years ago, so if I had to guess as to why this problem
exists, it's probably because I did not know that shell scripts could
not act as full-blown interpreters back then.

Rewritting the wrapper in C is a good answer and should be relatively
straightforward; it's a pretty rudimentary program.

The reason this is not simply a symlink is because the wrapper allows
users (not just root) to customize their preferred alternatives via
configuration files, and those are processed when the wrappers are
invoked.  (This is different to, say, Debian's alternatives system,
which is only configurable by root system-wide and thus a symlink
suffices.  Now, whether what pkg_alternatives does is a good idea or
not, I won't go into.)

Unfortunately, I have no time to work on this, so I cannot keep this
bug.  I'm going to mark it as analyzed as the problem is understood
though.


From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: Julio Merino <jmmv@netbsd.org>
Subject: Re: pkg/51152 (pkg_alternatives via shebang won't continue and will
 use previous shell)
Date: Tue, 5 Jul 2016 01:38:50 +0000

 On Sun, Jul 03, 2016 at 02:41:13AM +0000, jmmv@NetBSD.org wrote:
  > Yup... This was years ago, so if I had to guess as to why this problem
  > exists, it's probably because I did not know that shell scripts could
  > not act as full-blown interpreters back then.
  > 
  > Rewritting the wrapper in C is a good answer and should be relatively
  > straightforward; it's a pretty rudimentary program.

 Unfortunately it looks like it needs to be recompiled on the fly
 outside the pkgsrc compilation environment... that or the whole thing
 needs to be restructured so there can be one binary wrapper compiled
 in advance and a way to look up what's currently baked into the script
 file at creation time.

 That or do some kind of binary patching horror.

 None of these are very appealing. :-/

 (Or we could fix the kernel to allow nested interpreters, but that
 doesn't help other pkgsrc platforms.)

 any suggestions you might have would be welcome...

  > The reason this is not simply a symlink is because the wrapper allows
  > users (not just root) to customize their preferred alternatives via
  > configuration files, and those are processed when the wrappers are
  > invoked.  (This is different to, say, Debian's alternatives system,
  > which is only configurable by root system-wide and thus a symlink
  > suffices.  Now, whether what pkg_alternatives does is a good idea or
  > not, I won't go into.)

 Seems reasonable, actually.

 -- 
 David A. Holland
 dholland@netbsd.org

From: Julio Merino <julio@meroh.net>
To: David Holland <dholland-pbugs@netbsd.org>
Cc: gnats-bugs@netbsd.org
Subject: Re: pkg/51152 (pkg_alternatives via shebang won't continue and will
 use previous shell)
Date: Mon, 4 Jul 2016 23:14:26 -0400

 On Mon, Jul 4, 2016 at 9:38 PM, David Holland <dholland-pbugs@netbsd.org> wrote:
 >
 > On Sun, Jul 03, 2016 at 02:41:13AM +0000, jmmv@NetBSD.org wrote:
 >  > Yup... This was years ago, so if I had to guess as to why this problem
 >  > exists, it's probably because I did not know that shell scripts could
 >  > not act as full-blown interpreters back then.
 >  >
 >  > Rewritting the wrapper in C is a good answer and should be relatively
 >  > straightforward; it's a pretty rudimentary program.
 >
 > Unfortunately it looks like it needs to be recompiled on the fly
 > outside the pkgsrc compilation environment... that or the whole thing
 > needs to be restructured so there can be one binary wrapper compiled
 > in advance and a way to look up what's currently baked into the script
 > file at creation time.
 >
 > That or do some kind of binary patching horror.

 From a second look... I see these three classes of "settings" in wrapper.sh:

 * ID, ROOT_USER: These are static variables defined at "configure
 time" so they can just be baked into the binary.

 * WRAPPER: The name of the actual wrapper. Could possibly just be
 derived from "basename of argv[0]" (?), in which case the wrapper
 binary would suit any alternative.

 * DB_FILE, CONFFILE: Paths to files. These are built from a static
 setting determined at "configure time" (e.g. path to the system-wide
 configuration directory, or a path to a home directory), and a dynamic
 piece. The latter seems to be just the value of WRAPPER. If that's
 correct, then the same as in the previous applies.

 In other words: if the wrapper code is changed to rely on the contents
 of argv[0], it should be possible to reuse the exact same binary for
 all alternatives.

 -- 
 Julio Merino -- http://julio.meroh.net/

>Unformatted:

Home	PR Database Search