NetBSD Problem Report #45444

From ryo_on@yk.rim.or.jp  Sat Oct  8 14:28:08 2011
Return-Path: <ryo_on@yk.rim.or.jp>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id 4513F63B955
	for <gnats-bugs@gnats.netbsd.org>; Sat,  8 Oct 2011 14:28:08 +0000 (UTC)
Message-Id: <20111008142808.4513F63B955@www.NetBSD.org>
Date: Sat,  8 Oct 2011 14:28:08 +0000 (UTC)
From: ryoon@NetBSD.org
Reply-To: ryoon@NetBSD.org
To: gnats-bugs@gnats.NetBSD.org
Subject: bootstrap failure on OpenIndiana 151a
X-Send-Pr-Version: 3.95

>Number:         45444
>Category:       pkg
>Synopsis:       bootstrap failure on OpenIndiana 151a
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    pkg-manager
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Oct 08 14:30:00 +0000 2011
>Closed-Date:    Tue Jun 12 08:49:04 +0000 2012
>Last-Modified:  Thu Jun 21 06:05:02 +0000 2012
>Originator:     Ryo ONODERA
>Release:        SunOS 5.11
>Organization:

>Environment:


System: SunOS openindiana 5.11 oi_151a i86pc i386 i86pc Solaris
Architecture: i386
Machine: i86pc
>Description:
On OpenIndiana/i386 151a, bmake self test fails.
As far as I know, SunPro is not provided to OpenIndiana 151a.

# gcc -v
Reading specs from /usr/sfw/lib/gcc/i386-pc-solaris2.11/3.4.3/specs
Configured with: /home/jt/sfw-central/oi_151_dev-il_rebuild/usr/src/cmd/gcc/gcc-3.4.3/configure --prefix=/usr/sfw --with-as=/usr/sfw/bin/gas --with-gnu-as --with-ld=/usr/ccs/bin/ld --without-gnu-ld --enable-languages=c,c++,f77,objc --enable-shared
Thread model: posix
gcc version 3.4.3 (csl-sol210-3_4-20050802)

# LANG=C ./bootstrap --workdir /tmp/bs
(snip)
cd /tmp/bs/bmake/SunOS/unit-tests && /tmp/bs/bmake/SunOS/bmake -r -m / TEST_MAKE=/tmp/bs/bmake/SunOS/bmake test
/tmp/bs/bmake/SunOS/bmake -f Makefile > test.out 2>&1
/usr/gnu/bin/diff -u /tmp/bs/bmake/unit-tests/test.exp test.out
--- /tmp/bs/bmake/unit-tests/test.exp   2011-10-08 23:08:39.867642311 +0900
+++ test.out    2011-10-08 23:09:57.555441448 +0900
@@ -75,11 +75,11 @@
 shared.2.1
 shared.2.99
 shared.2.99
+cycle.1.99
 make: Graph cycles through `cycle.2.99'
 make: Graph cycles through `cycle.2.98'
 make: Graph cycles through `cycle.2.97'
 cycle.1.99
-cycle.1.99
 .for with :S;... OK
 b2af338b
 3360ac65
*** Error code 1

Stop.
bmake: stopped in /tmp/bs/bmake/SunOS/unit-tests
*** Error code 1

Stop.
bmake: stopped in /tmp/bs/bmake/SunOS
===> exited with status 1
aborted.

>How-To-Repeat:
On OpenIndiana/i386 151a, bootstrap pkgsrc.

>Fix:

I have no idea.


>Release-Note:

>Audit-Trail:
From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/45444: bootstrap failure on OpenIndiana 151a
Date: Sun, 9 Oct 2011 11:14:08 +0000

 On Sat, Oct 08, 2011 at 02:30:00PM +0000, ryoon@NetBSD.org wrote:
  > --- /tmp/bs/bmake/unit-tests/test.exp   2011-10-08 23:08:39.867642311 +0900
  > +++ test.out    2011-10-08 23:09:57.555441448 +0900
  > @@ -75,11 +75,11 @@
  >  shared.2.1
  >  shared.2.99
  >  shared.2.99
  > +cycle.1.99
  >  make: Graph cycles through `cycle.2.99'
  >  make: Graph cycles through `cycle.2.98'
  >  make: Graph cycles through `cycle.2.97'
  >  cycle.1.99
  > -cycle.1.99
  >  .for with :S;... OK
  >  b2af338b
  >  3360ac65
  > *** Error code 1

 This isn't a new problem, but nobody can ever remember what causes
 it... the next time someone tracks it down, can we figure out how to
 add the info to the make tests?

 -- 
 David A. Holland
 dholland@netbsd.org

From: Ryo ONODERA <ryo_on@yk.rim.or.jp>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/45444: bootstrap failure on OpenIndiana 151a
Date: Tue, 11 Oct 2011 21:06:45 +0900 (JST)

 Hi,

 From: David Holland <dholland-pbugs@netbsd.org>, Date: Sun,  9 Oct 2011 11:15:04 +0000 (UTC)

 > The following reply was made to PR pkg/45444; it has been noted by GNATS.
 > 
 > From: David Holland <dholland-pbugs@netbsd.org>
 > To: gnats-bugs@NetBSD.org
 > Cc: 
 > Subject: Re: pkg/45444: bootstrap failure on OpenIndiana 151a
 > Date: Sun, 9 Oct 2011 11:14:08 +0000
 > 
 >  On Sat, Oct 08, 2011 at 02:30:00PM +0000, ryoon@NetBSD.org wrote:
 >   > --- /tmp/bs/bmake/unit-tests/test.exp   2011-10-08 23:08:39.867642311 +0900
 >   > +++ test.out    2011-10-08 23:09:57.555441448 +0900
 >   > @@ -75,11 +75,11 @@
 >   >  shared.2.1
 >   >  shared.2.99
 >   >  shared.2.99
 >   > +cycle.1.99
 >   >  make: Graph cycles through `cycle.2.99'
 >   >  make: Graph cycles through `cycle.2.98'
 >   >  make: Graph cycles through `cycle.2.97'
 >   >  cycle.1.99
 >   > -cycle.1.99
 >   >  .for with :S;... OK
 >   >  b2af338b
 >   >  3360ac65
 >   > *** Error code 1
 >  
 >  This isn't a new problem, but nobody can ever remember what causes
 >  it... the next time someone tracks it down, can we figure out how to
 >  add the info to the make tests?

 I want to understand bmake and its test mechanism.

 Thank you.

 --
 Ryo ONODERA // ryo_on@yk.rim.or.jp
 PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3


From: "Filip Hajny" <filip@joyent.com>
To: <gnats-bugs@NetBSD.org>
Cc: =?iso-8859-1?Q?Mads_Wors=F8e_Duun?= <mads@worsoe.dk>,
	"David Holland" <dholland-pkgtech@netbsd.org>,
	"Packages Technical List NetBSD Discussion" <tech-pkg@NetBSD.org>,
	"Emile Heitor" <imil@home.imil.net>,
	<david@l8s.co.uk>
Subject: Re: pkg/45444
Date: Fri, 17 Feb 2012 20:45:37 +0100

 I have figured out the culprit here, though I'm not sure
 about the cause yet. Turns out that with the following
 patch, pkgsrc bootstraps just fine on SmartOS (I take it
 it will work the same way on Illumos/OpenIndiana).
 I haven't figured out yet which part of xpg4 is biting
 us here, but I'll keep looking.

 -F

 --- a/bootstrap/bootstrap
 +++ b/bootstrap/bootstrap
 @@ -661,9 +661,6 @@ QNX)
         machine_arch=`uname -p | sed -e 's/x86/i386/'`
         ;;
  SunOS)
 -       if [ -d "/usr/xpg4/bin" ]; then
 -               overpath="/usr/xpg4/bin:$overpath"
 -       fi
         root_group=root
         need_bsd_install=yes
         if [ -x "/usr/gnu/bin/awk" ]; then

From: "Jonathan Perkin" <jperkin@joyent.com>
To: <gnats-bugs@NetBSD.org>
Cc: 
Subject: Re: pkg/45444
Date: Thu, 7 Jun 2012 15:58:38 +0100

 I dug further into this today, and have narrowed down the problem, but
 as yet cannot explain it.

 The offending section is in unit-tests/dotwait:

   all:
   .for t in ${TESTS}
   	@${.MAKE} -f ${THISMAKEFILE} -j4 $t | grep -v "^--- "
   .endfor

 According to truss(1), the grep is never executed when /usr/xpg4/bin
 is first in $PATH.  However, and oddly, if the grep call is changed to
 a full path (and apparently to any version of grep), then it's fine -
 for example:

   all:
   .for t in ${TESTS}
   	@${.MAKE} -f ${THISMAKEFILE} -j4 $t | /usr/xpg4/bin/grep -v "^--- "
   .endfor

 works fine, as does /usr/bin/grep (PATH=/usr/xpg4/bin:/usr/bin).

 We will most likely go ahead with Filip's change, once I have verified
 it's ok on older Solaris, and I probably won't spend any more time
 trying to figure out what exactly is happening here - but if someone
 wants to give me some hints I'm happy to try (adding debug MAKEFLAGS
 didn't provide any further enlightenment).

 Regards,

 -- 
 Jonathan Perkin  -  Joyent, Inc.  -  www.joyent.com

From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/45444
Date: Thu, 7 Jun 2012 20:21:17 +0000

 On Thu, Jun 07, 2012 at 03:00:08PM +0000, Jonathan Perkin wrote:
  >  I dug further into this today, and have narrowed down the problem, but
  >  as yet cannot explain it.
  >  
  >  The offending section is in unit-tests/dotwait:
  >  
  >    all:
  >    .for t in ${TESTS}
  >    	@${.MAKE} -f ${THISMAKEFILE} -j4 $t | grep -v "^--- "
  >    .endfor
  >  
  >  According to truss(1), the grep is never executed when /usr/xpg4/bin
  >  is first in $PATH.  However, and oddly, if the grep call is changed to
  >  a full path (and apparently to any version of grep), then it's fine -
  >  for example:
  >  
  >    all:
  >    .for t in ${TESTS}
  >    	@${.MAKE} -f ${THISMAKEFILE} -j4 $t | /usr/xpg4/bin/grep -v "^--- "
  >    .endfor
  >  
  >  works fine, as does /usr/bin/grep (PATH=/usr/xpg4/bin:/usr/bin).

 That is bizarre.

 Is this happening because make is passing the whole thing to the
 shell, and the shell is losing the second half of the pipeline, or
 because make is incorrectly thinking it can execute the command
 directly and then doing it wrong? It should be fairly straightforward
 to check this in the truss output.

 -- 
 David A. Holland
 dholland@netbsd.org

From: "Jonathan Perkin" <jperkin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/45444 CVS commit: pkgsrc/bootstrap
Date: Fri, 8 Jun 2012 12:15:29 +0000

 Module Name:	pkgsrc
 Committed By:	jperkin
 Date:		Fri Jun  8 12:15:29 UTC 2012

 Modified Files:
 	pkgsrc/bootstrap: bootstrap

 Log Message:
 Remove /usr/xpg4/bin from $PATH on Solaris, it causes weird issues in the
 bmake test suite, and is unnecessary.  Instead, ensure we use the xpg4 id.

 Tested on Solaris 9 and SmartOS.  Fixes PR#45444.


 To generate a diff of this commit:
 cvs rdiff -u -r1.175 -r1.176 pkgsrc/bootstrap/bootstrap

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Jonathan Perkin" <jperkin@joyent.com>
To: <gnats-bugs@NetBSD.org>
Cc: 
Subject: Re: pkg/45444
Date: Fri, 8 Jun 2012 14:46:00 +0100

 Ok, looking further at the truss output, the difference is that the
 broken case doesn't vfork, whereas the working one does:

   Working (PATH=/usr/bin):

     vforkx(0)
     ..
     vforkx()
     ..
     execve("/usr/xpg4/bin/sh", 0x0808A4E0, 0x08047CC4)  argc = 3
      argv: /usr/xpg4/bin/sh -c
       /tmp/pkgsrc-xpg4/bmake/SunOS/bmake -f /tmp/pkgsrc-xpg4/bmake/unit-tests/dotwait -j4 cycle | grep -v "^--- "
     ..
     execve("/usr/bin/amd64/ksh93", 0x08047C98, 0x08047CA8)  argc = 3
      argv: /usr/xpg4/bin/sh -c
       /tmp/pkgsrc-xpg4/bmake/SunOS/bmake -f /tmp/pkgsrc-xpg4/bmake/unit-tests/dotwait -j4 cycle | grep -v "^--- "
     ..
     forkx(0)
     ..
     forkx()
     ..
     vforkx(0)
     ..
     execve("/tmp/pkgsrc-xpg4/bmake/SunOS/bmake", 0x0049F430, 0x0049F5A8)  argc = 5
      argv: /tmp/pkgsrc-xpg4/bmake/SunOS/bmake -f
       /tmp/pkgsrc-xpg4/bmake/unit-tests/dotwait -j4 cycle
     ..
     vforkx()
     ..
     execve("/usr/bin/grep", 0x0049F540, 0x0049F5C8)  argc = 3
      argv: grep -v ^---

   Broken (PATH=/usr/xpg4/bin:/usr/bin):

     vforkx(0)
     ..
     vforkx()
     ..
     execve("/usr/xpg4/bin/sh", 0x0808A4E0, 0x08047CB4)  argc = 3
      argv: /usr/xpg4/bin/sh -c
       /tmp/pkgsrc-xpg4/bmake/SunOS/bmake -f /tmp/pkgsrc-xpg4/bmake/unit-tests/dotwait -j4 cycle | grep -v "^--- "
     ..
     execve("/usr/bin/amd64/ksh93", 0x08047C88, 0x08047C98)  argc = 3
      argv: /usr/xpg4/bin/sh -c
       /tmp/pkgsrc-xpg4/bmake/SunOS/bmake -f /tmp/pkgsrc-xpg4/bmake/unit-tests/dotwait -j4 cycle | grep -v "^--- "
     ..
     forkx(0)
     ..
     forkx()
     ..
     execve("/tmp/pkgsrc-xpg4/bmake/SunOS/bmake", 0x0049F430, 0x0049F5A8)  argc = 5
      argv: /tmp/pkgsrc-xpg4/bmake/SunOS/bmake -f
       /tmp/pkgsrc-xpg4/bmake/unit-tests/dotwait -j4 cycle

 If you want more details the complete outputs are at

   http://www.netbsd.org/~jperkin/bmake-xpg4/truss-bin.out
   http://www.netbsd.org/~jperkin/bmake-xpg4/truss-xpg4.out

 truss-bin='PATH=/usr/bin', truss-xpg4='PATH=/usr/xpg4/bin:/usr/bin'.

 Regards,

 -- 
 Jonatha Perkin  -  Joyent, Inc.  -  www.joyent.com

From: "Jonathan Perkin" <jperkin@joyent.com>
To: <gnats-bugs@NetBSD.org>
Cc: 
Subject: Re: pkg/45444
Date: Fri, 8 Jun 2012 15:12:26 +0100

 For completeness there is also

   http://www.netbsd.org/~jperkin/bmake-xpg4/truss-fix.out

 which is using the broken PATH but with /usr/xpg4/bin/grep hard-coded
 in unit-tests/dotwait, making the test pass..

 -- 
 Jonathan Perkin  -  Joyent, Inc.  -  www.joyent.com

State-Changed-From-To: open->closed
State-Changed-By: jperkin@NetBSD.org
State-Changed-When: Tue, 12 Jun 2012 08:49:04 +0000
State-Changed-Why:
Closing this PR as the reported problem is fixed.  If we wish to pursue the
reasons why xpg4 in $PATH causes the issues we should do that separately.


From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/45444
Date: Thu, 21 Jun 2012 06:04:15 +0000

 On Fri, Jun 08, 2012 at 01:50:04PM +0000, Jonathan Perkin wrote:
  >  Ok, looking further at the truss output, the difference is that the
  >  broken case doesn't vfork, whereas the working one does:
  >  [snip]

 If I understand that correctly, it's the shell that's misbehaving and
 not make. In which case as far as I'm concerned it's Somebody Else's
 Problem :-)

 -- 
 David A. Holland
 dholland@netbsd.org

>Unformatted:

Home	PR Database Search