NetBSD Problem Report #46382

From he@granny-smith.urc.uninett.no  Thu Apr 26 13:03:32 2012
Return-Path: <he@granny-smith.urc.uninett.no>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	by www.NetBSD.org (Postfix) with ESMTP id C46A463C4B1
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 26 Apr 2012 13:03:32 +0000 (UTC)
Message-Id: <20120426114702.683B731CBD@granny-smith.urc.uninett.no>
Date: Thu, 26 Apr 2012 13:47:02 +0200 (CEST)
From: he@NetBSD.org
Reply-To: he@NetBSD.org
To: gnats-bugs@gnats.NetBSD.org
Subject: lintpkgsrc may overflow /bin/sh command line length
X-Send-Pr-Version: 3.95

>Number:         46382
>Category:       pkg
>Synopsis:       lintpkgsrc may overflow /bin/sh command line length
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    pkg-manager
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Apr 26 13:05:00 +0000 2012
>Last-Modified:  Thu Apr 26 16:15:02 +0000 2012
>Originator:     Havard Eidnes
>Release:        NetBSD 5.1
>Organization:
	None
>Environment:
System: NetBSD granny-smith.urc.uninett.no 5.1 NetBSD 5.1 (GRANNY-SMITH) #4: Fri Apr 16 16:03:14 CEST 2010 he@granny-smith.urc.uninett.no:/usr/obj/sys/arch/macppc/compile/GRANNY-SMITH macppc
Architecture: powerpc
Machine: macppc
>Description:
	The old pkgsrc build scripts try to clean out distfiles
	which are no longer referenced, or where the checksum does
	not match.  To do this task, they use lintpkgsrc, like so:

	lintpkgsrc -K /usr/pkgsrc/packages/powerpc \
		-P /usr/pkgsrc \
		-M /usr/pkgsrc/distfiles \
		-o -m -r

	However, in the phase where lintpkgsrc tries to checksum all
	the existing distfiles, the command line may overflow, causing
	this function to fail.  Apparently some use of xargs is required.

>How-To-Repeat:
	Have lots of old distfiles in a bulk pkgsrc build, watch
	lintpkgsrc bail out when trying to checksum the old distfiles,
	causing the old distfiles never to be cleaned up.
>Fix:
	Here is a draft fix which works, but which uses the "mktemp"
	utility.  It is possible or maybe even probable that this is
	not sufficiently portable, and that we'd instead have to resort
	to something like /tmp/pl-sumfiles.$$, which isn't quite as
	safe.  Suggestions welcome.

Index: lintpkgsrc.pl
===================================================================
RCS file: /cvsroot/pkgsrc/pkgtools/pkglint/files/lintpkgsrc.pl,v
retrieving revision 1.115
diff -u -p -r1.115 lintpkgsrc.pl
--- lintpkgsrc.pl	22 Nov 2007 09:36:38 -0000	1.115
+++ lintpkgsrc.pl	26 Apr 2012 11:40:14 -0000
@@ -1538,7 +1538,13 @@ sub scan_pkgsrc_distfiles_vs_distinfo($$
                 next;
             }

-            open( DIGEST, "digest $sum @{$sumfiles{$sum}}|" )
+	    my($tmpfile) = `mktemp /tmp/pl-sumfiles.XXXXXXX`;
+	    open(SFILES, ">" . $tmpfile);
+	    foreach my $f (@{$sumfiles{$sum}}) {
+		printf(SFILES "%s\n", $f);
+	    }
+	    close(SFILES);
+            open(DIGEST, "cat $tmpfile | xargs digest $sum |" )
               || fail("Run digest: $!");
             while (<DIGEST>) {
                 if (m/^$sum ?\(([^\)]+)\) = (\S+)/) {
@@ -1549,6 +1555,7 @@ sub scan_pkgsrc_distfiles_vs_distinfo($$
                 }
             }
             close(DIGEST);
+	    unlink($tmpfile);
         }
         safe_chdir('/');    # Do not want to stay in $pkgdistdir
     }

>Audit-Trail:
From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: pkg/46382: lintpkgsrc may overflow /bin/sh command line length
Date: Thu, 26 Apr 2012 16:11:59 +0000

 On Thu, Apr 26, 2012 at 01:05:00PM +0000, he@NetBSD.org wrote:
  > 	Here is a draft fix which works, but which uses the "mktemp"
  > 	utility.  It is possible or maybe even probable that this is
  > 	not sufficiently portable, and that we'd instead have to resort
  > 	to something like /tmp/pl-sumfiles.$$, which isn't quite as
  > 	safe.  Suggestions welcome.

 You have a useless use of cat in there.

 But more importantly: does perl not allow something like

    open(DIGEST, "| xargs digest $sum |")

 so you can just feed the file list through? Or is Perl paralyzed by
 the same foolishness that disallows this kind of thing using popen(3)?

 (And failing that, doesn't Perl have mkstemp?) 

 -- 
 David A. Holland
 dholland@netbsd.org

Home	PR Database Search