NetBSD Problem Report #48385

From dholland@netbsd.org  Sun Nov 17 16:50:10 2013
Return-Path: <dholland@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 830CBA60ED
	for <gnats-bugs@gnats.NetBSD.org>; Sun, 17 Nov 2013 16:50:10 +0000 (UTC)
Message-Id: <20131117165010.164BF14A24E@mail.netbsd.org>
Date: Sun, 17 Nov 2013 16:50:10 +0000 (UTC)
From: dholland@NetBSD.org
Reply-To: dholland@NetBSD.org
To: gnats-bugs@gnats.NetBSD.org
Subject: gmake 4.0 loops on tmpfs
X-Send-Pr-Version: 3.95

>Number:         48385
>Category:       kern
>Synopsis:       gmake 4.0 loops on tmpfs
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    rmind
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Nov 17 16:55:00 +0000 2013
>Closed-Date:    Thu Dec 05 17:39:52 +0000 2013
>Last-Modified:  Thu Dec 05 17:39:52 +0000 2013
>Originator:     David A. Holland
>Release:        NetBSD 6.99.19 (20130404)
>Organization:
>Environment:
System: NetBSD valkyrie 6.99.19 NetBSD 6.99.19 (VALKYRIE) #13: Sun Nov  3 17:07:36 EST 2013  dholland@valkyrie:/usr/src/sys/arch/amd64/compile/VALKYRIE amd64
Architecture: x86_64
Machine: amd64
>Description:

With the enclosed makefile, running on tmpfs, gmake 4.0 loops forever
rerunning the recipe for objdir/depend.mk. It does not happen with
gmake 3.82.

I originally ran into this in audio/flite's bulk build.

There is some chance it reflects a problem in tmpfs, but that doesn't
seem very likely.

>How-To-Repeat:

Place the following on a tmpfs volume and run "gmake"; it will loop
forever.

   ------ begin Makefile ------
all:
	echo all

objdir:
	mkdir -p objdir || true

objdir/depend.mk: objdir
	rm -f objdir/depend.mk
	echo 'foo: bar' > objdir/depend.mk

include objdir/depend.mk
   ------ end Makefile ------

>Fix:

>Release-Note:

>Audit-Trail:
From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/48385: gmake 4.0 loops on tmpfs
Date: Sun, 17 Nov 2013 16:58:30 +0000

 On Sun, Nov 17, 2013 at 04:55:00PM +0000, dholland@NetBSD.org wrote:
  > rerunning the recipe for objdir/depend.mk. It does not happen with
  > gmake 3.82.

 Upstream report:

 http://savannah.gnu.org/bugs/index.php?40610

 -- 
 David A. Holland
 dholland@netbsd.org

From: David Holland <dholland-pbugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: pkg/48385: gmake 4.0 loops on tmpfs
Date: Sun, 17 Nov 2013 21:17:06 +0000

 On Sun, Nov 17, 2013 at 05:00:01PM +0000, David Holland wrote:
  >  On Sun, Nov 17, 2013 at 04:55:00PM +0000, dholland@NetBSD.org wrote:
  >   > rerunning the recipe for objdir/depend.mk. It does not happen with
  >   > gmake 3.82.
  >  
  >  Upstream report:
  >  
  >  http://savannah.gnu.org/bugs/index.php?40610

 So it looks like gmake going into an infinite loop is a symptom, and
 the trigger is that

    echo stuff > objdir/depend.mk

 when objdir/depend.mk doesn't yet exist leaves objdir newer than
 depend.mk, typically by about 1 ms. Then the make rule

    objdir/depend.mk: objdir

 causes gmake to rebuild depend.mk again, and of course it never
 stops. I think the reason this doesn't happen in gmake 3.82 is that
 gmake 3.82 doesn't read the nanoseconds field of the mtime.

 I think this qualifies as a tmpfs bug; the open() that creates
 depend.mk and therefore sets the directory timestamp necessarily comes
 before the write() that puts stuff into the file and sets the file
 timestamp. The timestamps ought to reflect this ordering, and don't.

 -- 
 David A. Holland
 dholland@netbsd.org

Responsible-Changed-From-To: pkg-manager->kern-bug-people
Responsible-Changed-By: dholland@NetBSD.org
Responsible-Changed-When: Sun, 17 Nov 2013 21:32:24 +0000
Responsible-Changed-Why:
core problem seems to be in tmpfs


Responsible-Changed-From-To: kern-bug-people->rmind
Responsible-Changed-By: dholland@NetBSD.org
Responsible-Changed-When: Sun, 17 Nov 2013 21:32:54 +0000
Responsible-Changed-Why:
by request (thanks!)


State-Changed-From-To: open->analyzed
State-Changed-By: rmind@NetBSD.org
State-Changed-When: Sun, 17 Nov 2013 22:32:12 +0000
State-Changed-Why:
It is also broken on netbsd-{4,5,6}, so probably been broken ~forever.
Can be reproduced with the following:

mkdir testdir ; echo meow > testdir/whatever
sleep 2
stat -f "%m" testdir ; stat -f "%m" testdir/whatever

tmpfs defers timestamp updates, e.g. the need to update the directory 
mtime is flagged on creat(3).  However, the actual update happens at some
point later, which may lead to directory having newer timestamp than the
created file. 

It is probably worth to audit the whole deferral logic.


From: David Holland <dholland@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/48385 (gmake 4.0 loops on tmpfs)
Date: Sun, 17 Nov 2013 23:01:04 +0000

 On Sun, Nov 17, 2013 at 10:32:12PM +0000, rmind@NetBSD.org wrote:
  > It is probably worth to audit the whole deferral logic.

 If it's like (or worse, based on) the similar logic in ffs, it's
 highly bogus and not easily fixed.

 (also, for tmpfs, what's the point of deferring timestamp updates
 anyway? fetching the current time less often?)

 -- 
 David A. Holland
 dholland@netbsd.org

From: "David A. Holland" <dholland@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/48385 CVS commit: pkgsrc/audio/flite
Date: Mon, 18 Nov 2013 02:22:30 +0000

 Module Name:	pkgsrc
 Committed By:	dholland
 Date:		Mon Nov 18 02:22:30 UTC 2013

 Modified Files:
 	pkgsrc/audio/flite: distinfo
 	pkgsrc/audio/flite/patches: patch-config_common__make__rules

 Log Message:
 Improve the workaround for what is now PR 48385.


 To generate a diff of this commit:
 cvs rdiff -u -r1.7 -r1.8 pkgsrc/audio/flite/distinfo
 cvs rdiff -u -r1.1 -r1.2 \
     pkgsrc/audio/flite/patches/patch-config_common__make__rules

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: analyzed->feedback
State-Changed-By: rmind@NetBSD.org
State-Changed-When: Sat, 23 Nov 2013 16:38:12 +0000
State-Changed-Why:
Can you cvs up and try (or, preferably, do extra testing)?


From: "Mindaugas Rasiukevicius" <rmind@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/48385 CVS commit: src/sys/fs/tmpfs
Date: Sat, 23 Nov 2013 16:35:32 +0000

 Module Name:	src
 Committed By:	rmind
 Date:		Sat Nov 23 16:35:32 UTC 2013

 Modified Files:
 	src/sys/fs/tmpfs: tmpfs.h tmpfs_fifoops.c tmpfs_rename.c
 	    tmpfs_specops.c tmpfs_subr.c tmpfs_vfsops.c tmpfs_vnops.c

 Log Message:
 - Simplify tmpfs_update(), eliminate tmpfs_note_t::tn_status and deferred
   timestamp updates.  Fix some incorrect updates and plug some missing ones.
   Should fix PR/48385.
 - tmpfs_rmdir: avoid O(n) scan when the directory is not empty and whiteout
   entries were never added.


 To generate a diff of this commit:
 cvs rdiff -u -r1.47 -r1.48 src/sys/fs/tmpfs/tmpfs.h
 cvs rdiff -u -r1.9 -r1.10 src/sys/fs/tmpfs/tmpfs_fifoops.c
 cvs rdiff -u -r1.5 -r1.6 src/sys/fs/tmpfs/tmpfs_rename.c
 cvs rdiff -u -r1.10 -r1.11 src/sys/fs/tmpfs/tmpfs_specops.c
 cvs rdiff -u -r1.89 -r1.90 src/sys/fs/tmpfs/tmpfs_subr.c
 cvs rdiff -u -r1.54 -r1.55 src/sys/fs/tmpfs/tmpfs_vfsops.c
 cvs rdiff -u -r1.106 -r1.107 src/sys/fs/tmpfs/tmpfs_vnops.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: feedback->closed
State-Changed-By: rmind@NetBSD.org
State-Changed-When: Thu, 05 Dec 2013 17:39:52 +0000
State-Changed-Why:
This has been fixed.


>Unformatted:

Home	PR Database Search