NetBSD Problem Report #42484

From wiz@yt.nih.at  Sun Dec 20 14:59:45 2009
Return-Path: <wiz@yt.nih.at>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id 4DA1963C4ED
	for <gnats-bugs@gnats.netbsd.org>; Sun, 20 Dec 2009 14:59:45 +0000 (UTC)
Message-Id: <20091220145935.9D27839FE4C@yt.nih.at>
Date: Sun, 20 Dec 2009 15:59:35 +0100 (CET)
From: Thomas Klausner <wiz@NetBSD.org>
Reply-To: Thomas Klausner <wiz@NetBSD.org>
To: gnats-bugs@gnats.NetBSD.org
Subject: tmpfs problem: make: Cannot open `.' (Permission denied)
X-Send-Pr-Version: 3.95

>Number:         42484
>Category:       kern
>Synopsis:       tmpfs problem: make: Cannot open `.' (Permission denied)
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Dec 20 15:00:00 +0000 2009
>Closed-Date:    
>Last-Modified:  Wed Sep 06 07:00:52 +0000 2023
>Originator:     Thomas Klausner
>Release:        NetBSD 5.99.22
>Organization:
Curiosity is the very basis of education and if you tell me that
curiosity killed the cat, I say only that the cat died nobly.
- Arnold Edinborough
>Environment:


System: NetBSD yt.nih.at 5.99.22 NetBSD 5.99.22 (YT) #29: Sat Dec 12 01:20:02 CET 2009 wiz@yt.nih.at:/archive/cvs/src/sys/arch/amd64/compile/obj/YT amd64
Architecture: x86_64
Machine: amd64
>Description:
I'm regularly running bulk builds of about 1200 packages completely in
a tmpfs on 5.99.22/amd64.

tmpfs on /home/wiz/sandbox type tmpfs (local)
/bin on /home/wiz/sandbox/bin type null (read-only, local)
/sbin on /home/wiz/sandbox/sbin type null (read-only, local)
/lib on /home/wiz/sandbox/lib type null (read-only, local)
/libexec on /home/wiz/sandbox/libexec type null (read-only, local)
/usr/X11R7 on /home/wiz/sandbox/usr/X11R7 type null (read-only, local)
/usr/bin on /home/wiz/sandbox/usr/bin type null (read-only, local)
/usr/games on /home/wiz/sandbox/usr/games type null (read-only, local)
/usr/include on /home/wiz/sandbox/usr/include type null (read-only, local)
/usr/lib on /home/wiz/sandbox/usr/lib type null (read-only, local)
/usr/libdata on /home/wiz/sandbox/usr/libdata type null (read-only, local)
/usr/libexec on /home/wiz/sandbox/usr/libexec type null (read-only, local)
/usr/share on /home/wiz/sandbox/usr/share type null (read-only, local)
/usr/sbin on /home/wiz/sandbox/usr/sbin type null (read-only, local)
/var/mail on /home/wiz/sandbox/var/mail type null (read-only, local)
/archive/cvs/src on /home/wiz/sandbox/usr/src type null (read-only, local)
/archive/cvs/pkgsrc on /home/wiz/sandbox/usr/pkgsrc type null (local)
/archive/cvs/xsrc on /home/wiz/sandbox/usr/xsrc type null (read-only, local)
/disk/1/archive/packages/5.99.22 on /home/wiz/sandbox/packages type null (local)
/disk/1/archive/distfiles on /home/wiz/sandbox/distfiles type null (local)

Sometimes single packages or a few packages immediately after another
fail with "make: Cannot open `.' (Permission denied)". Usually, during
configure, but sometimes during the build as today in kdenetwork4:

[ 72%] Generating smsadd.cpp
[ 72%] Built target iris_kopete
[ 72%] Building CXX object kopete/protocols/bonjour/CMakeFiles/kopete_bonjour.dir/bonjouraddcontactpage.o
[ 72%] Generating smsadd.moc.cpp
[ 72%] Building CXX object
+kget/transfer-plugins/bittorrent/CMakeFiles/kget_bittorrentfactory.dir/libbtcore/torrent/torrentcontrol.o
[ -3%] Building CXX object kopete/protocols/testbed/CMakeFiles/kopete_testbed.dir/testbedaccount.o
Linking CXX shared module ../../../lib/kopete_wlm.so
Scanning dependencies of target kopete_sms
[ -3%] Building CXX object kopete/protocols/groupwise/CMakeFiles/kopete_groupwise.dir/gwconnector.o
make: Cannot open `.' (Permission denied)
--- kopete/protocols/sms/CMakeFiles/kopete_sms.dir/all ---
*** [kopete/protocols/sms/CMakeFiles/kopete_sms.dir/all] Error code 1
A failure has been detected in another branch of the parallel make

I think this might be a bug in tmpfs, since I've never seen this when
I was using ffs as an underlying file system for the sandbox.

The problem is, that, while the problem happens about in every second
or third bulk build I run, I have no sure way to reproduce it. Perhaps
it's a timing or locking related problem.

It usually causes a few packages in a row to fail, but then the build
continues and finishes with most packages built.

On stdout, I see:

[1120/1122] Starting build of  trac-0.11.5
cd: can't cd to /scratch/www/trac/work
*** Error code 2

Stop.
make: stopped in /usr/pkgsrc/www/trac
[1120/1122] Failed to build    trac-0.11.5


It offen happens during the build of openoffice3. The last time it
happened, the first error in the openoffice3 build log was:


--- NOOPTFILES OVER ---
dmake:  Error: --
`/scratch/misc/openoffice3/work/OOO310_m19/solver/310/unxbsdx3
.pro/inc/310minor.mk' not found, and can't be made
Running processes: 15
making .dpslo
Running processes: 16
/scratch/misc/openoffice3/work/OOO310_m19/starmath/source
dmake:  Error: --
`/scratch/misc/openoffice3/work/OOO310_m19/solver/310/unxbsdx3
.pro/inc/310minor.mk' not found, and can't be made
Running processes: 15
Running processes: 16
/scratch/misc/openoffice3/work/OOO310_m19/sw/source/core/SwNumberTree
if test -e ../../../unxbsdx3.pro/slo/slideshowviewimpl.o ; then touch
../../../u
nxbsdx3.pro/slo/slideshowviewimpl.obj ; fi
making .dpslo
/scratch/misc/openoffice3/work/OOO310_m19/solver/310/unxbsdx3.pro/bin/makedepend
 @/tmp/mk18527i > ../../../unxbsdx3.pro/misc/s_slideshowviewimpl.dpcc


The setup is as follows:
tmpfs sandbox containing all files for the bulk build.
12GB RAM in the machine, 24GB swap, so it should be sufficient.
Sometimes oo3 build works even in this setup. There are complete
bulk build runs where it doesn't occur.

There are enough inodes available:
tmpfs          34408948    1312212   33096736   3%    10564 162937777 0% /home/wiz/sandbox

>How-To-Repeat:
Run a limited bulk build as described above and put in some big
packages like openoffice3.
>Fix:


>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: kern-bug-people->jmmv
Responsible-Changed-By: wiz@NetBSD.org
Responsible-Changed-When: Sun, 20 Dec 2009 15:02:48 +0000
Responsible-Changed-Why:
jmmv said to create this PR :)


From: Thomas Klausner <wiz@NetBSD.org>
To: NetBSD bugtracking <gnats-bugs@NetBSD.org>
Cc: 
Subject: Re: kern/42484
Date: Wed, 23 Jun 2010 09:01:01 +0200

 On Wed, Jun 23, 2010 at 08:55:27AM +0200, Thomas Klausner wrote:
 > After rmind's changes from a few hours ago, I tried again, and see a
 > different error now:
 > /scratch/x11/kdelibs4/work/kdelibs-4.4.4/kioslave/http/http_cache_cleaner.cpp:864: fatal error: error writing to /var/tmp//ccJnzaB1.s: No space left on device compilation terminated.
 > 
 > I've never seen that before, and I didn't see the previous error in
 > this pbulk run, so perhaps that's a better error message for my
 > previously seen error.

 When I'm watching in top, I don't see the Memory "Free" line falling
 below 1GB.
 I've mounted the tmpfs without an explicit size:
    mount -t tmpfs tmpfs /some/path
  Thomas

From: Thomas Klausner <wiz@NetBSD.org>
To: NetBSD bugtracking <gnats-bugs@NetBSD.org>
Cc: 
Subject: Re: kern/42484
Date: Sun, 4 Jul 2010 13:27:08 +0200

 On Wed, Jun 23, 2010 at 09:01:01AM +0200, Thomas Klausner wrote:
 > After rmind's changes from a few hours ago, I tried again, and see a
 > different error now:
 > /scratch/x11/kdelibs4/work/kdelibs-4.4.4/kioslave/http/http_cache_cleaner.cpp:864: fatal error: error writing to /var/tmp//ccJnzaB1.s: No space left on device compilation terminated.
 > 
 > I've never seen that before, and I didn't see the previous error in
 > this pbulk run, so perhaps that's a better error message for my
 > previously seen error.

 More fixes later, I'm back to my original error:
 make: Cannot open `.' (Permission denied)

  Thomas

From: Antti Kantee <pooka@cs.hut.fi>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/42484
Date: Fri, 16 Jul 2010 13:59:10 +0300

 tests/fs/vfs/t_renamerace:tmpfs_renamerace rarely (less than 1/20 times
 on my machine) fails with:

 	failed: create: Operation not supported

 I wonder if it's related to this PR?  Or just that tmpfs rename is a
 bit shabby (which also might be related to this PR ;).

From: "Mindaugas Rasiukevicius" <rmind@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/42484 CVS commit: src/sys/fs/tmpfs
Date: Sun, 29 May 2011 22:29:08 +0000

 Module Name:	src
 Committed By:	rmind
 Date:		Sun May 29 22:29:07 UTC 2011

 Modified Files:
 	src/sys/fs/tmpfs: tmpfs.h tmpfs_subr.c tmpfs_vfsops.c tmpfs_vnops.c

 Log Message:
 - Rework and document inode reference counting.  Also document inode life
   cycle (destruction part).  Perform link counting in tmpfs_dir_attach()
   and tmpfs_dir_detach(), instead of alloc/free and arbitrary places.
   Fixes PR/44285, PR/44288, PR/44657 and likely PR/42484.

 - Fix the race between the lookup and inode destruction.  Fixes PR/43167
   and its duplicates PR/40088, PR/40757.

 - Improve tmpfs_rename() locking a little, fix kqueue event notifications
   and also fix PR/43617.  Add simplistic tmpfs_parentcheck_p(); to be
   expanded and used for further rename() locking fixes.

 - Cache directory entry "hint" in the tmpfs node, add tmpfs_dir_cached(),
   and thus avoid unnecessary lookup in tmpfs_remove() and tmpfs_rmdir().

 - Set correct _PC_FILESIZEBITS value in tmpfs_pathconf().  Fixes PR/43576.

 - Few minor fixes.


 To generate a diff of this commit:
 cvs rdiff -u -r1.43 -r1.44 src/sys/fs/tmpfs/tmpfs.h
 cvs rdiff -u -r1.70 -r1.71 src/sys/fs/tmpfs/tmpfs_subr.c
 cvs rdiff -u -r1.50 -r1.51 src/sys/fs/tmpfs/tmpfs_vfsops.c
 cvs rdiff -u -r1.84 -r1.85 src/sys/fs/tmpfs/tmpfs_vnops.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

Responsible-Changed-From-To: jmmv->rmind
Responsible-Changed-By: rmind@NetBSD.org
Responsible-Changed-When: Sun, 29 May 2011 23:16:13 +0000
Responsible-Changed-Why:


State-Changed-From-To: open->feedback
State-Changed-By: rmind@NetBSD.org
State-Changed-When: Sun, 29 May 2011 23:16:13 +0000
State-Changed-Why:
Thomas, can you try latest -current and confirm whether it still happens?

Thanks.


State-Changed-From-To: feedback->open
State-Changed-By: wiz@NetBSD.org
State-Changed-When: Thu, 02 Jun 2011 07:49:47 +0000
State-Changed-Why:
I've updated to 5.99.52 from around 6 UTC/20110530 and started a bulk build.
I still got many package failures due to
make: Cannot open `.' (Permission denied)


From: Thomas Klausner <wiz@NetBSD.org>
To: NetBSD bugtracking <gnats-bugs@NetBSD.org>
Cc: 
Subject: Re: kern/42484 (tmpfs problem: make: Cannot open `.' (Permission
 denied))
Date: Mon, 21 Jan 2013 10:44:56 +0100

 On Thu, Jun 02, 2011 at 07:49:48AM +0000, Thomas Klausner wrote:
 > I still got many package failures due to
 > make: Cannot open `.' (Permission denied)

 This particular error has become quite rare.

 However, this night I had a reboot with:

 savecore: reboot after panic: panic: kernel diagnostic assertion "cookie != TMPFS_DIRCOOKIE_DOT" failed: filAeR N"I/aNrGc:h iSvPeL/ cNvOsT/ sLrOcW/EsRyEsD/ OsN/t mSpYfs/CtAmLp
 Lf s.h"6,7  l4i nEeX I9T3  4
  6

 The machine was doing a bulk build on tmpfs.
  Thomas

From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/42484 (tmpfs problem: make: Cannot open `.' (Permission
 denied))
Date: Mon, 21 Jan 2013 10:59:58 +0000

 On Mon, Jan 21, 2013 at 09:50:10AM +0000, Thomas Klausner wrote:
  >  However, this night I had a reboot with:
  >  
  >  savecore: reboot after panic: panic: kernel diagnostic assertion "cookie != TMPFS_DIRCOOKIE_DOT" failed: filAeR N"I/aNrGc:h iSvPeL/ cNvOsT/ sLrOcW/EsRyEsD/ OsN/t mSpYfs/CtAmLp
  >  Lf s.h"6,7  l4i nEeX I9T3  4
  >   6

 That's an ... interesting pair of things to happen simultaneously.

 -- 
 David A. Holland
 dholland@netbsd.org

From: Thomas Klausner <wiz@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/42484 (tmpfs problem: make: Cannot open `.' (Permission
 denied))
Date: Mon, 21 Jan 2013 12:36:47 +0100

 On Mon, Jan 21, 2013 at 11:00:31AM +0000, David Holland wrote:
 >  On Mon, Jan 21, 2013 at 09:50:10AM +0000, Thomas Klausner wrote:
 >   >  However, this night I had a reboot with:
 >   >  
 >   >  savecore: reboot after panic: panic: kernel diagnostic assertion "cookie != TMPFS_DIRCOOKIE_DOT" failed: filAeR N"I/aNrGc:h iSvPeL/ cNvOsT/ sLrOcW/EsRyEsD/ OsN/t mSpYfs/CtAmLp
 >   >  Lf s.h"6,7  l4i nEeX I9T3  4
 >   >   6
 >  
 >  That's an ... interesting pair of things to happen simultaneously.

 I'm not sure if I made a wrong impression here -- I didn't see the
 make error today, I was sleeping when it happened. I added this
 message to this PR because it's "my tmpfs PR" -- it could be two
 different problems.
  Thomas

From: Thomas Klausner <wiz@NetBSD.org>
To: NetBSD bugtracking <gnats-bugs@NetBSD.org>
Cc: 
Subject: Re: kern/42484 (tmpfs problem: make: Cannot open `.' (Permission
 denied))
Date: Mon, 21 Jan 2013 14:01:43 +0100

 On Mon, Jan 21, 2013 at 11:40:06AM +0000, Thomas Klausner wrote:
 >  I'm not sure if I made a wrong impression here -- I didn't see the
 >  make error today, I was sleeping when it happened. I added this
 >  message to this PR because it's "my tmpfs PR" -- it could be two
 >  different problems.

 I've filed PR 47480 for the cookie panic.
  Thomas

From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/42484 (tmpfs problem: make: Cannot open `.' (Permission
 denied))
Date: Wed, 23 Jan 2013 09:18:13 +0000

 On Mon, Jan 21, 2013 at 11:40:06AM +0000, Thomas Klausner wrote:
  >  >   >  savecore: reboot after panic: panic: kernel diagnostic assertion "cookie != TMPFS_DIRCOOKIE_DOT" failed: filAeR N"I/aNrGc:h iSvPeL/ cNvOsT/ sLrOcW/EsRyEsD/ OsN/t mSpYfs/CtAmLp
  >  >   >  Lf s.h"6,7  l4i nEeX I9T3  4
  >  >   >   6
  >  >  
  >  >  That's an ... interesting pair of things to happen simultaneously.
  >  
  >  I'm not sure if I made a wrong impression here -- I didn't see the
  >  make error today, I was sleeping when it happened. I added this
  >  message to this PR because it's "my tmpfs PR" -- it could be two
  >  different problems.

 I meant the two things in that panic message, acutally. But I'll take
 it to the other PR.

 -- 
 David A. Holland
 dholland@netbsd.org

State-Changed-From-To: open->feedback
State-Changed-By: rmind@NetBSD.org
State-Changed-When: Fri, 08 Nov 2013 16:30:20 +0000
State-Changed-Why:
wiz@ said he has not seen this for a while.  Let's wait more with the recent
changes and see if it really disappeared?

I still need to check PR/43615 i.e. audit what kauth is doing.  It might be
related with this problem.


State-Changed-From-To: feedback->closed
State-Changed-By: wiz@NetBSD.org
State-Changed-When: Thu, 05 Dec 2013 07:25:53 +0000
State-Changed-Why:
Didn't see this recently. Thanks!


State-Changed-From-To: closed->analyzed
State-Changed-By: wiz@NetBSD.org
State-Changed-When: Tue, 12 Jan 2016 14:39:45 +0000
State-Changed-Why:
Problem is back, for months.

Usually happens when building libreoffice, sometimes when building thunderbird


State-Changed-From-To: analyzed->feedback
State-Changed-By: rmind@NetBSD.org
State-Changed-When: Sat, 30 Jul 2016 18:11:56 +0000
State-Changed-Why:
src/sys/arch/x86/x86/pmap.c rev 1.190 probably fixed this.
Do you still see the problem?


From: Thomas Klausner <wiz@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/42484 (tmpfs problem: make: Cannot open `.' (Permission
 denied))
Date: Sun, 31 Jul 2016 10:31:05 +0200

 I recently noticed that I hadn't seen this problem for a while.

 January sounds too long ago though. I would have said since sometime
 in May or June.

 Anyway, looks fixed, thank you!
  Thomas

State-Changed-From-To: feedback->closed
State-Changed-By: rmind@NetBSD.org
State-Changed-When: Sun, 31 Jul 2016 23:39:40 +0000
State-Changed-Why:
Assume fixed.


State-Changed-From-To: closed->open
State-Changed-By: wiz@NetBSD.org
State-Changed-When: Sat, 22 Oct 2016 07:14:15 +0000
State-Changed-Why:
This problem reappeared in 7.99.39 around the start of
October.
Stronger than ever, in my last bulk build it broke 80 (of ~2800) packages


From: "J. Hannken-Illjes" <hannken@eis.cs.tu-bs.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/42484 (tmpfs problem: make: Cannot open `.' (Permission denied))
Date: Sat, 22 Oct 2016 12:04:28 +0200

 > On 22 Oct 2016, at 09:14, wiz@NetBSD.org wrote:
 >=20
 > Synopsis: tmpfs problem: make: Cannot open `.' (Permission denied)
 >=20
 > State-Changed-From-To: closed->open
 > State-Changed-By: wiz@NetBSD.org
 > State-Changed-When: Sat, 22 Oct 2016 07:14:15 +0000
 > State-Changed-Why:
 > This problem reappeared in 7.99.39 around the start of
 > October.
 > Stronger than ever, in my last bulk build it broke 80 (of ~2800) =
 packages

 Ich habe hier einen AMD64 Server mit 2 CPUs und 16GB unter -current und
 w=C3=BCrde es gerne selbst probieren.

 Hast Du ein Skript oder eine Step-by-step Anleitung f=C3=BCr Deinen Bulk
 Build im tmpfs?

 --
 J. Hannken-Illjes - hannken@eis.cs.tu-bs.de - TU Braunschweig (Germany)

State-Changed-From-To: open->closed
State-Changed-By: wiz@NetBSD.org
State-Changed-When: Thu, 03 Nov 2016 13:27:16 +0000
State-Changed-Why:
at least the current incarnation is not a tmpfs problem
see PR 51595 for details


Responsible-Changed-From-To: rmind->kern-bug-people
Responsible-Changed-By: wiz@NetBSD.org
Responsible-Changed-When: Wed, 06 Sep 2023 07:00:52 +0000
Responsible-Changed-Why:
rmind is an observer


State-Changed-From-To: closed->open
State-Changed-By: wiz@NetBSD.org
State-Changed-When: Wed, 06 Sep 2023 07:00:52 +0000
State-Changed-Why:
The problem appeared again


>Unformatted:

Home	PR Database Search