NetBSD Problem Report #60603

From www@netbsd.org  Sun Aug 16 13:59:14 2026
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mail.netbsd.org", Issuer "YR2" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 982881A923B
	for <gnats-bugs@gnats.NetBSD.org>; Sun, 16 Aug 2026 13:59:14 +0000 (UTC)
Message-Id: <20260816135913.303801A923C@mollari.NetBSD.org>
Date: Sun, 16 Aug 2026 13:59:13 +0000 (UTC)
From: campbell+netbsd@mumble.net
Reply-To: campbell+netbsd@mumble.net
To: gnats-bugs@NetBSD.org
Subject: Sisyphean vdrain task
X-Send-Pr-Version: www-1.0
X-From4GNATS: "campbell+netbsd@mumble.net via gnats" <gnats-admin@NetBSD.org>

>Number:         60603
>Category:       kern
>Synopsis:       Sisyphean vdrain task
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Aug 16 14:00:01 +0000 2026
>Last-Modified:  Tue Aug 18 16:00:02 +0000 2026
>Originator:     Taylor R Campbell
>Release:        current, 11
>Organization:
Desirable Vnode Classifieds, Inc.
>Environment:
>Description:

	The vdrain task runs in the background to keep the number of
	vnodes below desiredvnodes.  It is scheduled asynchronously
	from various vnode operations like vrele.  If it fails to bring
	the number of vnodes below its target, it pauses for a tick and
	tries again.

    701 /*
    702  * threadpool task to keep the number of vnodes below desiredvnodes.
    703  */
    704 static void
    705 vdrain_task(struct threadpool_job *job)
    706 {
    707 	u_int target;
    708 
    709 	target = desiredvnodes - desiredvnodes / 16;
    710 
    711 	mutex_enter(&vdrain_lock);
    712 
    713 	while (!vdrain_one(target))
    714 		kpause("vdrain", false, 1, &vdrain_lock);
    715 
    716 	threadpool_job_done(job);
    717 	mutex_exit(&vdrain_lock);
    718 }

https://nxr.netbsd.org/xref/src/sys/kern/vfs_vnode.c?r=1.158#701

	But there's a snag: in this loop, the vdrain task never
	reconsiders its target, even if desiredvnodes changes
	concurrently.

	If you set sysctl -w kern.maxvnodes=N, the sysctl backend will
	set desiredvnodes to N and then try to bring it down with
	vfs_drainvnodes, which fails and puts desiredvnodes back if we
	can't meet the target;

    694 static int
    695 sysctl_kern_maxvnodes(SYSCTLFN_ARGS)
    696 {
...
    720 	old_vnodes = desiredvnodes;
    721 	desiredvnodes = new_vnodes;
    722 	error = vfs_drainvnodes();
    723 	if (error) {
    724 		desiredvnodes = old_vnodes;
    725 		return (error);
    726 	}

https://nxr.netbsd.org/xref/src/sys/kern/init_sysctl.c?r=1.230#689

   2112 int
   2113 vfs_drainvnodes(void)
   2114 {
   2115 
   2116 	mutex_enter(&vdrain_lock);
   2117 
   2118 	if (!vdrain_one(desiredvnodes)) {
   2119 		mutex_exit(&vdrain_lock);
   2120 		return SET_ERROR(EBUSY);
   2121 	}
   2122 
   2123 	mutex_exit(&vdrain_lock);
...
   2128 	return 0;
   2129 }

https://nxr.netbsd.org/xref/src/sys/kern/vfs_vnode.c?r=1.158#2112

	But if the vdrain task started with the unattainably low
	desiredvnodes, it will remain stuck indefinitely like Sisyphus,
	perpetually rolling a vnode limit up the hill until it loses
	grip and the limit rolls right back down the hill.

	Also, all access to desiredvnodes is uncoordinated and racy,
	and should be serialized by a mutex or something.

>How-To-Repeat:

	sysctl -w kern.maxvnodes=<small> on a system with a lot of
	files open

>Fix:

	1. Recompute target inside the loop like we did before the
	   thread -> task conversion in vfs_vnode.c rev. 1.152.

	2. Serialize access to desiredvnodes with a mutex or rwlock
	   or pserialize or something.

>Audit-Trail:
From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/60603 CVS commit: src/sys/kern
Date: Tue, 18 Aug 2026 15:57:34 +0000

 Module Name:	src
 Committed By:	riastradh
 Date:		Tue Aug 18 15:57:33 UTC 2026

 Modified Files:
 	src/sys/kern: vfs_vnode.c

 Log Message:
 vdrain: Recompute target after each iteration.

 This way if sysctl -w kern.maxvnodes=N fails, the vdrain task doesn't
 perpetually keep trying to attain the unattainable.

 Doesn't fix everything about racy access to desiredvnodes, but it
 should help with the major symptom.

 PR kern/60603: Sisyphean vdrain task


 To generate a diff of this commit:
 cvs rdiff -u -r1.158 -r1.159 src/sys/kern/vfs_vnode.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

Home	PR Database Search