NetBSD Problem Report #54158
From www@netbsd.org Sat May 4 14:02:14 2019
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id D9D427A153
for <gnats-bugs@gnats.NetBSD.org>; Sat, 4 May 2019 14:02:14 +0000 (UTC)
Message-Id: <20190504140213.8DF947A1BB@mollari.NetBSD.org>
Date: Sat, 4 May 2019 14:02:13 +0000 (UTC)
From: tho@useless-ficus.net
Reply-To: tho@useless-ficus.net
To: gnats-bugs@NetBSD.org
Subject: poll(2) does not allow polling all possible fds (hardcoded limit to ~1000+ fds)
X-Send-Pr-Version: www-1.0
>Number: 54158
>Category: kern
>Synopsis: poll(2) does not allow polling all possible fds (hardcoded limit to ~1000+ fds)
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: kern-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Sat May 04 14:05:00 +0000 2019
>Last-Modified: Sun Mar 08 09:50:01 +0000 2020
>Originator: Anthony Mallet
>Release: -current
>Organization:
>Environment:
NetBSD ficus 8.99.37 NetBSD 8.99.37 (FICUS) #18: Sat Apr 27 15:54:05 WEST 2019
>Description:
Trying to poll a bit more than 1000 file descriptors returns EINVAL.
I think this basically comes from the test in pollcommon(), in
kern/sys_select.c, that does this:
if (nfds > 1000 + curlwp->l_fd->fd_dt->dt_nfiles)
return EINVAL;
I have a modified login.conf that allows for 1024 descriptors, but poll(&fds, 1024, 0) always fails.
I came accross this issue while tracking down bugs in a real software,
so this is not a theoretical bug report :) Attached is a sample test
program that exhibit the issue.
(the idea of the code is to count open fds)
Maybe the test in pollcommon() should be based on the current hard limit?
>How-To-Repeat:
# cat poll.c
#include <sys/poll.h>
#include <sys/resource.h>
#include <err.h>
#include <stdio.h>
#include <stdlib.h>
int
main()
{
struct rlimit limit;
struct pollfd *pollfds;
size_t i, free_fds;
if (getrlimit(RLIMIT_NOFILE, &limit)) err(2, "getrlimit()");
if (limit.rlim_cur == RLIM_INFINITY) errx(2, "too many");
printf("descriptors: %d\n", limit.rlim_cur);
pollfds = calloc(limit.rlim_cur, sizeof(*pollfds));
if (!pollfds) err(2, "calloc");
for (i = 0; i < limit.rlim_cur; i++) {
pollfds[i].fd = i;
pollfds[i].events = 0;
}
if (poll(pollfds, limit.rlim_cur, 0) == -1) err(2, "poll()");
free_fds = 0;
for (i = 0; i < limit.rlim_cur; i++) {
if (pollfds[i].revents & POLLNVAL) free_fds++;
}
printf("free fds: %zu\n", free_fds);
return 0;
}
# gcc -o poll poll.c
# ./poll
descriptors: 1024
poll: poll(): Invalid argument
>Fix:
>Audit-Trail:
From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/54158 CVS commit: src/sys/kern
Date: Sat, 4 May 2019 11:46:58 -0400
Module Name: src
Committed By: christos
Date: Sat May 4 15:46:58 UTC 2019
Modified Files:
src/sys/kern: sys_select.c
Log Message:
PR/54158: Anthony Mallet: poll(2) does not allow polling all possible fds
(hardcoded limit to 1000 + #<open-fds>). Changed to limit by the max of
the resource limit of open descriptors and the above.
To generate a diff of this commit:
cvs rdiff -u -r1.41 -r1.42 src/sys/kern/sys_select.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Anthony Mallet <tho@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: PR/54158 CVS commit: src/sys/kern
Date: Tue, 7 May 2019 21:36:13 +0200
On Saturday 4 May 2019, at 15:50, Christos Zoulas wrote:
> Modified Files:
> src/sys/kern: sys_select.c
>
> Log Message: PR/54158: Anthony Mallet: poll(2) does not allow
> polling all possible fds (hardcoded limit to 1000 +
> #<open-fds>). Changed to limit by the max of the resource limit of
> open descriptors and the above.
Thanks!
I'm fine with sys_select.c rev. 1.43.
Even though
http://mail-index.netbsd.org/source-changes-d/2019/05/06/msg011270.html
makes a lot a sense, I think there is no need to change the test
further, because POSIX (and Linux, for that matter) implicitly state
that poll(2) might return EINVAL if "The nfds argument is greater than
{OPEN_MAX}". (see http://pubs.opengroup.org/onlinepubs/9699919799/)
What about using "rlim_max" instead of "rlim_cur"? This would be more
inline with "{OPEN_MAX}", and this would partially handle the point
made by kre@ in his message above.
From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/54158 CVS commit: [netbsd-8] src/sys/kern
Date: Sun, 8 Mar 2020 09:47:28 +0000
Module Name: src
Committed By: martin
Date: Sun Mar 8 09:47:28 UTC 2020
Modified Files:
src/sys/kern [netbsd-8]: sys_select.c
Log Message:
Pull up following revision(s) (requested by mlelstv in ticket #1515):
sys/kern/sys_select.c: revision 1.42-1.45
PR/54158: Anthony Mallet: poll(2) does not allow polling all possible fds
(hardcoded limit to 1000 + #<open-fds>). Changed to limit by the max of
the resource limit of open descriptors and the above.
Remove the slop code. Suggested by mrg@
Use the max limit (aka maxfiles or the moral equivalent of OPEN_MAX) which
makes poll(2) align with the Posix documentation (which allows EINVAL if
nfds > OPEN_MAX). From: Anthony Mallet
Add slop of 1000 and explain why.
To generate a diff of this commit:
cvs rdiff -u -r1.40 -r1.40.2.1 src/sys/kern/sys_select.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
(Contact us)
$NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.