NetBSD Problem Report #60513

From storchaka@gmail.com  Tue Jul 28 14:55:48 2026
Return-Path: <storchaka@gmail.com>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mail.netbsd.org", Issuer "YR2" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id CCAB71A923B
	for <gnats-bugs@gnats.NetBSD.org>; Tue, 28 Jul 2026 14:55:48 +0000 (UTC)
Message-Id: <1bc0affc-3c4f-4291-8fb6-2003f4759e33@gmail.com>
Date: Tue, 28 Jul 2026 17:55:42 +0300
From: Serhiy Storchaka <storchaka@gmail.com>
To: gnats-bugs@netbsd.org
Subject: lib/curses: wenclose() accepts coordinates one row/column outside the
 window
X-From4GNATS: "Serhiy Storchaka via gnats" <gnats-admin@NetBSD.org>

>Number:         60513
>Category:       lib
>Synopsis:       curses: wenclose() is off by one, so wmouse_trafo() transforms coordinates outside the window
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    lib-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Jul 28 15:00:01 +0000 2026
>Originator:     Serhiy Storchaka
>Release:        NetBSD 10.1
>Environment:
System: NetBSD 10.1 amd64 (GENERIC), libcurses.so.9.1
>Description:

wenclose() returns true for the row just below a window and for the
column just to its right, so a coordinate outside the window is
reported as enclosed by it. wmouse_trafo() tests the coordinates with
wenclose(), so it transforms and accepts those coordinates too instead
of returning false.

curses_mouse.3 says wenclose() "returns true if the screen relative x
and y co-ordinates are enclosed by the window win, otherwise false",
and the row below a window is not enclosed by it.  ncurses, whose
extension this is, documents the same and returns false there.

The cause is a field with a different meaning in the two
implementations.  The test in libcurses/mouse.c is the one ncurses
uses in ncurses/base/lib_mouse.c:

	/* NetBSD, libcurses/mouse.c */
	if (y < win->begy || y > win->begy + win->maxy ||
	    x < win->begx || x > win->begx + win->maxx)
		return false;

	/* ncurses, ncurses/base/lib_mouse.c */
	result = ((win->_begy <= y && win->_begx <= x &&
		   (win->_begx + win->_maxx) >= x &&
		   (win->_begy + win->_maxy) >= y) ? TRUE : FALSE);

but ncurses' _maxy is the largest valid coordinate ("maximums of x and
y, NOT window size" in its curses.h), while NetBSD's maxy is the
number of lines (newwin.c: "win->maxy = nlines").  So begy + maxy is
the last row in ncurses and one past the last row here.  The rest of
libcurses uses maxy as a count, for example chgat.c "if (x >=
win->maxx || y >= win->maxy)" and move.c "if (x > win->maxx || y >=
win->maxy)"; mouse.c is the only place that reads it the ncurses way.

The code is the same in -current (mouse.c has not changed since the
NULL-pointer checks of 2024-12-23).

>How-To-Repeat:

	/* cc wenclose_off.c -lcurses && TERM=xterm ./a.out */
	#include <curses.h>
	#include <stdio.h>
	int main(void) {
		FILE *f = fopen("/dev/null", "w+");
		SCREEN *sp = newterm("xterm", f, f);
		/* 5 rows x 15 columns at (2,5): rows 2..6, columns 5..19 */
		WINDOW *w = newwin(5, 15, 2, 5);
		printf("wenclose(win, 6, 19) = %d (expected 1)\n", wenclose(w, 6, 19));
		printf("wenclose(win, 7, 19) = %d (expected 0)\n", wenclose(w, 7, 19));
		printf("wenclose(win, 6, 20) = %d (expected 0)\n", wenclose(w, 6, 20));
		endwin(); delscreen(sp);
		return 0;
	}

prints

	wenclose(win, 6, 19) = 1 (expected 1)
	wenclose(win, 7, 19) = 1 (expected 0)
	wenclose(win, 6, 20) = 1 (expected 0)

The same program against ncurses 6.6 prints 1, 0, 0.

Found through Python's curses module, whose test suite checks these
boundaries (test_curses: test_enclose and test_mouse_trafo).

>Fix:

--- a/libcurses/mouse.c
+++ b/libcurses/mouse.c
@@
-	if (y < win->begy || y > win->begy + win->maxy ||
-	    x < win->begx || x > win->begx + win->maxx)
+	if (y < win->begy || y >= win->begy + win->maxy ||
+	    x < win->begx || x >= win->begx + win->maxx)
  		return false;
  	return true;

so that the comparison uses maxy/maxx as the counts they are here.
This should make the reproducer print 1, 0, 0 and wmouse_trafo()
reject the same coordinates; I have not rebuilt libcurses to confirm
it.

>Unformatted:

Home	PR Database Search