NetBSD Problem Report #54350

From gson@gson.org  Sat Jul  6 11:28:31 2019
Return-Path: <gson@gson.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 6F6797A13E
	for <gnats-bugs@gnats.NetBSD.org>; Sat,  6 Jul 2019 11:28:31 +0000 (UTC)
Message-Id: <20190706112828.829C1989E15@guava.gson.org>
Date: Sat,  6 Jul 2019 14:28:28 +0300 (EEST)
From: gson@gson.org (Andreas Gustafsson)
Reply-To: gson@gson.org (Andreas Gustafsson)
To: gnats-bugs@NetBSD.org
Subject: lib/libcurses/t_curses/mvscanw test fails
X-Send-Pr-Version: 3.95

>Number:         54350
>Category:       lib
>Synopsis:       lib/libcurses/t_curses/mvscanw test fails
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    blymn
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Jul 06 11:30:00 +0000 2019
>Closed-Date:    Thu Apr 28 07:04:40 +0000 2022
>Last-Modified:  Thu Apr 28 07:04:40 +0000 2022
>Originator:     Andreas Gustafsson
>Release:        NetBSD-current
>Organization:

>Environment:
System: NetBSD
Architecture: i386
Machine: i386
>Description:

The mvscanw test case of the lib/libcurses/t_curses test is failing
on multiple testbeds:

  http://releng.netbsd.org/b5reports/i386/2019/2019.07.05.06.19.49/test.html#lib_libcurses_t_curses_mvscanw
  http://www.gson.org/netbsd/bugs/build/amd64-baremetal/2019/2019.07.05.17.14.48/test.html#lib_libcurses_t_curses_mvscanw

Looks like the test was re-enabled by this commit:

  2019.06.30.22.21.33 blymn src/tests/lib/libcurses/t_curses.sh 1.16 

with the commit message "Enable mvscanw test since libcurses is now correct."

>How-To-Repeat:

Run the ATF tests.

>Fix:

>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: lib-bug-people->blymn
Responsible-Changed-By: gson@NetBSD.org
Responsible-Changed-When: Sat, 06 Jul 2019 11:35:04 +0000
Responsible-Changed-Why:
Over to committer.


From: Brett Lymn <blymn@internode.on.net>
To: gnats-bugs@netbsd.org
Cc: lib-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: lib/54350: lib/libcurses/t_curses/mvscanw test fails
Date: Mon, 8 Jul 2019 07:37:53 +0930

 This test works for me on my machine.  The test is dependent on at least
 version 8.2 of libcurses, the major was bumped just before the test was
 commited and is required due to the scanw functions return changing.

 Please provide a ldd output from the slave.

 -- 
 Brett Lymn
 --
 Sent from my NetBSD device.

 "We are were wolves",
 "You mean werewolves?",
 "No we were wolves, now we are something else entirely",
 "Oh"

From: Martin Husemann <martin@duskware.de>
To: Brett Lymn <blymn@internode.on.net>
Cc: gnats-bugs@netbsd.org
Subject: Re: lib/54350: lib/libcurses/t_curses/mvscanw test fails
Date: Mon, 8 Jul 2019 07:11:41 +0200

 It fails on my machines too:

 tc-start: 1562562544.267360, mvscanw
 tc-se:director: Validate expected >2< matching >< line 8 of file /usr/tests/lib/libcurses/tests/mvscanw
 tc-end: 1562562545.17485, mvscanw, failed, test /usr/tests/lib/libcurses/tests/mvscanw failed

  > ldd ./slave 
 ./slave:
         -lcurses.8 => /usr/lib/libcurses.so.8
         -lterminfo.1 => /usr/lib/libterminfo.so.1
         -lc.12 => /usr/lib/libc.so.12
         -latf-c.0 => /usr/lib/libatf-c.so.0

  > ll /usr/lib/libcurses.so.8*
 lrwxr-xr-x  1 root  wheel      16 Jul  6 10:27 /usr/lib/libcurses.so.8@ -> libcurses.so.8.2
 -r--r--r--  1 root  wheel  226312 Jul  6 10:26 /usr/lib/libcurses.so.8.2

 Martin

From: Brett Lymn <blymn@internode.on.net>
To: gnats-bugs@netbsd.org
Cc: blymn@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org,
        Andreas Gustafsson <gson@gson.org>
Subject: Re: lib/54350: lib/libcurses/t_curses/mvscanw test fails
Date: Tue, 9 Jul 2019 07:53:14 +0930

 but it works on mine:

 tc-start: 1562624409.35934, mvscanw
 tc-end: 1562624410.116130, mvscanw, passed

 [blymn@rover] ldd ./slave
 ./slave:
         -lcurses.8 => /usr/lib/libcurses.so.8
         -lterminfo.1 => /usr/lib/libterminfo.so.1
         -lc.12 => /usr/lib/libc.so.12
         -latf-c.0 => /usr/lib/libatf-c.so.0

 the rest of my kernel/userland is older though.  I am suspecting some
 sort of timing/tty issue.  I will try updating my kernel and see what
 happens.

 -- 
 Brett Lymn
 --
 Sent from my NetBSD device.

 "We are were wolves",
 "You mean werewolves?",
 "No we were wolves, now we are something else entirely",
 "Oh"

From: Andreas Gustafsson <gson@gson.org>
To: blymn@netbsd.org
Cc: gnats-bugs@netbsd.org
Subject: Re: lib/54350: lib/libcurses/t_curses/mvscanw test fails
Date: Mon, 29 Jul 2019 21:29:36 +0300

 Brett Lymn wrote:
 >  but it works on mine:
 >  
 >  tc-start: 1562624409.35934, mvscanw
 >  tc-end: 1562624410.116130, mvscanw, passed

 Perhaps you didn't have JEMALLOC_DEBUG enabled?  The test
 started passing on the TNF testbed after JEMALLOC_DEBUG was
 turned off again recently:

   http://releng.netbsd.org/b5reports/i386/commits-2019.07.html#2019.07.23.06.31.20

 -- 
 Andreas Gustafsson, gson@gson.org

From: Brett Lymn <blymn@internode.on.net>
To: gnats-bugs@netbsd.org
Cc: blymn@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org,
        Andreas Gustafsson <gson@gson.org>
Subject: Re: lib/54350: lib/libcurses/t_curses/mvscanw test fails
Date: Tue, 30 Jul 2019 07:58:46 +0930

 On Mon, Jul 29, 2019 at 06:30:01PM +0000, Andreas Gustafsson wrote:
 >  
 >  Perhaps you didn't have JEMALLOC_DEBUG enabled?  The test
 >  started passing on the TNF testbed after JEMALLOC_DEBUG was
 >  turned off again recently:
 >  
 >    http://releng.netbsd.org/b5reports/i386/commits-2019.07.html#2019.07.23.06.31.20
 >  

 No I didn't have JEMALLOC_DEBUG enabled so that may be the reason why.
 Good pickup.  The question now is does jemalloc break scanw generically
 or is it a curses test problem?

 -- 
 Brett Lymn
 --
 Sent from my NetBSD device.

 "We are were wolves",
 "You mean werewolves?",
 "No we were wolves, now we are something else entirely",
 "Oh"

From: coypu@sdf.org
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: lib/54350: lib/libcurses/t_curses/mvscanw test fails
Date: Tue, 30 Jul 2019 06:38:22 +0000

 > The question now is does jemalloc break scanw generically
 > or is it a curses test problem?

 Most of the problems detected by jemalloc were double frees.
 It was idenitifiable by it segfaulting while dereferencing something,
 and the object having the address 0x5a5a5a...

 Another option that is very good at detecting these bugs is compiling
 the test programs with -fsanitize=address and turning off ASLR.

From: Martin Husemann <martin@duskware.de>
To: Brett Lymn <blymn@internode.on.net>
Cc: gnats-bugs@netbsd.org, blymn@netbsd.org
Subject: Re: lib/54350: lib/libcurses/t_curses/mvscanw test fails
Date: Tue, 30 Jul 2019 09:51:00 +0200

 On Tue, Jul 30, 2019 at 07:58:46AM +0930, Brett Lymn wrote:
 > No I didn't have JEMALLOC_DEBUG enabled so that may be the reason why.
 > Good pickup.  The question now is does jemalloc break scanw generically
 > or is it a curses test problem?

 My bet is on a test or curses problem.

 You could also try to build both libcurses and the tests with
 MKSANITIZER=yes (which currently requires you to turn of 
 security.pax.aslr.global).

 Martin

From: Brett Lymn <blymn@internode.on.net>
To: gnats-bugs@netbsd.org
Cc: blymn@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org,
        Andreas Gustafsson <gson@gson.org>
Subject: Re: lib/54350: lib/libcurses/t_curses/mvscanw test fails
Date: Fri, 4 Oct 2019 17:48:06 +0930

 Can you please re-test.  During the process of adding wide character
 support to the curse test frame I found errors in how variables were
 being handled when creating the arguments for a call.

 the mvscanw tests runs to completion with MKSANITIZE=yes now.

 -- 
 Brett Lymn
 --
 Sent from my NetBSD device.

 "We are were wolves",
 "You mean werewolves?",
 "No we were wolves, now we are something else entirely",
 "Oh"

From: Andreas Gustafsson <gson@gson.org>
To: Brett Lymn <blymn@internode.on.net>
Cc: gnats-bugs@netbsd.org
Subject: Re: lib/54350: lib/libcurses/t_curses/mvscanw test fails
Date: Thu, 6 Feb 2020 19:19:09 +0200

 On Oct 4, 2019 Brett Lymn wrote:
 >  Can you please re-test.  During the process of adding wide character
 >  support to the curse test frame I found errors in how variables were
 >  being handled when creating the arguments for a call.
 >  
 >  the mvscanw tests runs to completion with MKSANITIZE=yes now.

 I re-tested on amd64 using today's current, re-enabling JEMALLOC_DEBUG
 using this patch:

 Index: src/external/bsd/jemalloc/lib/Makefile.inc
 ===================================================================
 RCS file: /bracket/repo/src/external/bsd/jemalloc/lib/Makefile.inc,v
 retrieving revision 1.10
 retrieving revision 1.9
 diff -u -r1.10 -r1.9
 --- src/external/bsd/jemalloc/lib/Makefile.inc	23 Jul 2019 06:31:20 -0000	1.10
 +++ src/external/bsd/jemalloc/lib/Makefile.inc	15 Apr 2019 20:40:53 -0000	1.9
 @@ -38,10 +38,7 @@
  .SUFFIXES: .3
  .PATH.3: ${JEMALLOC}/dist/doc
  .for i in ${JEMALLOC_SRCS}
 -# helps in tracking bad malloc/pointer usage, but has a serious
 -# performance penalty:
 -#   CPPFLAGS.${i}+=-I${JEMALLOC}/include -DJEMALLOC_PROTECT_NOSTD -DJEMALLOC_DEBUG
 -CPPFLAGS.${i}+=-I${JEMALLOC}/include -DJEMALLOC_PROTECT_NOSTD
 +CPPFLAGS.${i}+=-I${JEMALLOC}/include -DJEMALLOC_PROTECT_NOSTD -DJEMALLOC_DEBUG
  COPTS.${i}+= -fvisibility=hidden -funroll-loops
  COPTS.${i}+= ${${ACTIVE_CC} == "clang":? -Wno-atomic-alignment :}
  .endfor

 The test still failed.  Log output is at:

   http://www.gson.org/netbsd/bugs/build/amd64-debug-baremetal/2020/2020.02.06.10.47.33/test.html#lib_libcurses_t_curses_mvscanw

 Next time please set the PR to the feedback state, it may help get a
 faster response.
 -- 
 Andreas Gustafsson, gson@gson.org

From: Valery Ushakov <uwe@stderr.spb.ru>
To: gnats-bugs@netbsd.org
Cc: blymn@netbsd.org
Subject: Re: lib/54350: lib/libcurses/t_curses/mvscanw test fails
Date: Sat, 8 Feb 2020 01:18:53 +0300

 Not sure about amd64, but the mvscanw test command is wrong as it
 always prints the string[] buffer with %s.

 Consider

   input "50 12\n"
   call2 OK "2" mvscanw 3 5 "%d"

 Now, on a big-endian host, say macppc.  mvscanw "%d" will read "50"
 and store "\x00\x00\x00\x32" at the beginning of string[].  When
 string[] is printed with %s, nothing is printed, b/c the first byte is
 NUL.

 -uwe

State-Changed-From-To: open->feedback
State-Changed-By: blymn@NetBSD.org
State-Changed-When: Mon, 25 Apr 2022 22:15:47 +0000
State-Changed-Why:
I think that rin@ fixed this, test is not failing AFAIK.  OK to close?


From: Andreas Gustafsson <gson@gson.org>
To: blymn@NetBSD.org
Cc: gnats-bugs@netbsd.org
Subject: Re: lib/54350 (lib/libcurses/t_curses/mvscanw test fails)
Date: Thu, 28 Apr 2022 09:28:57 +0300

 blymn@NetBSD.org wrote:
 > Synopsis: lib/libcurses/t_curses/mvscanw test fails
 > 
 > State-Changed-From-To: open->feedback
 > State-Changed-By: blymn@NetBSD.org
 > State-Changed-When: Mon, 25 Apr 2022 22:15:47 +0000
 > State-Changed-Why:
 > I think that rin@ fixed this, test is not failing AFAIK.  OK to close?

 I have now confirmed that the test now passes even when JEMALLOC_DEBUG
 is re-enabled using the patch from an earlier comment.  It started
 passing with this commit:

   2021.04.04.09.42.08 rin src/tests/lib/libcurses/slave/curses_commands.c 1.24
   2021.04.04.09.42.08 rin src/tests/lib/libcurses/tests/mvscanw 1.6

 I have also confirmed that it passes on a big-endian host (sparc)
 without the patch.
 -- 
 Andreas Gustafsson, gson@gson.org

State-Changed-From-To: feedback->closed
State-Changed-By: gson@NetBSD.org
State-Changed-When: Thu, 28 Apr 2022 07:04:40 +0000
State-Changed-Why:
The test no longer fails.


>Unformatted:

Home	PR Database Search