NetBSD Problem Report #55549

From gson@gson.org  Fri Aug  7 14:50:41 2020
Return-Path: <gson@gson.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 6FAF41A923A
	for <gnats-bugs@gnats.NetBSD.org>; Fri,  7 Aug 2020 14:50:41 +0000 (UTC)
Message-Id: <20200807145035.9D3BF253EDE@guava.gson.org>
Date: Fri,  7 Aug 2020 17:50:35 +0300 (EEST)
From: gson@gson.org (Andreas Gustafsson)
Reply-To: gson@gson.org (Andreas Gustafsson)
To: gnats-bugs@NetBSD.org
Subject: evbarm-aarch64 tests randomly fail with "out of swap"
X-Send-Pr-Version: 3.95

>Number:         55549
>Category:       port-evbarm
>Synopsis:       evbarm-aarch64 tests randomly fail with "out of swap"
>Confidential:   no
>Severity:       serious
>Priority:       high
>Responsible:    port-evbarm-maintainer
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Aug 07 14:55:01 +0000 2020
>Closed-Date:    Sat Jun 05 10:05:20 +0000 2021
>Last-Modified:  Mon Jun 07 07:50:02 +0000 2021
>Originator:     Andreas Gustafsson
>Release:        NetBSD-current
>Organization:
>Environment:
System: NetBSD
Architecture: aarch64
Machine: evbarm
>Description:

Multiple test cases have started randomly failing with "out of swap"
on the TNF evbarm-aarch64 testbed.  For example, in

  http://releng.netbsd.org/b5reports/evbarm-aarch64/2020/2020.08.06.07.38.54/test.log

all the test cases of the usr.bin/c++/t_call_once test failed (along with
a few others):

  usr.bin/c++/t_call_once (665/878): 10 test cases
      call_once: [ 66472.7833543] UVM: pid 26411 (cc1plus), uid 0 killed: out of swap
  [30.002953s] Failed: atf-check failed; see the output of the test for details
      call_once_32: [3.344120s] Skipped: c++ -m32 not supported on this architecture
      call_once_pic: [ 66495.2189112] UVM: pid 15162 (cc1plus), uid 0 killed: out of swap
  [19.007283s] Failed: atf-check failed; see the output of the test for details
      call_once_pic_32: [2.347975s] Skipped: c++ -m32 not supported on this architecture
      call_once_pic_profile: [ 66514.2407960] UVM: pid 21953 (cc1plus), uid 0 killed: out of swap
  [16.042760s] Failed: atf-check failed; see the output of the test for details
      call_once_pic_profile_32: [2.808502s] Skipped: c++ -m32 not supported on this architecture
      call_once_pie: [ 66533.8851851] UVM: pid 29060 (cc1plus), uid 0 killed: out of swap
  [16.766816s] Failed: atf-check failed; see the output of the test for details
      call_once_profile: [ 66549.1048997] UVM: pid 29354 (cc1plus), uid 0 killed: out of swap
  [15.206544s] Failed: atf-check failed; see the output of the test for details
      call_once_profile_32: [3.606928s] Skipped: c++ -m32 not supported on this architecture
      call_once_static: [ 66569.3160469] UVM: pid 21310 (cc1plus), uid 0 killed: out of swap
  [16.520687s] Failed: atf-check failed; see the output of the test for details
  [126.076767s]

This is running under qemu with 512 MB of emulated RAM and no swap
(because the images in the release aren't configured with any).

The failures appear to be random, but in such a way that in any given
test run, either none of the tests fail with "out of swap", or many of
them do.

The first recorded failure like this was with sources from
2020.06.27.04.31.06:

  http://releng.netbsd.org/b5reports/evbarm-aarch64/commits-2020.06.html#2020.06.27.04.31.06

but the problem may have started as much as a few days before that;
full bisection is difficult due to the random nature of the failures
and build breakage around the time when the problem started.

>How-To-Repeat:

>Fix:

>Release-Note:

>Audit-Trail:
From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: port-evbarm/55549: evbarm-aarch64 tests randomly fail with "out of swap"
Date: Sat, 5 Jun 2021 12:55:18 +0300

 Most of the "out of swap" errors appear to be gone since some time
 around Dec 17-20, 2020.  Instead of multiple tests triggering it in
 each run, it is now only being triggered occasionally by
 lib/libc/regex/t_exhaust:regcomp_too_big.  The log from one such run
 is here:

   http://releng.netbsd.org/b5reports/evbarm-aarch64/2021/2021.06.02.00.00.39/test.log

 That looks like a different bug, so I'm closing the present PR.
 -- 
 Andreas Gustafsson, gson@gson.org

State-Changed-From-To: open->closed
State-Changed-By: gson@NetBSD.org
State-Changed-When: Sat, 05 Jun 2021 10:05:20 +0000
State-Changed-Why:
The issue appears to have been fixed without anyone taking credit.


From: Andreas Gustafsson <gson@gson.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: port-evbarm/55549: evbarm-aarch64 tests randomly fail with "out of swap"
Date: Mon, 7 Jun 2021 10:45:23 +0300

 The other day, I wrote:
 >  That looks like a different bug, so I'm closing the present PR.

 The other bug now has its own PR: port-arm/56239.
 -- 
 Andreas Gustafsson, gson@gson.org

>Unformatted:

Home	PR Database Search