NetBSD Problem Report #55650

From www@netbsd.org  Thu Sep 10 05:20:09 2020
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id EADC81A9239
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 10 Sep 2020 05:20:08 +0000 (UTC)
Message-Id: <20200910052007.BBEDF1A923A@mollari.NetBSD.org>
Date: Thu, 10 Sep 2020 05:20:07 +0000 (UTC)
From: roland.illig@gmx.de
Reply-To: roland.illig@gmx.de
To: gnats-bugs@NetBSD.org
Subject: atf-run crashes on parse error
X-Send-Pr-Version: www-1.0

>Number:         55650
>Category:       toolchain
>Synopsis:       atf-run crashes on parse error
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    toolchain-manager
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Sep 10 05:25:00 +0000 2020
>Closed-Date:    Mon Oct 05 17:12:12 +0000 2020
>Last-Modified:  Mon Oct 05 17:12:12 +0000 2020
>Originator:     Roland Illig
>Release:        8.0
>Organization:
>Environment:
NetBSD 8.0 amd64
>Description:
$ cd /usr/tests/usr.bin/make
$ atf-run t_make
...
terminate called after throwing an instance of 'tools::parser::parse_errors'
  what():  1: Unexpected token `-s.'; expected eof, `;' or new line
Abort trap (core dumped)

1. This should not be a crash but regular error handling.
2. The error message is missing the filename.
3. The missing filename makes it hard to identify the "1:" as being the line number.
4. The first line of "t_make" is "#! /usr/bin/atf-sh", it does not contain the token "-s." mentioned in the parse error message.
5. Changing the line to "#! /usr/bin/atf-th" yields a completely different error message:

Failed to execute 't_make': No such file or directory
tp-start: 1599715042.128949, t_make, 0
tp-end: 1599715042.128999, t_make, Invalid format for test case list: 1: Unexpected token `<<EOF>>'; expected a header name

6. The manual page of atf-run says the argument is a "test_program", and as far as I understood, "t_make" is the name of a test program.
>How-To-Repeat:

>Fix:

>Release-Note:

>Audit-Trail:
From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: toolchain/55650: atf-run crashes on parse error
Date: Thu, 10 Sep 2020 07:54:28 +0200

 I can not reproduce that problem with -current as of yesterday:

  > atf-run t_make
 Content-Type: application/X-atf-tps; version="3"

 info: atf.version, Automated Testing Framework 0.20 (atf-0.20)
 info: tests.root, /usr/tests/usr.bin/make
 [..]

 The file is just a shell script, if you broke its syntax, you get shell
 errors. If you modify the interpreter shebang, bad things happen.

 If you want to test it manually, invoke it with -l:

  > ./t_make -l
 Content-Type: application/X-atf-tp; version="1"

 ident: archive_suffix

 ident: archive

 ident: cmd_interrupt


 .... and note the *significant* delay it takes before outputing the first
 line, this is what the other PR is about.

 Martin

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: toolchain/55650: atf-run crashes on parse error
Date: Thu, 10 Sep 2020 17:34:49 +0700

     Date:        Thu, 10 Sep 2020 05:25:00 +0000 (UTC)
     From:        roland.illig@gmx.de
     Message-ID:  <20200910052500.4D5B71A923B@mollari.NetBSD.org>


   | terminate called after throwing an instance of 'tools::parser::parse_errors'
   |   what():  1: Unexpected token `-s.'; expected eof, `;' or new line
   | Abort trap (core dumped)

 The most trivial of syntax errors in an ATF test file can cause
 symptoms like that - the error might be almost anywhere in the file.

 Look at whatever changed recently, and make sure there are no syntax
 errors.

 The problem is almost certainly nothing related to the invocation, and
 probably not t_make itself (if it is, I'll find it in a few minutes when
 I test my update to it - in that case I will fix it).   It is more likely
 related to one of the test programs it runs however.

 Line number 1 might be line number 1 in almost anything (the first line of
 any function just about).

 And yes, I agree, ATF's reporting of things like this is poor, but there
 is really only so much a sh script can do.

 kre

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: toolchain/55650: atf-run crashes on parse error
Date: Thu, 10 Sep 2020 21:37:46 +0700

 OK, I am seeing that failure, so I will work out the cause and
 fix it.   But it looks related to ...

 Test_Cases=' comment cond_late} cond1 cond2 dollar doterror dotwait error escape export_all} export_env} export forloop forsubst hash impsrc include_main} misc moderrs'

 My guess is that those '}' chars are not intended to be there.

 Not sure yet why they are.

 kre

From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: toolchain/55650: atf-run crashes on parse error
Date: Thu, 10 Sep 2020 21:44:26 +0700

 Ignore my last message, the '}' was caused by a stupid cu&pasto of mine...

 kre

State-Changed-From-To: open->feedback
State-Changed-By: kre@NetBSD.org
State-Changed-When: Thu, 10 Sep 2020 17:39:31 +0000
State-Changed-Why:
Can this PR be closed, there doesn't seem to be a problem (aside from
ATF's fragility)?


From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: toolchain/55650: atf-run crashes on parse error
Date: Fri, 11 Sep 2020 00:38:15 +0700

 I see no problem now either (that is after I fixed my own lunacy).

 Perhaps you were attempting to debug changes to t_make.sh and
 were doing so using echo or printf in the code?

 ATF test programs (as distinct from the tests that they run)
 cannot use stdout for anything - the ATF system uses a pipe from
 the shell running the test (t_make.sh) back to itself, and expects
 only properly formatted messages (from the internal ATF functions)
 to appear there.

 If you need diagnostic output, make sure it all goes to stderr 
 instead of stdout.

 I believe that this PR can be closed?

 kre

State-Changed-From-To: feedback->closed
State-Changed-By: rillig@NetBSD.org
State-Changed-When: Mon, 05 Oct 2020 17:12:12 +0000
State-Changed-Why:
Seems to be fixed in NetBSD 9.


>Unformatted:

Home	PR Database Search