NetBSD Problem Report #58791

From www@netbsd.org  Wed Oct 30 10:46:39 2024
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id EA74A1A9238
	for <gnats-bugs@gnats.NetBSD.org>; Wed, 30 Oct 2024 10:46:38 +0000 (UTC)
Message-Id: <20241030104637.D2C2E1A923B@mollari.NetBSD.org>
Date: Wed, 30 Oct 2024 10:46:37 +0000 (UTC)
From: campbell+netbsd@mumble.net
Reply-To: campbell+netbsd@mumble.net
To: gnats-bugs@NetBSD.org
Subject: npftest stops early
X-Send-Pr-Version: www-1.0

>Number:         58791
>Category:       bin
>Synopsis:       npftest stops early
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    bin-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Oct 30 10:50:00 +0000 2024
>Originator:     Taylor R Campbell
>Release:        current, 10, 9, ...
>Organization:
>Environment:
>Description:
npftest stops early as soon as any test case fails, instead of printing all the failed test cases.  Makes for slow turnaround time when diagnosing bugs, and testing fixes, that affect multiple cases at once.

Also, t_npf doesn't record the individual failed test cases, so they are not included in the releng test bed output like: https://releng.netbsd.org/b5reports/i386/2024/2024.10.29.22.24.30/test.html#net_npf_t_npf_npf_rule
>How-To-Repeat:
1. run the atf tests
2. npfctl debug -c /usr/tests/net/npf/npftest.conf -o /tmp/npftest.db && npftest -c /tmp/npftest.db -T rule -v
>Fix:
Yes, please!

Home	PR Database Search