NetBSD Problem Report #43394
From www@NetBSD.org Mon May 31 13:36:27 2010
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
by www.NetBSD.org (Postfix) with ESMTP id E477F63B8E3
for <gnats-bugs@gnats.NetBSD.org>; Mon, 31 May 2010 13:36:26 +0000 (UTC)
Message-Id: <20100531133626.A600963B8CF@www.NetBSD.org>
Date: Mon, 31 May 2010 13:36:26 +0000 (UTC)
From: pooka@iki.fi
Reply-To: pooka@iki.fi
To: gnats-bugs@NetBSD.org
Subject: adding regression tests is too complicated
X-Send-Pr-Version: www-1.0
>Number: 43394
>Category: bin
>Synopsis: adding regression tests is too complicated
>Confidential: no
>Severity: non-critical
>Priority: high
>Responsible: bin-bug-people
>State: open
>Class: change-request
>Submitter-Id: net
>Arrival-Date: Mon May 31 13:40:01 +0000 2010
>Last-Modified: Tue Jul 13 21:15:02 +0000 2010
>Originator: Antti Kantee
>Release:
>Organization:
Test'ng
>Environment:
>Description:
Adding a new atf tests requires (off the top of my head):
1) copypasting the test skeleton
2) writing the actual test
3) creating/editing a Makefile
4) creating/editing an Atffile
5) adding the test to several places in set lists
Additionally, if the test creates a new subdirectory in the
test hierarchy:
6) editing another Makefile
7) editing another Atffile
8) editing the mtree base
9) adding more lines to the set lists
Tests should be easy and simple to add. Almost 50% of the time
in adding a test is spent appeasing various gods of build
mechanics.
>How-To-Repeat:
Attempt to make regression testing easy for yourself and others.
>Fix:
Most likely the easiest solution is to provide a script which
takes care of all the mechanical tasks. It can take care of
everything apart from step 2.
>Audit-Trail:
From: Julio Merino <jmmv@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/43394 CVS commit: src/share/mk
Date: Tue, 13 Jul 2010 21:11:42 +0000
Module Name: src
Committed By: jmmv
Date: Tue Jul 13 21:11:41 UTC 2010
Modified Files:
src/share/mk: bsd.test.mk
Log Message:
Implement Atffile automatic generation.
This change adds the ATFFILE variable, which can take the following values:
- yes: The source directory contains a static Atffile to be installed.
- auto: The Atffile is automatically generated.
- no: Do not install any Atffile (replaces the previous NOATFFILE variable).
The ugly bit of this is that I have to add a TESTS_SUBDIRS variable that
lists the subdirectories that contain test cases (so that they can be listed
in the Atffiles, without including auxiliary directories without tests).
This is a subset of SUBDIR.
Addresses a minor part of PR bin/43394.
To generate a diff of this commit:
cvs rdiff -u -r1.10 -r1.11 src/share/mk/bsd.test.mk
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.