NetBSD Problem Report #43394

From  Mon May 31 13:36:27 2010
Return-Path: <>
Received: from ( [])
	by (Postfix) with ESMTP id E477F63B8E3
	for <>; Mon, 31 May 2010 13:36:26 +0000 (UTC)
Message-Id: <>
Date: Mon, 31 May 2010 13:36:26 +0000 (UTC)
Subject: adding regression tests is too complicated
X-Send-Pr-Version: www-1.0

>Number:         43394
>Category:       bin
>Synopsis:       adding regression tests is too complicated
>Confidential:   no
>Severity:       non-critical
>Priority:       high
>Responsible:    bin-bug-people
>State:          open
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Mon May 31 13:40:01 +0000 2010
>Last-Modified:  Tue Jul 13 21:15:02 +0000 2010
>Originator:     Antti Kantee
Adding a new atf tests requires (off the top of my head):

1) copypasting the test skeleton
2) writing the actual test
3) creating/editing a Makefile
4) creating/editing an Atffile
5) adding the test to several places in set lists

Additionally, if the test creates a new subdirectory in the
test hierarchy:

6) editing another Makefile
7) editing another Atffile
8) editing the mtree base
9) adding more lines to the set lists

Tests should be easy and simple to add.  Almost 50% of the time
in adding a test is spent appeasing various gods of build
Attempt to make regression testing easy for yourself and others.
Most likely the easiest solution is to provide a script which
takes care of all the mechanical tasks.  It can take care of
everything apart from step 2.

From: Julio Merino <>
Subject: PR/43394 CVS commit: src/share/mk
Date: Tue, 13 Jul 2010 21:11:42 +0000

 Module Name:	src
 Committed By:	jmmv
 Date:		Tue Jul 13 21:11:41 UTC 2010

 Modified Files:

 Log Message:
 Implement Atffile automatic generation.

 This change adds the ATFFILE variable, which can take the following values:
 - yes: The source directory contains a static Atffile to be installed.
 - auto: The Atffile is automatically generated.
 - no: Do not install any Atffile (replaces the previous NOATFFILE variable).

 The ugly bit of this is that I have to add a TESTS_SUBDIRS variable that
 lists the subdirectories that contain test cases (so that they can be listed
 in the Atffiles, without including auxiliary directories without tests).
 This is a subset of SUBDIR.

 Addresses a minor part of PR bin/43394.

 To generate a diff of this commit:
 cvs rdiff -u -r1.10 -r1.11 src/share/mk/

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD:,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.