NetBSD Problem Report #36128

From dholland@eecs.harvard.edu  Fri Apr  6 22:48:23 2007
Return-Path: <dholland@eecs.harvard.edu>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by narn.NetBSD.org (Postfix) with ESMTP id C3F1963B400
	for <gnats-bugs@gnats.NetBSD.org>; Fri,  6 Apr 2007 22:48:23 +0000 (UTC)
Message-Id: <20070406224619.04209F8CF@tanaqui.eecs.harvard.edu>
Date: Fri,  6 Apr 2007 18:46:18 -0400 (EDT)
From: dholland@eecs.harvard.edu
Reply-To: dholland@eecs.harvard.edu
To: gnats-bugs@NetBSD.org
Subject: CRLF handling for make
X-Send-Pr-Version: 3.95

>Number:         36128
>Category:       bin
>Synopsis:       CRLF handling for make
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    bin-bug-people
>State:          open
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Fri Apr 06 22:50:00 +0000 2007
>Last-Modified:  Sat Apr 14 18:30:00 +0000 2007
>Originator:     David A. Holland <dholland@eecs.harvard.edu>
>Release:        NetBSD 4.99.9 (20070221)
>Organization:
>Environment:
System: NetBSD tanaqui 4.99.9 NetBSD 4.99.9 (TANAQUI) #8: Tue Jan 30 17:29:55 EST 2007 dholland@tanaqui:/usr/src/sys/arch/i386/compile/TANAQUI i386
Architecture: i386
Machine: i386
>Description:

It was mentioned on #pkgsrc irc that our make doesn't handle makefiles
that have embedded carriage returns (DOS-style CR/LF end of lines)...
...and there seems no particular reason it shouldn't, since at least
one such makefile seems to exist in the wild.

Patch follows.

I've tested this myself and it seems to work with no ill effects,
although I don't have the actual offending makefile and I haven't
really tested all the possible strange corner cases. But I believe it
to be sound.

make currently fails regression, but this doesn't appear to be because
of anything I've done.


>How-To-Repeat:
n/a

>Fix:

Patch against latest parse.c:

Index: parse.c
===================================================================
RCS file: /cvsroot/src/usr.bin/make/parse.c,v
retrieving revision 1.133
diff -u -r1.133 parse.c
--- parse.c	24 Feb 2007 17:55:54 -0000	1.133
+++ parse.c	6 Apr 2007 22:46:41 -0000
@@ -2179,9 +2179,15 @@
 		/* Don't treat next character as special, remember first one */
 		if (escaped == NULL)
 		    escaped = ptr;
-		if (ptr[1] == '\n')
+		if (ptr[1] == '\r' && ptr[2] == '\n') {
 		    cf->lineno++;
-		ptr += 2;
+		    ptr += 3;
+		} else if (ptr[1] == '\n') {
+		    cf->lineno++;
+		    ptr += 2;
+		} else {
+		    ptr += 2;
+		}
 		line_end = ptr;
 		continue;
 	    }
@@ -2189,6 +2195,10 @@
 		/* Remember first '#' for comment stripping */
 		comment = line_end;
 	    }
+	    if (ch == '\r' && ptr[1] == '\n') {
+	        ptr++;
+		ch = '\n';
+	    }
 	    ptr++;
 	    if (ch == '\n')
 		break;
@@ -2261,6 +2271,9 @@
 	    /* Delete '\\' from before '#' on non-command lines */
 	    continue;

+	if (ch == '\r' && ptr[0] == '\n')
+	    ch = *ptr++;
+
 	if (ch != '\n') {
 	    /* Leave '\\' in buffer for later */
 	    *tp++ = '\\';

>Audit-Trail:
From: Alan Barrett <apb@cequrux.com>
To: gnats-bugs@NetBSD.org
Cc: netbsd-bugs@NetBSD.org
Subject: Re: bin/36128: CRLF handling for make
Date: Sat, 14 Apr 2007 16:26:45 +0200

 On Fri, 06 Apr 2007, dholland@eecs.harvard.edu wrote:
 > It was mentioned on #pkgsrc irc that our make doesn't handle makefiles
 > that have embedded carriage returns (DOS-style CR/LF end of lines)...
 > ...and there seems no particular reason it shouldn't, since at least
 > one such makefile seems to exist in the wild.

 I am not at all convinced that this is a good idea.  I think that people
 who write their Makefiles under DOS should convert them to unix file
 format before trying to use them under NetBSD.

 before running make.

 --apb (Alan Barrett)

From: David Laight <david@l8s.co.uk>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/36128: CRLF handling for make
Date: Sat, 14 Apr 2007 18:16:08 +0100

 On Sat, Apr 14, 2007 at 03:20:02PM +0000, Alan Barrett wrote:
 > 
 >  On Fri, 06 Apr 2007, dholland@eecs.harvard.edu wrote:
 >  > It was mentioned on #pkgsrc irc that our make doesn't handle makefiles
 >  > that have embedded carriage returns (DOS-style CR/LF end of lines)...
 >  > ...and there seems no particular reason it shouldn't, since at least
 >  > one such makefile seems to exist in the wild.
 >  
 >  I am not at all convinced that this is a good idea.  I think that people
 >  who write their Makefiles under DOS should convert them to unix file
 >  format before trying to use them under NetBSD.

 I tend to agree, some makefiles might even contain a <cr> character in
 a variable assignment!
 (I've certainly used them in shell scripts.)

 Also it is rather the thin end of a big wedge!
 For instance the sun C compiler (or rather the preprocessor) doesn't
 treat \<cr><lf> as a line continuation.
 So do we start converting broken source files as well ?

 	David

 -- 
 David Laight: david@l8s.co.uk

From: dholland@eecs.harvard.edu (David Holland)
To: gnats-bugs@NetBSD.org
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org,
	dholland@eecs.harvard.edu
Subject: Re: bin/36128: CRLF handling for make
Date: Sat, 14 Apr 2007 14:27:51 -0400 (EDT)

 David Laight <david@l8s.co.uk> wrote:
  >> I am not at all convinced that this is a good idea.  I think that people
  >> who write their Makefiles under DOS should convert them to unix file
  >> format before trying to use them under NetBSD.
  >
  >  I tend to agree, some makefiles might even contain a <cr> character in
  >  a variable assignment!
  >  (I've certainly used them in shell scripts.)
  >  
  >  Also it is rather the thin end of a big wedge!
  >  For instance the sun C compiler (or rather the preprocessor) doesn't
  >  treat \<cr><lf> as a line continuation.
  >  So do we start converting broken source files as well ?

 For the record, this is about using our make with third-party packages
 that ship with makefiles apparently created under DOS. Nobody's
 advocating commiting DOS text files into NetBSD.

 And, you know, gmake already handles such makefiles. (gcc handles such
 source files, too, and has for years. I don't see how the Sun compiler
 is pertinent.)

 Meanwhile, if you use a literal control character of any kind, *maybe*
 excepting tabs, in a shell script or makefile you deserve the
 consequences. :-)  If you really need one, make one with tr or awk.

 -- 
    - David A. Holland / dholland@eecs.harvard.edu

Home	PR Database Search