NetBSD Problem Report #50111

From www@NetBSD.org  Sat Aug  1 12:21:18 2015
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 53EBAA6551
	for <gnats-bugs@gnats.NetBSD.org>; Sat,  1 Aug 2015 12:21:18 +0000 (UTC)
Message-Id: <20150801122117.5D01AA6555@mollari.NetBSD.org>
Date: Sat,  1 Aug 2015 12:21:17 +0000 (UTC)
From: achowe@snert.com
Reply-To: achowe@snert.com
To: gnats-bugs@NetBSD.org
Subject: sed does not support \n newline in replacement patterns.
X-Send-Pr-Version: www-1.0

>Number:         50111
>Category:       bin
>Synopsis:       sed does not support \n newline in replacement patterns.
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    bin-bug-people
>State:          open
>Class:          change-request
>Submitter-Id:   net
>Arrival-Date:   Sat Aug 01 12:25:00 +0000 2015
>Last-Modified:  Sun Aug 02 19:45:01 +0000 2015
>Originator:     Anthony Howe
>Release:        6.1.5
>Organization:
>Environment:
NetBSD kitsune.snert.com 6.1.5 NetBSD 6.1.5 (GENERIC) amd64

>Description:
Historically, sed did not support "\n" sequences in the substitution replacement string.  However, newer implementations, like GNU sed, do.
It would be useful for clarity and legibility of shell scripts to support "\n" in the replacement string.

So for example the current manner of splitting text with a newline is:

$ printf '123' | sed -e 's/./&\
/g'
1
2
3
$

When the above is used a shell script, any line/block indention would be disrupted, because of requiring a backslash and literal newline.

The proposed enhancement would allow:

$ printf '123' | sed -e 's/./&\n/g'
1
2
3
$

Which is cleaner representation.
>How-To-Repeat:

>Fix:

>Audit-Trail:
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/50111: sed does not support \n newline in replacement patterns.
Date: Sun, 02 Aug 2015 02:59:58 +0700

     Date:        Sat,  1 Aug 2015 12:25:00 +0000 (UTC)
     From:        achowe@snert.com
     Message-ID:  <20150801122500.515D9A654F@mollari.NetBSD.org>

 Please no.

 Because of ...

   | Historically, sed did not support "\n" sequences in the substitution

 it will always be non-portable to do so - whatever NetBSD's (or gnu's) sed
 decides to do.   If you really want to use non-portable extensions, just
 require & use gnu sed and be done with it.

 As for ...

   | So for example the current manner of splitting text with a newline is:
 [...]
   | When the above is used a shell script, any line/block indention would
   | be disrupted, because of requiring a backslash and literal newline.

 Not necessarily.   An example from one of my scripts that does some
 manipulation of HTML (this is just a fragment)

 NL='
 '
 sed     -e 's!\(.\)\(</*script>\)!\1\'"${NL}"'\2!g'             \
         -e 's/\(script[^>]*>\)\(.\)/\1\'"${NL}"'\2/g'           \
         -e 's/\(.\)<div/\1\'"${NL}"'<div/g'                     \
 	... [much more] ...

 If ${NL} was only to be used in sed subst expressions, the preceding \
 could be included in it as well.

 kre

From: Anthony Howe <achowe@snert.com>
To: gnats-bugs@NetBSD.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc: 
Subject: Re: bin/50111: sed does not support \n newline in replacement
 patterns.
Date: Sat, 1 Aug 2015 17:19:26 -0400

 On 01 Aug 2015 16:05, Robert Elz wrote:
 >  Please no.
 >  
 >  Because of ...
 >  
 >    | Historically, sed did not support "\n" sequences in the substitution
 >  
 >  it will always be non-portable to do so - whatever NetBSD's (or gnu's) sed
 >  decides to do.   If you really want to use non-portable extensions, just
 >  require & use gnu sed and be done with it.

 So you're happy to introduce some GNU tools, like GNU grep, but not sed.
  Or move tools a little bit forward into 21st century.  Happy to adopt
 GNU -- options, but poo poo handling of \n.  Took me two hours to figure
 out why "\n" wouldn't work.  I'm not a Linux fan, but some extensions
 just should be.  I balloted POSIX.2 in '90s.  Portability is a mantra,
 but some things that people just expect as given should be supported.


 -- 
 Anthony C Howe            Skype: SirWumpus                  SnertSoft
                         Twitter: SirWumpus      BarricadeMX & Milters
 http://snert.com/      http://nanozen.info/     http://snertsoft.com/

From: christos@zoulas.com (Christos Zoulas)
To: Anthony Howe <achowe@snert.com>, gnats-bugs@NetBSD.org, 
	gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc: 
Subject: Re: bin/50111: sed does not support \n newline in replacement patterns.
Date: Sun, 2 Aug 2015 00:10:10 -0400

 On Aug 1,  5:19pm, achowe@snert.com (Anthony Howe) wrote:
 -- Subject: Re: bin/50111: sed does not support \n newline in replacement pat

 | >  it will always be non-portable to do so - whatever NetBSD's (or gnu's) sed
 | >  decides to do.   If you really want to use non-portable extensions, just
 | >  require & use gnu sed and be done with it.
 | 
 | So you're happy to introduce some GNU tools, like GNU grep, but not sed.
 |  Or move tools a little bit forward into 21st century.  Happy to adopt
 | GNU -- options, but poo poo handling of \n.  Took me two hours to figure
 | out why "\n" wouldn't work.  I'm not a Linux fan, but some extensions
 | just should be.  I balloted POSIX.2 in '90s.  Portability is a mantra,
 | but some things that people just expect as given should be supported.

 Actually we are actively trying to limit the use of GPL code in the base
 system. We have a BSD version of grep for example in the tree, but it is
 not the default yet. We also try to port interesting features from the GNU
 tools to help with compatibility. The \n feature could be one of them.

 The comment above 'it will always be non-portable' refers to today's state;
 most sed implementations, don't recognize \n. It is always easier for
 us to fix a PR when there is patch with it. Otherwise we have to do most
 of the work, and adding \n to sed, might not be our highest priority.

 christos

From: Robert Elz <kre@munnari.OZ.AU>
To: christos@zoulas.com (Christos Zoulas)
Cc: Anthony Howe <achowe@snert.com>, gnats-bugs@NetBSD.org,
        gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: bin/50111: sed does not support \n newline in replacement patterns.
Date: Sun, 02 Aug 2015 13:45:19 +0700

     Date:        Sun, 2 Aug 2015 00:10:10 -0400
     From:        christos@zoulas.com (Christos Zoulas)
     Message-ID:  <20150802041010.5549717FDAB@rebar.astron.com>

 On Aug 1,  5:19pm, achowe@snert.com (Anthony Howe) apparently wrote:

 ["apparently" because I have not seen the message yet - just Christos' reply
 to it - so if I am missing anything, apologies].

   | So you're happy to introduce some GNU tools, like GNU grep, but not sed.

 First, I am not a NetBSD developer, so the "you're happy" was not
 justified - I'm just a user stating an opinion, what NetBSD decide to
 do is up to the developers (and if necessary, the core group).

 And second, you presume too much, I would much prefer no GNU code at all,
 or at the very least, no GPL code at all (which probably means much the
 same).   Unfortunately, sometimes there is little choice given the
 resources available.

   | Or move tools a little bit forward into 21st century.

 Quite happy to do that.   Of course, we would have to agree which way is
 forward.   Adding unnecessary alternative methods is not forward, that
 just adds confusion.   One way that works (especially when it is consistent
 with the other commands that add multiple lines) is sufficient.

   | Happy to adopt GNU -- options,

 Not at all, they're all stupid.   Anything that can't be controlled with
 less than 26 options should almost certainly be split into multiple commands.

 kre

From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/50111: sed does not support \n newline in replacement
 patterns.
Date: Sun, 2 Aug 2015 19:41:11 +0000

 On Sun, Aug 02, 2015 at 04:15:01AM +0000, Christos Zoulas wrote:
  >  most sed implementations, don't recognize \n. It is always easier for
  >  us to fix a PR when there is patch with it. Otherwise we have to do most
  >  of the work, and adding \n to sed, might not be our highest priority.

 I kind of agree with the OP in this one: there is no downside to
 supporting \n. It should be a small localized change, and it doesn't
 create any new incompatibilities; that is, \n in the replacement side
 of a sed substitution isn't otherwise meaningful.

 There is a can of worms, though, which is that it should also support
 \t... and \r... and then what about \b, \a, or even \f and \v?

 in any event sending in a patch will make it happen much faster.

 -- 
 David A. Holland
 dholland@netbsd.org

Home	PR Database Search