NetBSD Problem Report #41136

From www@NetBSD.org  Sat Apr  4 00:14:06 2009
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id 319A463B946
	for <gnats-bugs@gnats.netbsd.org>; Sat,  4 Apr 2009 00:14:06 +0000 (UTC)
Message-Id: <20090404001405.93FA263B8C8@www.NetBSD.org>
Date: Sat,  4 Apr 2009 00:14:05 +0000 (UTC)
From: marcotte@panix.com
Reply-To: marcotte@panix.com
To: gnats-bugs@NetBSD.org
Subject: \<word search doesn't work in vi
X-Send-Pr-Version: www-1.0

>Number:         41136
>Category:       bin
>Synopsis:       \<word search doesn't work in vi
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    jld
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Apr 04 00:15:00 +0000 2009
>Closed-Date:    Fri Apr 17 02:58:37 +0000 2009
>Last-Modified:  Fri Apr 17 02:58:37 +0000 2009
>Originator:     Brian Marcotte
>Release:        5.0 RC3
>Organization:
Panix
>Environment:
NetBSD panix2.panix.com 5.0_RC3 NetBSD 5.0_RC3 (PANIX-XEN3U-USER) #0: Mon Mar 23 00:30:15 EDT 2009  root@juggler.panix.com:/misc3/obj/misc2/devel/netbsd/5.0-RC3/src/sys/arch/i386/compile/PANIX-XEN3U-USER i386

>Description:
\<word (beginning of word) search doesn't work in vi. It seems to ignore the \<.

word\> (end-of word) DOES work though.

>How-To-Repeat:
echo "aaaaa aaaaa aaa" > somefile
vi somefile

When in vi, search for \<aaa
It should bring you to the beginning of one of those words, but instead will go to any set of "aaa".

>Fix:
In src/dist/nvi/regex/regex2.h, OUT is now defined as WEOF which is -1.

Later in src/dist/nvi/regex/engine.c, OUT is used as a base to define BOL, EOL, BOW, etc. The important part is that BOL=OUT+1 (or 0).

In a couple different functions, flagch is initialized to 0 probably assuming none BOL, EOL, etc. could ever equal 0.

Presumably, OUT needs to be defined to something outside the normal character range.


>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: bin-bug-people->jld
Responsible-Changed-By: jld@NetBSD.org
Responsible-Changed-When: Sat, 04 Apr 2009 01:07:54 +0000
Responsible-Changed-Why:
I was looking at this (and 41134) before they were sent-PR.


From: Takehiko NOZAKI <takehiko.nozaki@gmail.com>
To: gnats-bugs@netbsd.org
Cc: marcotte@panix.com
Subject: Re: bin/41136: \<word search doesn't work in vi
Date: Sat, 4 Apr 2009 10:43:44 +0900

 --000e0cd2e12cb8c7230466b0cb70
 Content-Type: text/plain; charset=ISO-8859-1
 Content-Transfer-Encoding: 7bit

 hi,

 please try attached pacth.

 i don't have enough time to test this patch now.
 i'll commit it (if there is no regression), maybe next mid (or end) of week.
 i appologies for your inconvinience.

 very truly yours.
 -- 
 Takehiko NOZAKI<takehiko.nozaki@gmail.com>

 --000e0cd2e12cb8c7230466b0cb70
 Content-Type: application/octet-stream; name=patch-bin41136
 Content-Disposition: attachment; filename=patch-bin41136
 Content-Transfer-Encoding: base64
 X-Attachment-Id: f_ft3mxcpf0

 SW5kZXg6IHJlZ2V4L2VuZ2luZS5jCj09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09
 PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT0KUkNTIGZpbGU6IC9jdnNyb290L3NyYy9k
 aXN0L252aS9yZWdleC9lbmdpbmUuYyx2CnJldHJpZXZpbmcgcmV2aXNpb24gMS41CmRpZmYgLXUg
 LXIxLjUgZW5naW5lLmMKLS0tIHJlZ2V4L2VuZ2luZS5jCTIyIEZlYiAyMDA5IDExOjM0OjUzIC0w
 MDAwCTEuNQorKysgcmVnZXgvZW5naW5lLmMJNCBBcHIgMjAwOSAwMToyOTozMCAtMDAwMApAQCAt
 OTcsMTYgKzk3LDEzIEBACiBzdGF0aWMgUkNIQVJfVCAqYmFja3JlZiBfX1AoKHN0cnVjdCBtYXRj
 aCAqbSwgUkNIQVJfVCAqc3RhcnQsIFJDSEFSX1QgKnN0b3AsIHNvcG5vIHN0YXJ0c3QsIHNvcG5v
 IHN0b3BzdCwgc29wbm8gbGV2KSk7CiBzdGF0aWMgUkNIQVJfVCAqZmFzdCBfX1AoKHN0cnVjdCBt
 YXRjaCAqbSwgUkNIQVJfVCAqc3RhcnQsIFJDSEFSX1QgKnN0b3AsIHNvcG5vIHN0YXJ0c3QsIHNv
 cG5vIHN0b3BzdCkpOwogc3RhdGljIFJDSEFSX1QgKnNsb3cgX19QKChzdHJ1Y3QgbWF0Y2ggKm0s
 IFJDSEFSX1QgKnN0YXJ0LCBSQ0hBUl9UICpzdG9wLCBzb3BubyBzdGFydHN0LCBzb3BubyBzdG9w
 c3QpKTsKLXN0YXRpYyBzdGF0ZXMgc3RlcCBfX1AoKHN0cnVjdCByZV9ndXRzICpnLCBzb3BubyBz
 dGFydCwgc29wbm8gc3RvcCwgc3RhdGVzIGJlZiwgaW50IGNoLCBzdGF0ZXMgYWZ0KSk7Ci0jZGVm
 aW5lCUJPTAkoT1VUKzEpCitzdGF0aWMgc3RhdGVzIHN0ZXAgX19QKChzdHJ1Y3QgcmVfZ3V0cyAq
 Zywgc29wbm8gc3RhcnQsIHNvcG5vIHN0b3AsIHN0YXRlcyBiZWYsIGludCBmbGFnLCBSQ0hBUl9U
 IGNoLCBzdGF0ZXMgYWZ0KSk7CisjZGVmaW5lCUJPTAkoMSkKICNkZWZpbmUJRU9MCShCT0wrMSkK
 ICNkZWZpbmUJQk9MRU9MCShCT0wrMikKICNkZWZpbmUJTk9USElORwkoQk9MKzMpCiAjZGVmaW5l
 CUJPVwkoQk9MKzQpCiAjZGVmaW5lCUVPVwkoQk9MKzUpCi0jZGVmaW5lCUNPREVNQVgJKEJPTCs1
 KQkJLyogaGlnaGVzdCBjb2RlIHVzZWQgKi8KLSNkZWZpbmUJTk9OQ0hBUihjKQkoKFVDSEFSX1Qp
 KGMpID4gKFVDSEFSX1QpUkNIQVJfVF9NQVgpCi0jZGVmaW5lCU5OT05DSEFSCShDT0RFTUFYLUNI
 QVJfTUFYKQogI2lmZGVmIFJFREVCVUcKIHN0YXRpYyB2b2lkIHByaW50IF9fUCgoc3RydWN0IG1h
 dGNoICptLCBjaGFyICpjYXB0aW9uLCBzdGF0ZXMgc3QsIGludCBjaCwgRklMRSAqZCkpOwogI2Vu
 ZGlmCkBAIC03MjQsNyArNzIxLDcgQEAKIAogCUNMRUFSKHN0KTsKIAlTRVQxKHN0LCBzdGFydHN0
 KTsKLQlzdCA9IHN0ZXAobS0+Zywgc3RhcnRzdCwgc3RvcHN0LCBzdCwgTk9USElORywgc3QpOwor
 CXN0ID0gc3RlcChtLT5nLCBzdGFydHN0LCBzdG9wc3QsIHN0LCBOT1RISU5HLCBPVVQsIHN0KTsK
 IAlBU1NJR04oZnJlc2gsIHN0KTsKIAlTUCgic3RhcnQiLCBzdCwgKnApOwogCWNvbGRwID0gTlVM
 TDsKQEAgLTc1MCw3ICs3NDcsNyBAQAogCQl9CiAJCWlmIChpICE9IDApIHsKIAkJCWZvciAoOyBp
 ID4gMDsgaS0tKQotCQkJCXN0ID0gc3RlcChtLT5nLCBzdGFydHN0LCBzdG9wc3QsIHN0LCBmbGFn
 Y2gsIHN0KTsKKwkJCQlzdCA9IHN0ZXAobS0+Zywgc3RhcnRzdCwgc3RvcHN0LCBzdCwgZmxhZ2No
 LCBPVVQsIHN0KTsKIAkJCVNQKCJib2xlb2wiLCBzdCwgYyk7CiAJCX0KIApAQCAtNzY0LDcgKzc2
 MSw3IEBACiAJCQlmbGFnY2ggPSBFT1c7CiAJCX0KIAkJaWYgKGZsYWdjaCA9PSBCT1cgfHwgZmxh
 Z2NoID09IEVPVykgewotCQkJc3QgPSBzdGVwKG0tPmcsIHN0YXJ0c3QsIHN0b3BzdCwgc3QsIGZs
 YWdjaCwgc3QpOworCQkJc3QgPSBzdGVwKG0tPmcsIHN0YXJ0c3QsIHN0b3BzdCwgc3QsIGZsYWdj
 aCwgT1VULCBzdCk7CiAJCQlTUCgiYm93ZW93Iiwgc3QsIGMpOwogCQl9CiAKQEAgLTc3Niw5ICs3
 NzMsOSBAQAogCQlBU1NJR04odG1wLCBzdCk7CiAJCUFTU0lHTihzdCwgZnJlc2gpOwogCQlhc3Nl
 cnQoYyAhPSBPVVQpOwotCQlzdCA9IHN0ZXAobS0+Zywgc3RhcnRzdCwgc3RvcHN0LCB0bXAsIGMs
 IHN0KTsKKwkJc3QgPSBzdGVwKG0tPmcsIHN0YXJ0c3QsIHN0b3BzdCwgdG1wLCAwLCBjLCBzdCk7
 CiAJCVNQKCJhZnQiLCBzdCwgYyk7Ci0JCWFzc2VydChFUShzdGVwKG0tPmcsIHN0YXJ0c3QsIHN0
 b3BzdCwgc3QsIE5PVEhJTkcsIHN0KSwgc3QpKTsKKwkJYXNzZXJ0KEVRKHN0ZXAobS0+Zywgc3Rh
 cnRzdCwgc3RvcHN0LCBzdCwgTk9USElORywgT1VULCBzdCksIHN0KSk7CiAJCXArKzsKIAl9CiAK
 QEAgLTgxNyw3ICs4MTQsNyBAQAogCUNMRUFSKHN0KTsKIAlTRVQxKHN0LCBzdGFydHN0KTsKIAlT
 UCgic3N0YXJ0Iiwgc3QsICpwKTsKLQlzdCA9IHN0ZXAobS0+Zywgc3RhcnRzdCwgc3RvcHN0LCBz
 dCwgTk9USElORywgc3QpOworCXN0ID0gc3RlcChtLT5nLCBzdGFydHN0LCBzdG9wc3QsIHN0LCBO
 T1RISU5HLCBPVVQsIHN0KTsKIAltYXRjaHAgPSBOVUxMOwogCWZvciAoOzspIHsKIAkJLyogbmV4
 dCBjaGFyYWN0ZXIgKi8KQEAgLTgzOSw3ICs4MzYsNyBAQAogCQl9CiAJCWlmIChpICE9IDApIHsK
 IAkJCWZvciAoOyBpID4gMDsgaS0tKQotCQkJCXN0ID0gc3RlcChtLT5nLCBzdGFydHN0LCBzdG9w
 c3QsIHN0LCBmbGFnY2gsIHN0KTsKKwkJCQlzdCA9IHN0ZXAobS0+Zywgc3RhcnRzdCwgc3RvcHN0
 LCBzdCwgZmxhZ2NoLCBPVVQsIHN0KTsKIAkJCVNQKCJzYm9sZW9sIiwgc3QsIGMpOwogCQl9CiAK
 QEAgLTg1Myw3ICs4NTAsNyBAQAogCQkJZmxhZ2NoID0gRU9XOwogCQl9CiAJCWlmIChmbGFnY2gg
 PT0gQk9XIHx8IGZsYWdjaCA9PSBFT1cpIHsKLQkJCXN0ID0gc3RlcChtLT5nLCBzdGFydHN0LCBz
 dG9wc3QsIHN0LCBmbGFnY2gsIHN0KTsKKwkJCXN0ID0gc3RlcChtLT5nLCBzdGFydHN0LCBzdG9w
 c3QsIHN0LCBmbGFnY2gsIE9VVCwgc3QpOwogCQkJU1AoInNib3dlb3ciLCBzdCwgYyk7CiAJCX0K
 IApAQCAtODY3LDkgKzg2NCw5IEBACiAJCUFTU0lHTih0bXAsIHN0KTsKIAkJQVNTSUdOKHN0LCBl
 bXB0eSk7CiAJCWFzc2VydChjICE9IE9VVCk7Ci0JCXN0ID0gc3RlcChtLT5nLCBzdGFydHN0LCBz
 dG9wc3QsIHRtcCwgYywgc3QpOworCQlzdCA9IHN0ZXAobS0+Zywgc3RhcnRzdCwgc3RvcHN0LCB0
 bXAsIDAsIGMsIHN0KTsKIAkJU1AoInNhZnQiLCBzdCwgYyk7Ci0JCWFzc2VydChFUShzdGVwKG0t
 PmcsIHN0YXJ0c3QsIHN0b3BzdCwgc3QsIE5PVEhJTkcsIHN0KSwgc3QpKTsKKwkJYXNzZXJ0KEVR
 KHN0ZXAobS0+Zywgc3RhcnRzdCwgc3RvcHN0LCBzdCwgTk9USElORywgT1VULCBzdCksIHN0KSk7
 CiAJCXArKzsKIAl9CiAKQEAgLTg4MCwyNCArODc3LDIyIEBACiAvKgogIC0gc3RlcCAtIG1hcCBz
 ZXQgb2Ygc3RhdGVzIHJlYWNoYWJsZSBiZWZvcmUgY2hhciB0byBzZXQgcmVhY2hhYmxlIGFmdGVy
 CiAgPT0gc3RhdGljIHN0YXRlcyBzdGVwKHJlZ2lzdGVyIHN0cnVjdCByZV9ndXRzICpnLCBzb3Bu
 byBzdGFydCwgc29wbm8gc3RvcCwgXAotID09CXJlZ2lzdGVyIHN0YXRlcyBiZWYsIGludCBjaCwg
 cmVnaXN0ZXIgc3RhdGVzIGFmdCk7Ci0gPT0gI2RlZmluZQlCT0wJKE9VVCsxKQorID09CXJlZ2lz
 dGVyIHN0YXRlcyBiZWYsIGludCBmbGFnLCBSQ0hBUl9UIGNoLCByZWdpc3RlciBzdGF0ZXMgYWZ0
 KTsKKyA9PSAjZGVmaW5lCUJPTAkoMSkKICA9PSAjZGVmaW5lCUVPTAkoQk9MKzEpCiAgPT0gI2Rl
 ZmluZQlCT0xFT0wJKEJPTCsyKQogID09ICNkZWZpbmUJTk9USElORwkoQk9MKzMpCiAgPT0gI2Rl
 ZmluZQlCT1cJKEJPTCs0KQogID09ICNkZWZpbmUJRU9XCShCT0wrNSkKLSA9PSAjZGVmaW5lCUNP
 REVNQVgJKEJPTCs1KQkJLy8gaGlnaGVzdCBjb2RlIHVzZWQKLSA9PSAjZGVmaW5lCU5PTkNIQVIo
 YykJKChjKSA+IENIQVJfTUFYKQotID09ICNkZWZpbmUJTk5PTkNIQVIJKENPREVNQVgtQ0hBUl9N
 QVgpCiAgKi8KIHN0YXRpYyBzdGF0ZXMKLXN0ZXAoZywgc3RhcnQsIHN0b3AsIGJlZiwgY2gsIGFm
 dCkKK3N0ZXAoZywgc3RhcnQsIHN0b3AsIGJlZiwgZmxhZywgY2gsIGFmdCkKIHJlZ2lzdGVyIHN0
 cnVjdCByZV9ndXRzICpnOwogc29wbm8gc3RhcnQ7CQkJLyogc3RhcnQgc3RhdGUgd2l0aGluIHN0
 cmlwICovCiBzb3BubyBzdG9wOwkJCS8qIHN0YXRlIGFmdGVyIHN0b3Agc3RhdGUgd2l0aGluIHN0
 cmlwICovCiByZWdpc3RlciBzdGF0ZXMgYmVmOwkJLyogc3RhdGVzIHJlYWNoYWJsZSBiZWZvcmUg
 Ki8KLWludCBjaDsJCQkJLyogY2hhcmFjdGVyIG9yIE5PTkNIQVIgY29kZSAqLworaW50IGZsYWc7
 CQkJLyogTk9OQ0hBUiBmbGFnICovCitSQ0hBUl9UIGNoOwkJCS8qIGNoYXJhY3RlciBjb2RlICov
 CiByZWdpc3RlciBzdGF0ZXMgYWZ0OwkJLyogc3RhdGVzIGFscmVhZHkga25vd24gcmVhY2hhYmxl
 IGFmdGVyICovCiB7CiAJcmVnaXN0ZXIgY3NldCAqY3M7CkBAIC05MTcsMzMgKzkxMiwzMyBAQAog
 CQkJYnJlYWs7CiAJCWNhc2UgT0NIQVI6CiAJCQkvKiBvbmx5IGNoYXJhY3RlcnMgY2FuIG1hdGNo
 ICovCi0JCQlhc3NlcnQoIU5PTkNIQVIoY2gpIHx8IGNoICE9IGQpOworCQkJYXNzZXJ0KCFmbGFn
 IHx8IGNoICE9IGQpOwogCQkJaWYgKGNoID09IGQpCiAJCQkJRldEKGFmdCwgYmVmLCAxKTsKIAkJ
 CWJyZWFrOwogCQljYXNlIE9CT0w6Ci0JCQlpZiAoY2ggPT0gQk9MIHx8IGNoID09IEJPTEVPTCkK
 KwkJCWlmIChmbGFnID09IEJPTCB8fCBmbGFnID09IEJPTEVPTCkKIAkJCQlGV0QoYWZ0LCBiZWYs
 IDEpOwogCQkJYnJlYWs7CiAJCWNhc2UgT0VPTDoKLQkJCWlmIChjaCA9PSBFT0wgfHwgY2ggPT0g
 Qk9MRU9MKQorCQkJaWYgKGZsYWcgPT0gRU9MIHx8IGZsYWcgPT0gQk9MRU9MKQogCQkJCUZXRChh
 ZnQsIGJlZiwgMSk7CiAJCQlicmVhazsKIAkJY2FzZSBPQk9XOgotCQkJaWYgKGNoID09IEJPVykK
 KwkJCWlmIChmbGFnID09IEJPVykKIAkJCQlGV0QoYWZ0LCBiZWYsIDEpOwogCQkJYnJlYWs7CiAJ
 CWNhc2UgT0VPVzoKLQkJCWlmIChjaCA9PSBFT1cpCisJCQlpZiAoZmxhZyA9PSBFT1cpCiAJCQkJ
 RldEKGFmdCwgYmVmLCAxKTsKIAkJCWJyZWFrOwogCQljYXNlIE9BTlk6Ci0JCQlpZiAoIU5PTkNI
 QVIoY2gpKQorCQkJaWYgKCFmbGFnKQogCQkJCUZXRChhZnQsIGJlZiwgMSk7CiAJCQlicmVhazsK
 IAkJY2FzZSBPQU5ZT0Y6CiAJCQljcyA9ICZnLT5zZXRzW2RdOwotCQkJaWYgKCFOT05DSEFSKGNo
 KSAmJiBDSElOKGNzLCBjaCkpCisJCQlpZiAoIWZsYWcgJiYgQ0hJTihjcywgY2gpKQogCQkJCUZX
 RChhZnQsIGJlZiwgMSk7CiAJCQlicmVhazsKIAkJY2FzZSBPQkFDS186CQkvKiBpZ25vcmVkIGhl
 cmUgKi8K
 --000e0cd2e12cb8c7230466b0cb70--

From: Brian Marcotte <marcotte@panix.com>
To: Takehiko NOZAKI <takehiko.nozaki@gmail.com>
Cc: gnats-bugs@netbsd.org
Subject: Re: bin/41136: \<word search doesn't work in vi
Date: Fri, 3 Apr 2009 22:20:44 -0400

 > please try attached patch.

 I applied it to the netbsd-5 branch. It appears to work now, but I only
 spent a few minutes testing it.

 --
 - Brian

From: Jed Davis <jld@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/41136: \<word search doesn't work in vi
Date: Sat, 4 Apr 2009 02:14:55 -0400

 I've also tested the patch on netbsd-5, including in a UTF-8 locale with
 high characters; it seems to be correct.

From: Takehiko NOZAKI <tnozaki@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/41136 CVS commit: src/dist/nvi
Date: Sun, 12 Apr 2009 14:47:51 +0000

 Module Name:	src
 Committed By:	tnozaki
 Date:		Sun Apr 12 14:47:51 UTC 2009

 Modified Files:
 	src/dist/nvi/common: multibyte.h
 	src/dist/nvi/regex: engine.c

 Log Message:
 fix PR/41136: \<word search doesn't work in vi
 don't reuse RCHAR_T(=wchar_t)'s bits, CSI wchar_t is opaque object.


 To generate a diff of this commit:
 cvs rdiff -u -r1.2 -r1.3 src/dist/nvi/common/multibyte.h
 cvs rdiff -u -r1.5 -r1.6 src/dist/nvi/regex/engine.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->pending-pullups
State-Changed-By: tnozaki@NetBSD.org
State-Changed-When: Sun, 12 Apr 2009 14:54:55 +0000
State-Changed-Why:
patch applyed, and pending pullup.

From: Soren Jacobsen <snj@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/41136 CVS commit: [netbsd-5] src/dist/nvi
Date: Mon, 13 Apr 2009 20:59:21 +0000

 Module Name:	src
 Committed By:	snj
 Date:		Mon Apr 13 20:59:20 UTC 2009

 Modified Files:
 	src/dist/nvi/common [netbsd-5]: multibyte.h
 	src/dist/nvi/regex [netbsd-5]: engine.c

 Log Message:
 Pull up following revision(s) (requested by tnozaki in ticket #698):
 	dist/nvi/common/multibyte.h: revision 1.3
 	dist/nvi/regex/engine.c: revision 1.6
 fix PR/41136: \<word search doesn't work in vi
 don't reuse RCHAR_T(=wchar_t)'s bits, CSI wchar_t is opaque object.


 To generate a diff of this commit:
 cvs rdiff -u -r1.1.1.2.6.1 -r1.1.1.2.6.2 src/dist/nvi/common/multibyte.h
 cvs rdiff -u -r1.1.1.2.6.2 -r1.1.1.2.6.3 src/dist/nvi/regex/engine.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: pending-pullups->closed
State-Changed-By: tsutsui@NetBSD.org
State-Changed-When: Fri, 17 Apr 2009 11:58:37 +0900
State-Changed-Why:
Pulled up to netbsd-5.


>Unformatted:

Home	PR Database Search