NetBSD Problem Report #4960
Received: (qmail 24600 invoked from network); 9 Feb 1998 12:04:39 -0000
Message-Id: <199802091210.NAA24952@saruman.ics.muni.cz>
Date: Mon, 9 Feb 1998 13:10:44 +0100 (MET)
From: Jaromir Dolecek <dolecek@ics.muni.cz>
Reply-To: dolecek@ics.muni.cz
To: gnats-bugs@gnats.netbsd.org
Subject: new patch to python
X-Send-Pr-Version: 3.95
>Number: 4960
>Category: pkg
>Synopsis: new patch to python
>Confidential: no
>Severity: non-critical
>Priority: low
>Responsible: gnats-admin
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Mon Feb 09 04:05:02 +0000 1998
>Closed-Date: Mon Feb 09 18:14:26 +0000 1998
>Last-Modified: Mon Feb 09 18:15:01 +0000 1998
>Originator: Jaromir Dolecek
>Release: 1.3
>Organization:
ICS MU, Brno, Czech Republic
>Environment:
System: NetBSD saruman.ics.muni.cz 1.3 NetBSD 1.3 (SARUMAN) #3: Wed Jan 21 13:29:49 MET 1998 dolecek@saruman.ics.muni.cz:/usr/home/dolecek/N13/usr/src/sys/arch/i386/compile/SARUMAN i386
>Description:
There was found new bug in Python 1.5, related to dealing
with unsigned/signed chars.
>How-To-Repeat:
Try to use character 0xff in Python source (legitimate char
of some national encodings).
>Fix:
As Guido posted:
> Alexander Voinov <avv@isida.ipa.rssi.ru> writes:
> > Two most popular Russian encodings (koi8 and cp1251) use symbol with code 0x
FF.
> > This appears intolerable both for Python 1.4 and Python 1.5 (EOF).
Martin v. Loewis replies:
> As you might now, a work-around is to use \377 in the string instead.
> I agree that this is not really convenient, and should be fixed.
Agreed. Here's a patch for Parser/tokenizer.c. Let me know whether
this works:
--- Parser/tokenizer.c.orig Tue Apr 29 23:03:03 1997
+++ Parser/tokenizer.c Mon Feb 9 13:05:34 1998
@@ -46,6 +46,14 @@
/* Don't ever change this -- it would break the portability of Python code */
#define TABSIZE 8
+/* Convert a possibly signed character to a nonnegative int */
+/* XXX This assumes characters are 8 bits wide */
+#ifdef __CHAR_UNSIGNED__
+#define Py_CHARMASK(c) (c)
+#else
+#define Py_CHARMASK(c) ((c) & 0xff)
+#endif
+
/* Forward */
static struct tok_state *tok_new Py_PROTO((void));
static int tok_nextc Py_PROTO((struct tok_state *tok));
@@ -178,7 +186,7 @@
{
for (;;) {
if (tok->cur != tok->inp) {
- return *tok->cur++; /* Fast path */
+ return Py_CHARMASK(*tok->cur++); /* Fast path */
}
if (tok->done != E_OK)
return EOF;
@@ -197,7 +205,7 @@
tok->buf = tok->cur;
tok->lineno++;
tok->inp = end;
- return *tok->cur++;
+ return Py_CHARMASK(*tok->cur++);
}
if (tok->prompt != NULL) {
char *new = PyOS_Readline(tok->prompt);
>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->closed
State-Changed-By: tron
State-Changed-When: Mon Feb 9 10:14:26 PST 1998
State-Changed-Why:
Added supplied patch file, thanks.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.