NetBSD Problem Report #4960

Received: (qmail 24600 invoked from network); 9 Feb 1998 12:04:39 -0000
Message-Id: <199802091210.NAA24952@saruman.ics.muni.cz>
Date: Mon, 9 Feb 1998 13:10:44 +0100 (MET)
From: Jaromir Dolecek <dolecek@ics.muni.cz>
Reply-To: dolecek@ics.muni.cz
To: gnats-bugs@gnats.netbsd.org
Subject: new patch to python
X-Send-Pr-Version: 3.95

>Number:         4960
>Category:       pkg
>Synopsis:       new patch to python
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    gnats-admin
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Feb 09 04:05:02 +0000 1998
>Closed-Date:    Mon Feb 09 18:14:26 +0000 1998
>Last-Modified:  Mon Feb 09 18:15:01 +0000 1998
>Originator:     Jaromir Dolecek
>Release:        1.3
>Organization:
	ICS MU, Brno, Czech Republic
>Environment:

System: NetBSD saruman.ics.muni.cz 1.3 NetBSD 1.3 (SARUMAN) #3: Wed Jan 21 13:29:49 MET 1998 dolecek@saruman.ics.muni.cz:/usr/home/dolecek/N13/usr/src/sys/arch/i386/compile/SARUMAN i386


>Description:
	There was found new bug in Python 1.5, related to dealing
	with unsigned/signed chars.
>How-To-Repeat:
	Try to use character 0xff in Python source (legitimate char
	of some national encodings).

>Fix:
	As Guido posted:

> Alexander Voinov <avv@isida.ipa.rssi.ru> writes:
> > Two most popular Russian encodings (koi8 and cp1251) use symbol with code 0x
FF.
> > This appears intolerable both for Python 1.4 and Python 1.5 (EOF).

Martin v. Loewis replies:
> As you might now, a work-around is to use \377 in the string instead.
> I agree that this is not really convenient, and should be fixed.

Agreed.  Here's a patch for Parser/tokenizer.c.  Let me know whether
this works:

--- Parser/tokenizer.c.orig	Tue Apr 29 23:03:03 1997
+++ Parser/tokenizer.c	Mon Feb  9 13:05:34 1998
@@ -46,6 +46,14 @@
 /* Don't ever change this -- it would break the portability of Python code */
 #define TABSIZE 8

+/* Convert a possibly signed character to a nonnegative int */
+/* XXX This assumes characters are 8 bits wide */
+#ifdef __CHAR_UNSIGNED__
+#define Py_CHARMASK(c)		(c)
+#else
+#define Py_CHARMASK(c)		((c) & 0xff)
+#endif
+
 /* Forward */
 static struct tok_state *tok_new Py_PROTO((void));
 static int tok_nextc Py_PROTO((struct tok_state *tok));
@@ -178,7 +186,7 @@
 {
 	for (;;) {
 		if (tok->cur != tok->inp) {
-			return *tok->cur++; /* Fast path */
+			return Py_CHARMASK(*tok->cur++); /* Fast path */
 		}
 		if (tok->done != E_OK)
 			return EOF;
@@ -197,7 +205,7 @@
 				tok->buf = tok->cur;
 			tok->lineno++;
 			tok->inp = end;
-			return *tok->cur++;
+			return Py_CHARMASK(*tok->cur++);
 		}
 		if (tok->prompt != NULL) {
 			char *new = PyOS_Readline(tok->prompt);
>Release-Note:
>Audit-Trail:
State-Changed-From-To: open->closed 
State-Changed-By: tron 
State-Changed-When: Mon Feb 9 10:14:26 PST 1998 
State-Changed-Why:  
Added supplied patch file, thanks. 
>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.51 2026/08/10 02:28:17 riastradh Exp $
$NetBSD: gnats_config.sh,v 1.10 2026/05/13 22:00:09 riastradh Exp $
Copyright © 1994-2026 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.