NetBSD Problem Report #32445

From thesing@gigasun.cs.uni-sb.de  Tue Jan  3 14:00:11 2006
Return-Path: <thesing@gigasun.cs.uni-sb.de>
Received: from triton.rz.uni-saarland.de (triton.rz.uni-saarland.de [134.96.7.25])
	by narn.netbsd.org (Postfix) with ESMTP id A817063BA0A
	for <gnats-bugs@gnats.NetBSD.org>; Tue,  3 Jan 2006 14:00:10 +0000 (UTC)
Message-Id: <200601031400.k03E076b011672@gigasun.cs.uni-sb.de>
Date: Tue, 3 Jan 2006 15:00:07 +0100 (CET)
From: thesing@cs.uni-sb.de
Reply-To: thesing@cs.uni-sb.de
To: gnats-bugs@netbsd.org
Subject: x11/Xaw3d causes graphics/xfig to hang under kde
X-Send-Pr-Version: 3.95

>Number:         32445
>Category:       pkg
>Synopsis:       x11/Xaw3d has an overflow error in geometry computation code, causing xfig to hang under KDE
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    markd
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Tue Jan 03 14:05:00 +0000 2006
>Closed-Date:    Mon Mar 13 00:21:20 +0000 2006
>Last-Modified:  Mon Mar 13 00:21:20 +0000 2006
>Originator:     Stephan Thesing
>Release:        NetBSD 3.99.11
>Organization:
=  Tel.: +49-681-302-5571      = Universitaet des Saarlandes =
=  Fax.: +49-681-302-3065      = Postfach 15 11 50           =
=  Compiler Research Group     = 66041 Saarbruecken          =
=  FR 6.2 - Informatik         = GERMANY                     =
>Environment:


System: NetBSD gargoyle.cs.uni-sb.de 3.99.11 NetBSD 3.99.11 (Gargoyle) #5: Mon Nov 7 08:53:22 CET 2005 thesing@gargoyle.cs.uni-sb.de:/local/thesing/netbsd/current/obj/sys/arch/i386/compile.i386/Gargoyle i386
Architecture: i386
Machine: i386
>Description:
 The x11/Xaw3d library has an integer overflow error in the computation of the
  geometry for a Box Layout (file Box.c).  There, the box tries to extend its width until its
   height fits within the constraint height (influenced by the window manager).
 Unfortunately, widths are 16bit  unsigned and in the error case (occuring under KDE, windowmaker, twm
   work fine here:-), the constraint width is 65535, i.e. maximal.
 The code loops until either the computed height is smaller than the constraint height or
  the width exceeds the constraint width.  In each loop iteration, the width of the box layout is
  doubled.  This loop does not terminate, if one chooses unfortunate initial width, as the width
   wraps around if it overflows 16 bits and if the maximal constraint width is SHORT_INT_MAX.
>How-To-Repeat:
 Try to use xfig under KDE.
>Fix:
The following patch for X11/Xaw3d tries to capture the overflow before it wraps around the
 `width' variable, setting the width to the maximal one.
It makes xfig work under KDE.

--- Box.c.orig	1996-10-15 14:41:18.000000000 +0000
+++ Box.c	2006-01-02 15:13:32.000000000 +0000
@@ -352,8 +352,12 @@
 	}
 	else {
 	    width = preferred_width;
+           if (0==width) width=1;
 	    do { /* find some width big enough to stay within this height */
-		width *= 2;
+               if (width>=32768) /* overflow */
+                 width=constraint->width;
+               else
+                 width*=2;
 		if (width > constraint->width) width = constraint->width;
 		DoLayout(w, width, 0, &preferred_width, &preferred_height, FALSE);
 	    } while (preferred_height > constraint->height &&

>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: pkg-manager->markd
Responsible-Changed-By: markd@netbsd.org
Responsible-Changed-When: Fri, 10 Mar 2006 08:58:43 +0000
Responsible-Changed-Why:
I'll look at it.


From: Mark Davies <markd@netbsd.org>
To: gnats-bugs@netbsd.org
Cc: 
Subject: PR/32445 CVS commit: pkgsrc/x11/Xaw3d
Date: Mon, 13 Mar 2006 00:16:23 +0000 (UTC)

 Module Name:	pkgsrc
 Committed By:	markd
 Date:		Mon Mar 13 00:16:23 UTC 2006

 Modified Files:
 	pkgsrc/x11/Xaw3d: Makefile distinfo
 Added Files:
 	pkgsrc/x11/Xaw3d/patches: patch-ac

 Log Message:
 The x11/Xaw3d library has an integer overflow error in the computation
 of the geometry for a Box Layout (file Box.c).  There, the box tries
 to extend its width until its height fits within the constraint height
 (influenced by the window manager).  Unfortunately, widths are 16bit
 unsigned and in the error case (occuring under KDE), the constraint
 width is 65535, i.e. maximal.
 The code loops until either the computed height is smaller than the
 constraint height or the width exceeds the constraint width.  In each
 loop iteration, the width of the box layout is doubled.  This loop
 does not terminate, if one chooses unfortunate initial width, as the
 width wraps around if it overflows 16 bits and if the maximal
 constraint width is SHORT_INT_MAX.
 Patch tries to capture the overflow before it wraps around the
 `width' variable, setting the width to the maximal one.

 From Stephan Thesing in PR pkg/32445. Bump PKGREVISION.


 To generate a diff of this commit:
 cvs rdiff -r1.47 -r1.48 pkgsrc/x11/Xaw3d/Makefile
 cvs rdiff -r1.8 -r1.9 pkgsrc/x11/Xaw3d/distinfo
 cvs rdiff -r0 -r1.1 pkgsrc/x11/Xaw3d/patches/patch-ac

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->closed
State-Changed-By: markd@netbsd.org
State-Changed-When: Mon, 13 Mar 2006 00:21:20 +0000
State-Changed-Why:
patch applied, thanks


>Unformatted:

Home	PR Database Search