NetBSD Problem Report #60634

From mlelstv@arnold.1st.de  Sun Aug 23 07:15:22 2026
Return-Path: <mlelstv@arnold.1st.de>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mail.netbsd.org", Issuer "YR2" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 61CDF1A923C
	for <gnats-bugs@gnats.NetBSD.org>; Sun, 23 Aug 2026 07:15:22 +0000 (UTC)
Message-Id: <20260823071431.DF57EBC165@arnold.1st.de>
Date: Sun, 23 Aug 2026 09:14:31 +0200 (CEST)
From: mlelstv@serpens.de
Reply-To: mlelstv@serpens.de
To: gnats-bugs@NetBSD.org
Subject: digest rmd160 bug for files >= 4GB
X-Send-Pr-Version: 3.95
X-From4GNATS: "mlelstv@serpens.de via gnats" <gnats-admin@NetBSD.org>

>Number:         60634
>Category:       pkg
>Synopsis:       digest rmd160 bug for files >= 4GB
>Confidential:   no
>Severity:       critical
>Priority:       low
>Responsible:    agc
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Aug 23 07:20:00 +0000 2026
>Last-Modified:  Wed Aug 26 06:51:41 +0000 2026
>Originator:     Michael van Elst
>Release:        NetBSD 11.0_STABLE
>Organization:
>Environment:
System: NetBSD gossam 11.0_STABLE NetBSD 11.0_STABLE (GENERIC) #40: Fri Aug 21 16:29:44 CEST 2026 mlelstv@gossam:/home/netbsd11/obj.amd64/home/netbsd11/src/sys/arch/amd64/compile/GENERIC amd64
Architecture: x86_64
Machine: amd64
>Description:

The pktools/digest implementation of ripemd160 has a flaw that fails to
compute the correct checksum on 64bit systems.

When incrementing the total length value for a hash block, it checks
the lower 32 bits for an overflow to update the upper 32 bits:

         if (context->length[0] + nbytes < context->length[0])
                 context->length[1]++;           /* overflow to msb of length */
         context->length[0] += nbytes;

However, the nbytes value is declared as size_t, which on 64bit systems is 64bit,
so the sum never overflows.

The bug was introduced with rmd160.c 1.7 in 2007 where the nbytes type changed
from uint32_t to size_t.


>How-To-Repeat:
Run digest rmd160 on files larger than 4GB and compare results on 32bit and
64bit architectures.

>Fix:
For example:

Index: rmd160.c
===================================================================
RCS file: /cvsroot/pkgsrc/pkgtools/digest/files/rmd160.c,v
retrieving revision 1.8
diff -p -u -r1.8 rmd160.c
--- rmd160.c    3 Jan 2013 10:20:31 -0000       1.8
+++ rmd160.c    23 Aug 2026 07:08:02 -0000
@@ -365,9 +365,10 @@ RMD160Update(RMD160_CTX *context, const 
        _DIAGASSERT(data != NULL);

        /* update length[] */
-       if (context->length[0] + nbytes < context->length[0])
-               context->length[1]++;           /* overflow to msb of length */
+       i = context->length[0];
        context->length[0] += nbytes;
+       if (context->length[0] < i)
+               context->length[1]++;           /* overflow to msb of length */

        ZEROIZE(X, sizeof(X));


>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: pkg-manager->agc
Responsible-Changed-By: hauke@NetBSD.org
Responsible-Changed-When: Wed, 26 Aug 2026 06:51:41 +0000
Responsible-Changed-Why:
Over to maintainer.


>Unformatted:

Home	PR Database Search