NetBSD Problem Report #46930

From www@NetBSD.org  Sun Sep  9 12:46:38 2012
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	by www.NetBSD.org (Postfix) with ESMTP id 6C08563C785
	for <gnats-bugs@gnats.NetBSD.org>; Sun,  9 Sep 2012 12:46:38 +0000 (UTC)
Message-Id: <20120909124637.843E263C4B1@www.NetBSD.org>
Date: Sun,  9 Sep 2012 12:46:37 +0000 (UTC)
From: gnrp@komkon2.de
Reply-To: gnrp@komkon2.de
To: gnats-bugs@NetBSD.org
Subject: netpgpverify reports right signatures as invalid
X-Send-Pr-Version: www-1.0

>Number:         46930
>Category:       bin
>Synopsis:       netpgpverify reports right signatures as invalid
>Confidential:   no
>Severity:       non-critical
>Priority:       medium
>Responsible:    agc
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Sep 09 12:50:00 +0000 2012
>Closed-Date:    Mon Oct 07 06:38:55 +0000 2013
>Last-Modified:  Mon Oct 07 06:38:55 +0000 2013
>Originator:     Julian Fagir
>Release:        6.0_RC1
>Organization:
>Environment:
NetBSD  6.0_RC1 NetBSD 6.0_RC1 (GENERIC) i386
>Description:
Trying to verify the signature of the hashfile, netpgpverify returns it being invalid, while gpg itself marks it as valid.
>How-To-Repeat:
$ ftp ftp://ftp.netbsd.org/pub/NetBSD/security/PGP/security-officer@netbsd.org.asc
$ gpg --import 'security-officer@netbsd.org.asc'
$ ftp ftp://ftp.netbsd.org/pub/NetBSD/security/hashes/NetBSD-6.0_RC1_hashes.asc
$ netpgpverify --verify NetBSD-6.0_RC1_hashes.asc 
"NetBSD-6.0_RC1_hashes.asc": verification failure: 1 invalid signatures, 0 unknown signatures
$ gpg --verify NetBSD-6.0_RC1_hashes.asc 
gpg: Signature made Thu Aug 23 20:47:50 2012 CEST using RSA key ID 4C4A706E
gpg: Good signature from "NetBSD Security Officer <security-officer@NetBSD.org>"
gpg: WARNING: This key is not certified with a trusted signature!
gpg:          There is no indication that the signature belongs to the owner.
Primary key fingerprint: DDEE 2BDB 9C98 A0D1 D4FB  DBF7 0649 73AC 4C4A 706E
>Fix:

>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: bin-bug-people->agc
Responsible-Changed-By: wiz@NetBSD.org
Responsible-Changed-When: Sun, 09 Sep 2012 15:20:37 +0000
Responsible-Changed-Why:
Over to author.


State-Changed-From-To: open->analyzed
State-Changed-By: agc@NetBSD.org
State-Changed-When: Thu, 04 Oct 2012 20:58:34 +0000
State-Changed-Why:
i analysed the pr - thanks for that!

root cause is that gpg, when it signs an ascii armored signature, does
some unusual things (from some perspective anyway) to the input data.

each line in the input text (except the last one) has the line ending
modified to a DOS (\r\n) one. The last line does not get any terminating
character sequence whatsoever. The digest is then taken over that data,
along with the usual hashed data from the signing key, and the length
trailer. so it's not enough to calculate the data on the input data,
the hashed data and then the trailer, it must be modified in place.

found by using the --debug-all switch to gpg, and analysing the dbgmd files
produced.


From: "Alistair G. Crooks" <agc@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/46930 CVS commit: [agc-netpgp-standalone] src/crypto/external/bsd/netpgp
Date: Sat, 20 Oct 2012 04:59:54 +0000

 Module Name:	src
 Committed By:	agc
 Date:		Sat Oct 20 04:59:54 UTC 2012

 Modified Files:
 	src/crypto/external/bsd/netpgp/bin/netpgp [agc-netpgp-standalone]:
 	    Makefile
 	src/crypto/external/bsd/netpgp/bin/netpgpverify [agc-netpgp-standalone]:
 	    Makefile
 	src/crypto/external/bsd/netpgp/bin/pgp2ssh [agc-netpgp-standalone]:
 	    Makefile
 	src/crypto/external/bsd/netpgp/dist/include [agc-netpgp-standalone]:
 	    netpgp.h
 	src/crypto/external/bsd/netpgp/dist/src/lib [agc-netpgp-standalone]:
 	    validate.c
 	src/crypto/external/bsd/netpgp/dist/src/librsa [agc-netpgp-standalone]:
 	    rsa.c rsa.h
 	src/crypto/external/bsd/netpgp/dist/src/libverify [agc-netpgp-standalone]:
 	    Makefile libnetpgpverify.3 verify.h
 	src/crypto/external/bsd/netpgp/dist/src/netpgpverify [agc-netpgp-standalone]:
 	    netpgpverify.1
 	src/crypto/external/bsd/netpgp/lib [agc-netpgp-standalone]: Makefile
 	src/crypto/external/bsd/netpgp/lib/bn [agc-netpgp-standalone]: Makefile
 	    shlib_version
 	src/crypto/external/bsd/netpgp/lib/cipher [agc-netpgp-standalone]:
 	    shlib_version
 	src/crypto/external/bsd/netpgp/lib/mj [agc-netpgp-standalone]:
 	    shlib_version
 	src/crypto/external/bsd/netpgp/lib/netpgp [agc-netpgp-standalone]:
 	    shlib_version
 	src/crypto/external/bsd/netpgp/lib/paa [agc-netpgp-standalone]:
 	    shlib_version
 	src/crypto/external/bsd/netpgp/lib/rsa [agc-netpgp-standalone]:
 	    shlib_version
 	src/crypto/external/bsd/netpgp/lib/verify [agc-netpgp-standalone]:
 	    Makefile shlib_version
 Added Files:
 	src/crypto/external/bsd/netpgp/dist/src/libverify [agc-netpgp-standalone]:
 	    array.h b64.c b64.h dump.c libverify.c pgpsum.c pgpsum.h
 	src/crypto/external/bsd/netpgp/dist/src/netpgpverify [agc-netpgp-standalone]:
 	    main.c
 Removed Files:
 	src/crypto/external/bsd/netpgp/dist/src/netpgpverify [agc-netpgp-standalone]:
 	    verify.c

 Log Message:
 Replace the netpgpverify command and libnetpgpverify in the
 agc-netpgp-standalone branch with a completely rewritten "from the RFC
 up" version designed to be small, standalone, and easy to maintain.

 	% ldd bin/netpgpverify/netpgpverify
 	bin/netpgpverify/netpgpverify:
 		-lz.1 => /usr/lib/libz.so.1
 		-lgcc_s.1 => /usr/lib/libgcc_s.so.1
 		-lc.12 => /usr/lib/libc.so.12
 		-lbz2.1 => /usr/lib/libbz2.so.1
 		-lnetpgpverify.4 => /usr/lib/libnetpgpverify.so.4
 	% ldd lib/verify/libnetpgpverify.so
 	lib/verify/libnetpgpverify.so:
 		-lc.12 => /usr/lib/libc.so.12
 	% ls -al lib/verify/libnetpgpverify* bin/netpgpverify/netpgpverify
 	-rwxr-xr-x  1 agc  agc   10502 Oct 18 20:59 bin/netpgpverify/netpgpverify
 	-rw-r--r--  1 agc  agc  159720 Oct 18 20:59 lib/verify/libnetpgpverify.a
 	-rw-r--r--  1 agc  agc    4822 Oct 18 20:59 lib/verify/libnetpgpverify.html3
 	lrwxr-xr-x  1 agc  agc      22 Oct 18 20:59 lib/verify/libnetpgpverify.so -> libnetpgpverify.so.4.0
 	lrwxr-xr-x  1 agc  agc      22 Oct 18 20:59 lib/verify/libnetpgpverify.so.4 -> libnetpgpverify.so.4.0
 	-rwxr-xr-x  1 agc  agc  123069 Oct 18 20:59 lib/verify/libnetpgpverify.so.4.0
 	-rw-r--r--  1 agc  agc  169696 Oct 18 20:59 lib/verify/libnetpgpverify_p.a
 	-rw-r--r--  1 agc  agc  149968 Oct 18 20:59 lib/verify/libnetpgpverify_pic.a
 	%

 ("Small" here includes the full BIGNUM/mpi functionality required to
 verify signatures).

 Instead of using extensive callbacks for input data, which have proved
 to be fragile and difficult to maintain, as well as precluding uses
 elsewhere, this uses straight mmaping of input files where possible,
 and falls back to reading if unavailable.

 RFC 4880 makes provision for two types of data to be signed, binary
 data and text, and text is subject to modification of data before the
 signature is made, and is usually opaque.  The new netpgpverify(1) can
 handle this, our old version could not.  DSA signatures are not yet
 supported -- watch this space -- but full RSA ones, including those of
 text documents like the signed NetBSD release hashes (see PR
 bin/46930) are recognised and are included in the regression tests.

 	% env LD_LIBRARY_PATH=../../lib/verify ./netpgpverify < NetBSD-6.0_hashes.asc
 	Good signature for [stdin] made Mon Oct 15 09:28:54 2012
 	signature  4096/RSA (Encrypt or Sign) 064973ac4c4a706e 2009-06-23
 	fingerprint:  ddee 2bdb 9c98 a0d1 d4fb dbf7 0649 73ac 4c4a 706e
 	uid              NetBSD Security Officer <security-officer@NetBSD.org>
 	encryption 4096/RSA (Encrypt or Sign) 9ff2c24fdf2ce620 2009-06-23 [Expiry 2019-06-21]
 	fingerprint:  1915 0801 fbd8 f45d 89f2 0205 9ff2 c24f df2c e620

 	%

 Redirection from stdin is also supported, as are multiple files, and
 detached signatures.  Another interesting use is to verify the
 signatures, and to retrieve the data only if a signature matches -
 this was the old "--cat" command to netpgpverify(1), and it has been
 brought forward into the newer version.

 	% env LD_LIBRARY_PATH=../../lib/verify ./netpgpverify -c cat det.sig | diff det -
 	%

 This is implemented as a library and a small program to call so
 that it is easier to embed verification of signatures in scripting
 languages, or other source code.


 To generate a diff of this commit:
 cvs rdiff -u -r1.1.2.1 -r1.1.2.2 \
     src/crypto/external/bsd/netpgp/bin/netpgp/Makefile
 cvs rdiff -u -r1.1.2.1 -r1.1.2.2 \
     src/crypto/external/bsd/netpgp/bin/netpgpverify/Makefile
 cvs rdiff -u -r1.1.2.1 -r1.1.2.2 \
     src/crypto/external/bsd/netpgp/bin/pgp2ssh/Makefile
 cvs rdiff -u -r1.21 -r1.21.10.1 \
     src/crypto/external/bsd/netpgp/dist/include/netpgp.h
 cvs rdiff -u -r1.44 -r1.44.2.1 \
     src/crypto/external/bsd/netpgp/dist/src/lib/validate.c
 cvs rdiff -u -r1.1.2.1 -r1.1.2.2 \
     src/crypto/external/bsd/netpgp/dist/src/librsa/rsa.c \
     src/crypto/external/bsd/netpgp/dist/src/librsa/rsa.h
 cvs rdiff -u -r1.1.2.1 -r1.1.2.2 \
     src/crypto/external/bsd/netpgp/dist/src/libverify/Makefile \
     src/crypto/external/bsd/netpgp/dist/src/libverify/libnetpgpverify.3 \
     src/crypto/external/bsd/netpgp/dist/src/libverify/verify.h
 cvs rdiff -u -r0 -r1.1.2.1 \
     src/crypto/external/bsd/netpgp/dist/src/libverify/array.h \
     src/crypto/external/bsd/netpgp/dist/src/libverify/b64.c \
     src/crypto/external/bsd/netpgp/dist/src/libverify/b64.h \
     src/crypto/external/bsd/netpgp/dist/src/libverify/dump.c \
     src/crypto/external/bsd/netpgp/dist/src/libverify/libverify.c \
     src/crypto/external/bsd/netpgp/dist/src/libverify/pgpsum.c \
     src/crypto/external/bsd/netpgp/dist/src/libverify/pgpsum.h
 cvs rdiff -u -r0 -r1.1.2.1 \
     src/crypto/external/bsd/netpgp/dist/src/netpgpverify/main.c
 cvs rdiff -u -r1.5 -r1.5.10.1 \
     src/crypto/external/bsd/netpgp/dist/src/netpgpverify/netpgpverify.1
 cvs rdiff -u -r1.15 -r0 \
     src/crypto/external/bsd/netpgp/dist/src/netpgpverify/verify.c
 cvs rdiff -u -r1.13.6.1 -r1.13.6.2 \
     src/crypto/external/bsd/netpgp/lib/Makefile
 cvs rdiff -u -r1.1.2.1 -r1.1.2.2 \
     src/crypto/external/bsd/netpgp/lib/bn/Makefile \
     src/crypto/external/bsd/netpgp/lib/bn/shlib_version
 cvs rdiff -u -r1.1.2.1 -r1.1.2.2 \
     src/crypto/external/bsd/netpgp/lib/cipher/shlib_version
 cvs rdiff -u -r1.1.2.1 -r1.1.2.2 \
     src/crypto/external/bsd/netpgp/lib/mj/shlib_version
 cvs rdiff -u -r1.1.2.1 -r1.1.2.2 \
     src/crypto/external/bsd/netpgp/lib/netpgp/shlib_version
 cvs rdiff -u -r1.1.2.1 -r1.1.2.2 \
     src/crypto/external/bsd/netpgp/lib/paa/shlib_version
 cvs rdiff -u -r1.1.2.1 -r1.1.2.2 \
     src/crypto/external/bsd/netpgp/lib/rsa/shlib_version
 cvs rdiff -u -r1.1.2.1 -r1.1.2.2 \
     src/crypto/external/bsd/netpgp/lib/verify/Makefile \
     src/crypto/external/bsd/netpgp/lib/verify/shlib_version

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: analyzed->feedback
State-Changed-By: agc@NetBSD.org
State-Changed-When: Tue, 20 Nov 2012 05:31:04 +0000
State-Changed-Why:
should be fixed by the merge of netpgpverify(1) and libnetpgpverify(3)
from the agc-netpgp-standalone branch.

please can you try and confirm?

thanks,
alistair


From: "Alistair G. Crooks" <agc@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/46930 CVS commit: src/crypto/external/bsd/netpgp
Date: Tue, 20 Nov 2012 05:26:27 +0000

 Module Name:	src
 Committed By:	agc
 Date:		Tue Nov 20 05:26:26 UTC 2012

 Modified Files:
 	src/crypto/external/bsd/netpgp: Makefile
 Added Files:
 	src/crypto/external/bsd/netpgp/bin: Makefile
 	src/crypto/external/bsd/netpgp/bin/netpgpverify: Makefile
 	src/crypto/external/bsd/netpgp/dist/src/libbn: Makefile bignum.c bn.h
 	    libnetpgpbn.3 misc.c misc.h rand.c rand.h stubs.c stubs.h
 	src/crypto/external/bsd/netpgp/dist/src/libdigest: Makefile digest.c
 	    digest.h tiger.3 tiger.c tiger.h
 	src/crypto/external/bsd/netpgp/dist/src/librsa: Makefile libnetpgprsa.3
 	    rsa.c rsa.h rsastubs.c rsastubs.h
 	src/crypto/external/bsd/netpgp/dist/src/libverify: Makefile array.h
 	    b64.c b64.h dump.c libnetpgpverify.3 libverify.c pgpsum.c pgpsum.h
 	    verify.h
 	src/crypto/external/bsd/netpgp/dist/src/netpgpverify: main.c
 	src/crypto/external/bsd/netpgp/lib/verify: Makefile shlib_version

 Log Message:
 Merge netpgpverify(1) and libnetpgpverify(3) from the
 agc-netpgp-standalone branch.

 Rewrite the netpgpverify(1) functionality from RFC4880 up.  This is a
 completely new implementation, and uses its own bignum library derived
 from libtommath.  Apart from libz and libbz2, it just uses its own
 library and is self-contained - this makes it easier to embed, and to
 use from scripting languages.

 netpgpverify(1) now verifies all the signed files i've thrown at it,
 and the added bonus of using no functionality from libcrypto - all of
 its bignum functionality comes from its own libnetpgpverify.so.
 netpgpverify(1) now verifies not only signatures on binary files, but
 also signatures on text documents.  This fixes PR/46930.  Please don't
 start me on the hoops I had to jump through to calculate the digests
 on text files; trust me, you will regret it.

 % supersize `which netpgpverify`
    text    data     bss     dec     hex filename
    4452     860      72    5384    1508 /usr/bin/netpgpverify
   79542    1408       0   80950   13c36 /usr/lib/libz.so.1
   43994     984     488   45466    b19a /usr/lib/libgcc_s.so.1
 1318116   49644   69272 1437032  15ed68 /usr/lib/libc.so.12
   57253    4184       0   61437    effd /usr/lib/libbz2.so.1
  108726    1712       0  110438   1af66 /usr/lib/libnetpgpverify.so.4
   1612083    58792   69832   1740707   0x1a8fa3   total
 %

 % make t
 env LD_LIBRARY_PATH=/usr/src/crypto/external/bsd/netpgp-standalone/lib/verify ./netpgpverify -c verify b.gpg > output16
 diff expected16 output16
 rm -f output16
 env LD_LIBRARY_PATH=/usr/src/crypto/external/bsd/netpgp-standalone/lib/verify ./netpgpverify -c verify a.gpg > output17
 diff expected17 output17
 rm -f output17
 env LD_LIBRARY_PATH=/usr/src/crypto/external/bsd/netpgp-standalone/lib/verify ./netpgpverify -c verify gpgsigned-a.gpg > output18
 diff expected18 output18
 rm -f output18
 env LD_LIBRARY_PATH=/usr/src/crypto/external/bsd/netpgp-standalone/lib/verify ./netpgpverify -c verify NetBSD-6.0_RC2_hashes.asc > output19
 diff expected19 output19
 rm -f output19
 ...
 env LD_LIBRARY_PATH=/usr/src/crypto/external/bsd/netpgp-standalone/lib/verify ./netpgpverify -k dsa-pubring.gpg in2.asc > output45
 diff expected45 output45
 rm -f output45
 env LD_LIBRARY_PATH=/usr/src/crypto/external/bsd/netpgp-standalone/lib/verify ./netpgpverify -k problem-pubring.gpg NetBSD-6.0_hashes.asc > output46
 diff expected46 output46
 rm -f output46
 cd tests/netpgpverify && make && atf-run
 atf2kyua: I: Removing stale Kyuafiles from /tmp/.XXXXXX.004966aa
 atf2kyua: I: Converting /usr/src/crypto/external/bsd/netpgp-standalone/tests/netpgpverify/Atffile -> /tmp/.XXXXXX.004966aa/Kyuafile
 t_netpgpverify:netpgpverify_rsa  ->  passed  [0.221s]
 t_netpgpverify:netpgpverify_dsa  ->  passed  [0.117s]

 2/2 passed (0 failed)
 Committed action 19
 %


 To generate a diff of this commit:
 cvs rdiff -u -r1.5 -r1.6 src/crypto/external/bsd/netpgp/Makefile
 cvs rdiff -u -r0 -r1.4 src/crypto/external/bsd/netpgp/bin/Makefile
 cvs rdiff -u -r0 -r1.2 \
     src/crypto/external/bsd/netpgp/bin/netpgpverify/Makefile
 cvs rdiff -u -r0 -r1.2 src/crypto/external/bsd/netpgp/dist/src/libbn/Makefile \
     src/crypto/external/bsd/netpgp/dist/src/libbn/bignum.c \
     src/crypto/external/bsd/netpgp/dist/src/libbn/bn.h \
     src/crypto/external/bsd/netpgp/dist/src/libbn/libnetpgpbn.3 \
     src/crypto/external/bsd/netpgp/dist/src/libbn/misc.c \
     src/crypto/external/bsd/netpgp/dist/src/libbn/misc.h \
     src/crypto/external/bsd/netpgp/dist/src/libbn/rand.c \
     src/crypto/external/bsd/netpgp/dist/src/libbn/rand.h \
     src/crypto/external/bsd/netpgp/dist/src/libbn/stubs.c \
     src/crypto/external/bsd/netpgp/dist/src/libbn/stubs.h
 cvs rdiff -u -r0 -r1.2 \
     src/crypto/external/bsd/netpgp/dist/src/libdigest/Makefile \
     src/crypto/external/bsd/netpgp/dist/src/libdigest/digest.c \
     src/crypto/external/bsd/netpgp/dist/src/libdigest/digest.h \
     src/crypto/external/bsd/netpgp/dist/src/libdigest/tiger.3 \
     src/crypto/external/bsd/netpgp/dist/src/libdigest/tiger.c \
     src/crypto/external/bsd/netpgp/dist/src/libdigest/tiger.h
 cvs rdiff -u -r0 -r1.2 \
     src/crypto/external/bsd/netpgp/dist/src/librsa/Makefile \
     src/crypto/external/bsd/netpgp/dist/src/librsa/libnetpgprsa.3 \
     src/crypto/external/bsd/netpgp/dist/src/librsa/rsa.c \
     src/crypto/external/bsd/netpgp/dist/src/librsa/rsa.h \
     src/crypto/external/bsd/netpgp/dist/src/librsa/rsastubs.c \
     src/crypto/external/bsd/netpgp/dist/src/librsa/rsastubs.h
 cvs rdiff -u -r0 -r1.2 \
     src/crypto/external/bsd/netpgp/dist/src/libverify/Makefile \
     src/crypto/external/bsd/netpgp/dist/src/libverify/array.h \
     src/crypto/external/bsd/netpgp/dist/src/libverify/b64.c \
     src/crypto/external/bsd/netpgp/dist/src/libverify/b64.h \
     src/crypto/external/bsd/netpgp/dist/src/libverify/dump.c \
     src/crypto/external/bsd/netpgp/dist/src/libverify/libnetpgpverify.3 \
     src/crypto/external/bsd/netpgp/dist/src/libverify/libverify.c \
     src/crypto/external/bsd/netpgp/dist/src/libverify/pgpsum.c \
     src/crypto/external/bsd/netpgp/dist/src/libverify/pgpsum.h \
     src/crypto/external/bsd/netpgp/dist/src/libverify/verify.h
 cvs rdiff -u -r0 -r1.2 \
     src/crypto/external/bsd/netpgp/dist/src/netpgpverify/main.c
 cvs rdiff -u -r0 -r1.2 src/crypto/external/bsd/netpgp/lib/verify/Makefile \
     src/crypto/external/bsd/netpgp/lib/verify/shlib_version

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: feedback->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Mon, 07 Oct 2013 06:38:55 +0000
State-Changed-Why:
11-month feedback timeout


>Unformatted:

Home	PR Database Search