NetBSD Problem Report #51333

From www@NetBSD.org  Mon Jul 11 12:14:57 2016
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 425057A488
	for <gnats-bugs@gnats.NetBSD.org>; Mon, 11 Jul 2016 12:14:57 +0000 (UTC)
Message-Id: <20160711121456.54FF37AAB6@mollari.NetBSD.org>
Date: Mon, 11 Jul 2016 12:14:56 +0000 (UTC)
From: zafer@netbsd.org
Reply-To: zafer@netbsd.org
To: gnats-bugs@NetBSD.org
Subject: base openssl is 25% slower than pkgsrc openssl
X-Send-Pr-Version: www-1.0

>Number:         51333
>Category:       bin
>Synopsis:       base openssl is 25% slower than pkgsrc openssl
>Confidential:   no
>Severity:       critical
>Priority:       high
>Responsible:    bin-bug-people
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Mon Jul 11 12:15:00 +0000 2016
>Closed-Date:    Sun Nov 10 16:26:48 +0000 2024
>Last-Modified:  Sun Nov 10 16:26:48 +0000 2024
>Originator:     Zafer Aydogan
>Release:        7.0
>Organization:
TNF
>Environment:
amd64
>Description:
While deploying a NetBSD server to a cloud provider, I noticed that
openssl speed showed reduced results with NetBSD compared to SmartOS, FreeBSD or Linux.

Comparing a fresh build openssl from pkgsrc finally showed same results 
like the other OS.

Here is the comparison between base openssl and pkgsrc openssl.

The nominal expected value is 30000k for 16 bytes sha512.

Output shortened for readability:

Base openssl:
OpenSSL 1.0.1p 9 Jul 2015
NetBSD 7.0
gcc version 4.8.4 (NetBSD nb2 20150115)
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes 
sha512           21915.87k


Versus pkgsrc openssl:
OpenSSL 1.0.2h  3 May 2016
built on: reproducible build, date unspecified
The 'numbers' are in 1000s of bytes per second processed.
type             16 bytes
sha512           29093.30k

The difference between pkgsrc and base is 24.67%


>How-To-Repeat:
$ openssl speed sha512



>Fix:
yes, please.

>Release-Note:

>Audit-Trail:
From: Joerg Sonnenberger <joerg@bec.de>
To: gnats-bugs@NetBSD.org
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: bin/51333: base openssl is 25% slower than pkgsrc openssl
Date: Mon, 11 Jul 2016 14:38:47 +0200

 On Mon, Jul 11, 2016 at 12:15:00PM +0000, zafer@netbsd.org wrote:
 > >Description:
 > While deploying a NetBSD server to a cloud provider, I noticed that
 > openssl speed showed reduced results with NetBSD compared to SmartOS, FreeBSD or Linux.

 We don't use OpenSSL's SHA2 implementation as it overlaps with libc.

 Joerg

From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: bin/51333: base openssl is 25% slower than pkgsrc openssl
Date: Mon, 11 Jul 2016 14:20:00 +0000

 On Mon, Jul 11, 2016 at 12:40:01PM +0000, Joerg Sonnenberger wrote:
  >  > While deploying a NetBSD server to a cloud provider, I noticed that
  >  > openssl speed showed reduced results with NetBSD compared to SmartOS, FreeBSD or Linux.
  >  
  >  We don't use OpenSSL's SHA2 implementation as it overlaps with libc.

 so the sha2 in libc is slow?

 -- 
 David A. Holland
 dholland@netbsd.org

From: Joerg Sonnenberger <joerg@bec.de>
To: gnats-bugs@NetBSD.org
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, zafer@netbsd.org
Subject: Re: bin/51333: base openssl is 25% slower than pkgsrc openssl
Date: Mon, 11 Jul 2016 17:29:11 +0200

 On Mon, Jul 11, 2016 at 02:25:01PM +0000, David Holland wrote:
 > The following reply was made to PR bin/51333; it has been noted by GNATS.
 > 
 > From: David Holland <dholland-bugs@netbsd.org>
 > To: gnats-bugs@NetBSD.org
 > Cc: 
 > Subject: Re: bin/51333: base openssl is 25% slower than pkgsrc openssl
 > Date: Mon, 11 Jul 2016 14:20:00 +0000
 > 
 >  On Mon, Jul 11, 2016 at 12:40:01PM +0000, Joerg Sonnenberger wrote:
 >   >  > While deploying a NetBSD server to a cloud provider, I noticed that
 >   >  > openssl speed showed reduced results with NetBSD compared to SmartOS, FreeBSD or Linux.
 >   >  
 >   >  We don't use OpenSSL's SHA2 implementation as it overlaps with libc.
 >  
 >  so the sha2 in libc is slow?

 Well, it's portable C code. Whether the difference justies looking at an
 assembler version is a different question. There might even be potential
 for improving the existing version.

 Joerg

From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/51333 CVS commit: src/crypto/external/bsd/openssl
Date: Tue, 15 Oct 2024 21:04:06 +0000

 Module Name:	src
 Committed By:	riastradh
 Date:		Tue Oct 15 21:04:06 UTC 2024

 Modified Files:
 	src/crypto/external/bsd/openssl/dist/crypto/evp: legacy_sha.c
 	src/crypto/external/bsd/openssl/dist/include/openssl: sha.h
 	src/crypto/external/bsd/openssl/dist/providers/implementations/digests:
 	    sha2_prov.c
 	src/crypto/external/bsd/openssl/lib/libcrypto: crypto.expsym crypto.map
 	    sha.inc
 Removed Files:
 	src/crypto/external/bsd/openssl/lib/libcrypto: libc-sha1.c
 	    libc-sha256.c libc-sha2xx.c libc-sha512.c

 Log Message:
 openssl: Just rename the sha2 symbols.

 Ditch all our other local changes related to them.

 These symbols end up as private symbols in libcrypto, and our
 libcrypto has never exported the sha2 symbols anyway so that can't
 break existing applications.  So this might even be safe to pull up
 to branches.

 This changes some libcrypto symbols listed in crypto.map -- but those
 symbols weren't defined anyway!  And ld apparently doesn't care if
 they're not defined.

 PR bin/51333
 PR lib/58039


 To generate a diff of this commit:
 cvs rdiff -u -r1.2 -r1.3 \
     src/crypto/external/bsd/openssl/dist/crypto/evp/legacy_sha.c
 cvs rdiff -u -r1.3 -r1.4 \
     src/crypto/external/bsd/openssl/dist/include/openssl/sha.h
 cvs rdiff -u -r1.2 -r1.3 \
     src/crypto/external/bsd/openssl/dist/providers/implementations/digests/sha2_prov.c
 cvs rdiff -u -r1.2 -r1.3 \
     src/crypto/external/bsd/openssl/lib/libcrypto/crypto.expsym
 cvs rdiff -u -r1.13 -r1.14 \
     src/crypto/external/bsd/openssl/lib/libcrypto/crypto.map
 cvs rdiff -u -r1.2 -r0 \
     src/crypto/external/bsd/openssl/lib/libcrypto/libc-sha1.c
 cvs rdiff -u -r1.4 -r0 \
     src/crypto/external/bsd/openssl/lib/libcrypto/libc-sha256.c \
     src/crypto/external/bsd/openssl/lib/libcrypto/libc-sha2xx.c
 cvs rdiff -u -r1.5 -r0 \
     src/crypto/external/bsd/openssl/lib/libcrypto/libc-sha512.c
 cvs rdiff -u -r1.8 -r1.9 \
     src/crypto/external/bsd/openssl/lib/libcrypto/sha.inc

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/51333 CVS commit: src/crypto/external/bsd/openssl
Date: Tue, 15 Oct 2024 23:25:21 +0000

 Module Name:	src
 Committed By:	riastradh
 Date:		Tue Oct 15 23:25:21 UTC 2024

 Modified Files:
 	src/crypto/external/bsd/openssl/dist/util: libcrypto.num
 	src/crypto/external/bsd/openssl/lib/libcrypto: crypto.map

 Log Message:
 libcrypto: Record the SHA-2 symbol renaming as a local patch.

 By putting it under dist/, we keep it as a local patch from the
 upstream vendor import, so it won't be lost on next import when
 crypto.map is regenerated.

 Update crypto.map to sort like OpenSSL does while here.

 PR bin/51333
 PR lib/58039


 To generate a diff of this commit:
 cvs rdiff -u -r1.1.1.12 -r1.2 \
     src/crypto/external/bsd/openssl/dist/util/libcrypto.num
 cvs rdiff -u -r1.14 -r1.15 \
     src/crypto/external/bsd/openssl/lib/libcrypto/crypto.map

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->needs-pullups
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Wed, 30 Oct 2024 15:24:06 +0000
State-Changed-Why:
fixed in HEAD, maybe worth pullup-10, probably not worth pullup-9

Change strikes me as a bit risky, but if we pull up the bsd.lib.mk
EXPSYM changes to verify the exact list of symbols defined in
libcrypto.so, that might alleviate the risks.


State-Changed-From-To: needs-pullups->closed
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Sun, 10 Nov 2024 16:26:48 +0000
State-Changed-Why:
let's just leave this to 11


>Unformatted:

Home	PR Database Search