NetBSD Problem Report #46878

From ryo_on@yk.rim.or.jp  Thu Aug 30 16:03:11 2012
Return-Path: <ryo_on@yk.rim.or.jp>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
	by www.NetBSD.org (Postfix) with ESMTP id 4547A63B86D
	for <gnats-bugs@gnats.netbsd.org>; Thu, 30 Aug 2012 16:03:11 +0000 (UTC)
Message-Id: <20120830160311.4547A63B86D@www.NetBSD.org>
Date: Thu, 30 Aug 2012 16:03:11 +0000 (UTC)
From: ryoon@NetBSD.org
Reply-To: ryoon@NetBSD.org
To: gnats-bugs@gnats.NetBSD.org
Subject: connection to some https site using openssl causes freeze
X-Send-Pr-Version: 3.95

>Number:         46878
>Category:       lib
>Synopsis:       connection to some https site using openssl causes freeze
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    lib-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Aug 30 16:05:01 +0000 2012
>Last-Modified:  Wed Oct 03 13:35:01 +0000 2012
>Originator:     Ryo ONODERA
>Release:        NetBSD 6.99.10
>Organization:

>Environment:


System: NetBSD hydrogen.elements.tetera.org 6.99.10 NetBSD 6.99.10 (GENERIC) #3: Thu Aug 30 21:58:31 JST 2012 root@hydrogen.elements.tetera.org:/usr/obj/sys/arch/i386/compile/GENERIC i386
Architecture: i386
Machine: i386
>Description:
% openssl s_client -connect secure.nicovideo.jp:443
WARNING: can't open config file: /etc/openssl/openssl.cnf
CONNECTED(00000006)
(freeze)

I had reported to current-users@ about one year ago.
http://mail-index.netbsd.org/current-users/2011/07/28/msg017182.html

OpenSSL 1.0.1c resolves some connection to https sites,
but I cannot connect to, for example, secure.nicovide.jp properly.
This problem prevents net/nicovideo-dl use on NetBSD current and 6.0.

Original problem is mixture of some problems.
If I can analyse it correctly, the problem of secure.nicovideo.jp
 is caused from TLS 1.2 isdefault trial of openssl 1.0.1c
(thank you, Matthias Drochner).

With the patch in Fix: section, nicovideo-dl works well,
and connection to some erroneous sites with www/w3m also works well,
for example, owa.mit.edu.


>How-To-Repeat:

Run the following command on NetBSD current of today, or 6.0_RC1.
% openssl s_client -connect secure.nicovideo.jp:443

>Fix:

I have no idea about correct place for definition of OPENSSL_NO_TLS1_2_CLIENT.
The following patch works well.

Index: Makefile.openssl
===================================================================
RCS file: /cvsroot/src/crypto/Makefile.openssl,v
retrieving revision 1.10
diff -u -r1.10 Makefile.openssl
--- Makefile.openssl	23 Sep 2009 04:02:28 -0000	1.10
+++ Makefile.openssl	30 Aug 2012 15:45:34 -0000
@@ -9,5 +9,6 @@
 CPPFLAGS+=	-DOPENSSLDIR=\"/etc/openssl\"
 CPPFLAGS+=	-DENGINESDIR=\"/usr/lib/openssl\"
 CPPFLAGS+=	-DDSO_DLFCN -DHAVE_DLFCN_H
+CPPFLAGS+=	-DOPENSSL_NO_TLS1_2_CLIENT

 .endif


>Audit-Trail:
From: Ryo ONODERA <ryo_on@yk.rim.or.jp>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: lib/46878: connection to some https site using openssl causes
 freeze
Date: Tue, 02 Oct 2012 00:43:06 +0900 (JST)

 Hi,

 I have found that similar patch is used in Ubuntu Linux.

 Is this patch unacceptable?

 --
 Ryo ONODERA // ryo_on@yk.rim.or.jp
 PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3


From: Bernd Ernesti <veego@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: lib/46878: connection to some https site using openssl causes
 freeze
Date: Mon, 1 Oct 2012 21:29:03 +0200

 On Mon, Oct 01, 2012 at 03:45:01PM +0000, Ryo ONODERA wrote:
 >  Hi,
 >  
 >  I have found that similar patch is used in Ubuntu Linux.
 >  
 >  Is this patch unacceptable?

 For me it would be unacceptable because it disables TLS 1.2.

 Is there no option for the openssl s_client to disable TLS 1.2?

 If not then this should be reported to the openssl developers
 so they can add an option to disable that with an command
 line option.

 Bernd

From: Ryo ONODERA <ryo_on@yk.rim.or.jp>
To: gnats-bugs@NetBSD.org, veego@NetBSD.org
Cc: 
Subject: Re: lib/46878: connection to some https site using openssl causes
 freeze
Date: Tue, 02 Oct 2012 12:09:03 +0900 (JST)

 From: Bernd Ernesti <veego@NetBSD.org>, Date: Mon,  1 Oct 2012 19:35:04 +0000 (UTC)

 > The following reply was made to PR lib/46878; it has been noted by GNATS.
 > 
 > From: Bernd Ernesti <veego@NetBSD.org>
 > To: gnats-bugs@NetBSD.org
 > Cc: 
 > Subject: Re: lib/46878: connection to some https site using openssl causes
 >  freeze
 > Date: Mon, 1 Oct 2012 21:29:03 +0200
 > 
 >  On Mon, Oct 01, 2012 at 03:45:01PM +0000, Ryo ONODERA wrote:
 >  >  Hi,
 >  >  
 >  >  I have found that similar patch is used in Ubuntu Linux.
 >  >  
 >  >  Is this patch unacceptable?
 >  
 >  For me it would be unacceptable because it disables TLS 1.2.
 >  
 >  Is there no option for the openssl s_client to disable TLS 1.2?
 >  
 >  If not then this should be reported to the openssl developers
 >  so they can add an option to disable that with an command
 >  line option.

 openssl s_client is one example.
 At least, pkgsrc/www/w3m and net/nicovideo-dl are also affected.
 On NetBSD current and 6.0_RC2, pkgsrc/net/nicovideo-dl does not
 work at all, because it cannot connect to secure.nicovideo.jp .

 --
 Ryo ONODERA // ryo_on@yk.rim.or.jp
 PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3


From: Thomas Klausner <wiz@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: lib/46878: connection to some https site using openssl causes
 freeze
Date: Tue, 2 Oct 2012 09:30:29 +0200

 On Tue, Oct 02, 2012 at 03:10:07AM +0000, Ryo ONODERA wrote:
 >  On NetBSD current and 6.0_RC2, pkgsrc/net/nicovideo-dl does not
 >  work at all, because it cannot connect to secure.nicovideo.jp .

 This reminds me of a problem I had with bitbucket. bitbucket fixed a
 similar problem, once convinced it was a problem on their side.

 https://bitbucket.org/site/master/issue/2552/problem-checking-out-with-tlsv11

 If it's the same kind of issue, perhaps you can get nicovideo to fix
 it on the server side.

 I'm not saying that the situation shouldn't be improved on NetBSD's side as well.
  Thomas

From: Ryo ONODERA <ryo_on@yk.rim.or.jp>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: lib/46878: connection to some https site using openssl causes
 freeze
Date: Tue, 02 Oct 2012 19:36:53 +0900 (JST)

 From: Thomas Klausner <wiz@NetBSD.org>, Date: Tue,  2 Oct 2012 07:35:03 +0000 (UTC)

 > The following reply was made to PR lib/46878; it has been noted by GNATS.
 > 
 > From: Thomas Klausner <wiz@NetBSD.org>
 > To: gnats-bugs@NetBSD.org
 > Cc: 
 > Subject: Re: lib/46878: connection to some https site using openssl causes
 >  freeze
 > Date: Tue, 2 Oct 2012 09:30:29 +0200
 > 
 >  On Tue, Oct 02, 2012 at 03:10:07AM +0000, Ryo ONODERA wrote:
 >  >  On NetBSD current and 6.0_RC2, pkgsrc/net/nicovideo-dl does not
 >  >  work at all, because it cannot connect to secure.nicovideo.jp .
 >  
 >  This reminds me of a problem I had with bitbucket. bitbucket fixed a
 >  similar problem, once convinced it was a problem on their side.
 >  
 >  https://bitbucket.org/site/master/issue/2552/problem-checking-out-with-tlsv11
 >  
 >  If it's the same kind of issue, perhaps you can get nicovideo to fix
 >  it on the server side.
 >  
 >  I'm not saying that the situation shouldn't be improved on NetBSD's side as well.

 Thanks for your information.
 I am not sure it is same problem or not... I will investigate deeper.

 By the way,
 In my opinion, secure.nicovideo.jp, www.netbk.co.jp, and pc.sbisonpo.co.jp
 will not change server side settings, because NetBSD, w3m, or nicovideo-dl
 are not supported officially for them.

 nicovideo.jp is youtube like video sharing site in Japan.
 It broadcasts some official videos, and it will dislike
 download program like nicovideo-dl.

 www.netbk.co.jp and pc.sbisonpo.cp.jp are bank and insurance company.
 They require supported web browser, Microsoft Internet Explorer.
 At least spisonpo.co.jp does not allow Firefox, it allow only
 Internet Explorer.
 If I contact to netbk.co.jp or sbisonpo.co.jp, they will say
 "use supported web browser" only.

 In the end, NetBSD desktop users cannot use these sites with OpenSSL
 based https, and NetBSD desktop becomes less useful for usual users.

 --
 Ryo ONODERA // ryo_on@yk.rim.or.jp
 PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3


From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
To: gnats-bugs@NetBSD.org
Cc: tsutsui@ceres.dti.ne.jp
Subject: Re: lib/46878: connection to some https site using openssl causesfreeze
Date: Wed, 3 Oct 2012 00:18:23 +0900

 ryoon@ wrote:

 >  In the end, NetBSD desktop users cannot use these sites with OpenSSL
 >  based https, and NetBSD desktop becomes less useful for usual users.

 Unfortunately, most typical NetBSD users and developers have
 less interests in marketing and desktop environments ;-p
 You should rather mention technical merit on it.

 If you'd like to integrate a patch to solve your problem,
 you should mention at least:

  - what's the actual problem
   => SSL access hangs on *some* specific sites?

  - what's the route cause
   => you didn't track it, right?

  - how does the proposed patch fix (or work around) the problem
   => I guess your patch is a "kludge", not a real fix
      (even if ubuntu has accepted it)

  - if the patch is not a "real fix", what's the possible bad side effects
   => appearently network embeded users rather want proper TLS 1.2 support
      rather than nicovideo support (IIRC gnash doesn't support it either)

 On the other hand, openssl changelogs say:

 http://www.openssl.org/news/changelog.html
 ---
 Changes between 1.0.1 and 1.0.1a [19 Apr 2012]
 	:
   *) Workarounds for some broken servers that "hang" if a client hello
      record length exceeds 255 bytes:

      1. Do not use record version number > TLS 1.0 in initial client
         hello: some (but not all) hanging servers will now work.
      2. If we set OPENSSL_MAX_TLS1_2_CIPHER_LENGTH this will truncate
         the number of ciphers sent in the client hello. This should be
         set to an even number, such as 50, for example by passing:
         -DOPENSSL_MAX_TLS1_2_CIPHER_LENGTH=50 to config or Configure.
         Most broken servers should now work.
      3. If all else fails setting OPENSSL_NO_TLS1_2_CLIENT will disable
         TLS 1.2 client support entirely.
 ---

 - the problem was ack'ed by openssl guys

 - apearently the problem is at server side

 - "-DOPENSSL_MAX_TLS1_2_CIPHER_LENGTH=XX" could be another compromise

 And 1.0.1d (newer than our 1.0.1c) includes the following entry:
 ---
  Changes between 1.0.1c and 1.0.1d [xx XXX xxxx]
 	:
   *) Don't use TLS 1.0 record version number in initial client hello
      if renegotiating.
 ---

 Could you try this renegotiation change?
 (I have not checked actuall changes though)

 ---
 Izumi Tsutsui

From: Ryo ONODERA <ryo_on@yk.rim.or.jp>
To: gnats-bugs@NetBSD.org, tsutsui@ceres.dti.ne.jp
Cc: 
Subject: Re: lib/46878: connection to some https site using openssl
 causesfreeze
Date: Wed, 03 Oct 2012 01:16:16 +0900 (JST)

 Hi,

 From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>, Date: Tue,  2 Oct 2012 15:20:03 +0000 (UTC)

 > The following reply was made to PR lib/46878; it has been noted by GNATS.
 > 
 > From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
 > To: gnats-bugs@NetBSD.org
 > Cc: tsutsui@ceres.dti.ne.jp
 > Subject: Re: lib/46878: connection to some https site using openssl causesfreeze
 > Date: Wed, 3 Oct 2012 00:18:23 +0900
 > 
 >  ryoon@ wrote:
 >  
 >  >  In the end, NetBSD desktop users cannot use these sites with OpenSSL
 >  >  based https, and NetBSD desktop becomes less useful for usual users.
 >  
 >  Unfortunately, most typical NetBSD users and developers have
 >  less interests in marketing and desktop environments ;-p
 >  You should rather mention technical merit on it.
 >  
 >  If you'd like to integrate a patch to solve your problem,
 >  you should mention at least:
 >  
 >   - what's the actual problem
 >    => SSL access hangs on *some* specific sites?
 >  
 >   - what's the route cause
 >    => you didn't track it, right?
 >  
 >   - how does the proposed patch fix (or work around) the problem
 >    => I guess your patch is a "kludge", not a real fix
 >       (even if ubuntu has accepted it)
 >  
 >   - if the patch is not a "real fix", what's the possible bad side effects
 >    => appearently network embeded users rather want proper TLS 1.2 support
 >       rather than nicovideo support (IIRC gnash doesn't support it either)

 Thank you.
 I will also utilize this list in other PR.

 >  On the other hand, openssl changelogs say:
 >  
 >  http://www.openssl.org/news/changelog.html
 >  ---
 >  Changes between 1.0.1 and 1.0.1a [19 Apr 2012]
 >  	:
 >    *) Workarounds for some broken servers that "hang" if a client hello
 >       record length exceeds 255 bytes:
 >   
 >       1. Do not use record version number > TLS 1.0 in initial client
 >          hello: some (but not all) hanging servers will now work.
 >       2. If we set OPENSSL_MAX_TLS1_2_CIPHER_LENGTH this will truncate
 >          the number of ciphers sent in the client hello. This should be
 >          set to an even number, such as 50, for example by passing:
 >          -DOPENSSL_MAX_TLS1_2_CIPHER_LENGTH=50 to config or Configure.
 >          Most broken servers should now work.
 >       3. If all else fails setting OPENSSL_NO_TLS1_2_CLIENT will disable
 >          TLS 1.2 client support entirely.
 >  ---
 >  
 >  - the problem was ack'ed by openssl guys
 >  
 >  - apearently the problem is at server side
 >  
 >  - "-DOPENSSL_MAX_TLS1_2_CIPHER_LENGTH=XX" could be another compromise
 >  
 >  And 1.0.1d (newer than our 1.0.1c) includes the following entry:
 >  ---
 >   Changes between 1.0.1c and 1.0.1d [xx XXX xxxx]
 >  	:
 >    *) Don't use TLS 1.0 record version number in initial client hello
 >       if renegotiating.
 >  ---
 >  
 >  Could you try this renegotiation change?
 >  (I have not checked actuall changes though)

 I will try to use OpenSSL 1.0.1d.

 Thank you, again.

 --
 Ryo ONODERA // ryo_on@yk.rim.or.jp
 PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3


From: Ryo ONODERA <ryo_on@yk.rim.or.jp>
To: gnats-bugs@NetBSD.org, tsutsui@ceres.dti.ne.jp
Cc: 
Subject: Re: lib/46878: connection to some https site using openssl
 causesfreeze
Date: Wed, 03 Oct 2012 04:18:49 +0900 (JST)

 From: Ryo ONODERA <ryo_on@yk.rim.or.jp>, Date: Wed, 03 Oct 2012 01:16:16 +0900 (JST)

 >>  And 1.0.1d (newer than our 1.0.1c) includes the following entry:
 >>  ---
 >>   Changes between 1.0.1c and 1.0.1d [xx XXX xxxx]
 >>  	:
 >>    *) Don't use TLS 1.0 record version number in initial client hello
 >>       if renegotiating.
 >>  ---
 >>  
 >>  Could you try this renegotiation change?
 >>  (I have not checked actuall changes though)
 > 
 > I will try to use OpenSSL 1.0.1d.
 > 
 > Thank you, again.

 I have tested with openssl-1.0.1-stable-SNAP-20121002.tar.gz
 (OpenSSL 1.0.1d-dev).
 And I cannot connect to the servers.

 --
 Ryo ONODERA // ryo_on@yk.rim.or.jp
 PGP fingerprint = 82A2 DC91 76E0 A10A 8ABB  FD1B F404 27FA C7D1 15F3


From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
To: gnats-bugs@NetBSD.org
Cc: tsutsui@ceres.dti.ne.jp
Subject: Re: lib/46878: connection to some https site using opensslcausesfreeze
Date: Wed, 3 Oct 2012 21:30:15 +0900

 ryoon@ wrote:

 > I have tested with openssl-1.0.1-stable-SNAP-20121002.tar.gz
 > (OpenSSL 1.0.1d-dev).
 > And I cannot connect to the servers.

 Actually the renegotiation fix in 1.0.1d is unrelated, i.e.
 the following fix doesn't solve the "server hang" problem at all:
 http://rt.openssl.org/Ticket/Display.html?id=2811&user=guest&pass=guest
 http://cvs.openssl.org/chngview?cn=22565
 I'm afraid it means the server side problem can't be resolved
 by client side.

 On the other hand, using -DOPENSSL_MAX_TLS1_2_CIPHER_LENGTH=50
 on libssl build works around, but it seems several people still
 claim "it isn't a right fix."

 It looks Wine had the same problem and they fixed it
 by disabling TLS 1.2 in their applications per Windows settings:
 http://bugs.winehq.org/show_bug.cgi?id=30598
 http://source.winehq.org/patches/data/89343

 Then, I think at least site specific applications like nicovideo_dl
 should be fixed by disabling TLS 1.2 in it as well.

 It would also be worth to make openssl have configurable
 settings for equivalents of OPENSSL_MAX_TLS1_2_CIPHER_LENGTH and
 OPENSSL_NO_TLS1_2_CLIENT in openssl.cnf for long term workaround,
 rather than compile time settings.

 ---
 Izumi Tsutsui

From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: lib/46878: connection to some https site using opensslcausesfreeze
Date: Wed, 3 Oct 2012 14:52:27 +0200

 This sounds like an upstream interoperability problem - has it been
 reported there?

 Martin

From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
To: martin@duskware.de, gnats-bugs@NetBSD.org
Cc: tsutsui@ceres.dti.ne.jp
Subject: Re: lib/46878: connection to some https site using opensslcausesfreeze
Date: Wed, 3 Oct 2012 22:21:50 +0900

 >  This sounds like an upstream interoperability problem - has it been
 >  reported there?

 The actual problem is "broken servers" that can't handle
 TLS 1.2 properly and openssl guys know the issue:
 http://nxr.NetBSD.org/xref/src/crypto/external/bsd/openssl/dist/CHANGES#60
 but I'm not sure if they think it's worth to make the workarounds
 runtime settings. (someone may just claim "fix broken ones.")

 ---
 Izumi Tsutsui

From: Martin Husemann <martin@duskware.de>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: lib/46878: connection to some https site using opensslcausesfreeze
Date: Wed, 3 Oct 2012 15:32:41 +0200

 I see - sounds like having two environment variables that give the max
 length and disable tls 1.2 completely respectively would be a good idea, no
 matter wether upstream buys this back.

 Martin

>Unformatted:

Home	PR Database Search