NetBSD Problem Report #47894
From www@NetBSD.org Wed Jun 5 12:33:13 2013
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "Postmaster NetBSD.org" (verified OK))
by mollari.NetBSD.org (Postfix) with ESMTPS id 3CA8271B3B
for <gnats-bugs@gnats.NetBSD.org>; Wed, 5 Jun 2013 12:33:13 +0000 (UTC)
Message-Id: <20130605123311.B838071B3C@mollari.NetBSD.org>
Date: Wed, 5 Jun 2013 12:33:11 +0000 (UTC)
From: gergely@egervary.hu
Reply-To: gergely@egervary.hu
To: gnats-bugs@NetBSD.org
Subject: racoon w/NAT-T - pfkey update: wrong ports
X-Send-Pr-Version: www-1.0
>Number: 47894
>Category: bin
>Synopsis: racoon w/NAT-T - pfkey update: wrong ports
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: bin-bug-people
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Wed Jun 05 12:35:00 +0000 2013
>Last-Modified: Tue Jun 24 02:20:01 +0000 2014
>Originator: Gergely Egervary
>Release: NetBSD 6.1-RC2
>Organization:
>Environment:
NetBSD galileo.yyyy.hu 6.1_RC2 NetBSD 6.1_RC2 (GALILEO) #1: Mon May 6 21:45:23 CEST 2013 root@galileo.yyyy.hu:/usr/src/sys/arch/amd64/compile/GALILEO amd64
>Description:
I'm trying to set up an IPSEC VPN connection with PSK. Without NAT, everything works as expected. When client is behind NAT, I see the following log:
Jun 5 13:18:31 galileo racoon: INFO: respond new phase 1 negotiation: server.ip.address[500]<=>client.ip.address[500]
Jun 5 13:18:31 galileo racoon: INFO: begin Identity Protection mode.
Jun 5 13:18:31 galileo racoon: INFO: received broken Microsoft ID: MS NT5 ISAKMPOAKLEY
Jun 5 13:18:31 galileo racoon: INFO: received Vendor ID: RFC 3947
Jun 5 13:18:31 galileo racoon: INFO: received Vendor ID: draft-ietf-ipsec-nat-t-ike-02
Jun 5 13:18:31 galileo racoon: INFO: received Vendor ID: FRAGMENTATION
Jun 5 13:18:31 galileo racoon: [client.ip.address] INFO: Selected NAT-T version: RFC 3947
Jun 5 13:18:31 galileo racoon: [server.ip.address] INFO: Hashing server.ip.address[500] with algo #2
Jun 5 13:18:31 galileo racoon: INFO: NAT-D payload #0 verified
Jun 5 13:18:31 galileo racoon: [client.ip.address] INFO: Hashing client.ip.address[500] with algo #2
Jun 5 13:18:31 galileo racoon: INFO: NAT-D payload #1 doesn't match
Jun 5 13:18:31 galileo racoon: INFO: NAT detected: PEER
Jun 5 13:18:31 galileo racoon: [client.ip.address] INFO: Hashing client.ip.address[500] with algo #2
Jun 5 13:18:31 galileo racoon: [server.ip.address] INFO: Hashing server.ip.address[500] with algo #2
Jun 5 13:18:31 galileo racoon: INFO: Adding remote and local NAT-D payloads.
Jun 5 13:18:31 galileo racoon: INFO: NAT-T: ports changed to: client.ip.address[4500]<->server.ip.address[4500]
Jun 5 13:18:31 galileo racoon: INFO: KA list add: server.ip.address[4500]->client.ip.address[4500]
Jun 5 13:18:31 galileo racoon: INFO: ISAKMP-SA established server.ip.address[4500]-client.ip.address[4500] spi:cbaa8379251aee41:812c9e31092837d4
Jun 5 13:18:31 galileo racoon: INFO: respond new phase 2 negotiation: server.ip.address[4500]<=>client.ip.address[4500]
Jun 5 13:18:31 galileo racoon: INFO: Adjusting my encmode UDP-Transport->Transport
Jun 5 13:18:31 galileo racoon: INFO: Adjusting peer's encmode UDP-Transport(4)->Transport(2)
Jun 5 13:18:31 galileo racoon: INFO: IPsec-SA established: ESP/Transport server.ip.address[500]->client.ip.address[500] spi=153285607(0x922f3e7)
Jun 5 13:18:31 galileo racoon: INFO: IPsec-SA established: ESP/Transport server.ip.address[500]->client.ip.address[500] spi=2696216198(0xa0b4fe86)
See the last two entries - the IPsec-SA session should use udp/4500 instead of udp/500...
log debug;
path pre_shared_key "/etc/racoon/psk.txt";
listen {
isakmp server.ip.address [500];
isakmp_natt server.ip.address [4500];
}
remote anonymous {
exchange_mode main;
nat_traversal on;
generate_policy off;
proposal_check obey;
send_cr off;
send_cert off;
proposal {
encryption_algorithm 3des;
hash_algorithm sha1;
authentication_method pre_shared_key;
dh_group 2;
}
proposal {
encryption_algorithm 3des;
hash_algorithm md5;
authentication_method pre_shared_key;
dh_group 2;
}
proposal {
encryption_algorithm aes;
hash_algorithm sha1;
authentication_method pre_shared_key;
dh_group 2;
}
}
sainfo anonymous {
encryption_algorithm aes, 3des;
authentication_algorithm hmac_sha1, hmac_md5;
compression_algorithm deflate;
}
>How-To-Repeat:
>Fix:
>Audit-Trail:
From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@NetBSD.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc:
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Wed, 5 Jun 2013 09:29:55 -0400
On Jun 5, 12:35pm, gergely@egervary.hu (gergely@egervary.hu) wrote:
-- Subject: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Good debugging. Please see:
PR/47886: Dr. Wolfgang Stukenbrock: IPSEC_NAT_T enabled kernels may access
outdated pointers and pass ESP data to UPD-sockets.
While here, simplify the code and remove the IPSEC_NAT_T option; always
compile nat-traversal in so that it does not bitrot.
To generate a diff of this commit:
cvs rdiff -u -r1.218 -r1.219 src/sys/netinet/ip_output.c
cvs rdiff -u -r1.187 -r1.188 src/sys/netinet/udp_usrreq.c
cvs rdiff -u -r1.10 -r1.11 src/sys/netipsec/files.netipsec
cvs rdiff -u -r1.57 -r1.58 src/sys/netipsec/ipsec.c
cvs rdiff -u -r1.31 -r1.32 src/sys/netipsec/ipsec.h
cvs rdiff -u -r1.29 -r1.30 src/sys/netipsec/ipsec_input.c \
src/sys/netipsec/xform_ipcomp.c
cvs rdiff -u -r1.38 -r1.39 src/sys/netipsec/ipsec_output.c \
src/sys/netipsec/xform_ah.c
cvs rdiff -u -r1.79 -r1.80 src/sys/netipsec/key.c
cvs rdiff -u -r1.11 -r1.12 src/sys/netipsec/key.h
cvs rdiff -u -r1.12 -r1.13 src/sys/netipsec/keydb.h
cvs rdiff -u -r1.41 -r1.42 src/sys/netipsec/xform_esp.c
And the following commit.
christos
From: =?ISO-8859-1?Q?Egerv=E1ry_Gergely?= <gergely@egervary.hu>
To: gnats-bugs@NetBSD.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Cc:
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Sun, 09 Jun 2013 18:26:32 +0200
> Good debugging. Please see:
>
>
> PR/47886: Dr. Wolfgang Stukenbrock: IPSEC_NAT_T enabled kernels may access
> outdated pointers and pass ESP data to UPD-sockets.
> While here, simplify the code and remove the IPSEC_NAT_T option; always
> compile nat-traversal in so that it does not bitrot.
Now I have a NetBSD-6.99.21 sandbox with yesterday's CURRENT.
No difference - still UDP/500...
You can find my racoon debug log here:
http://pastebin.com/sNyxeA4V
Thank you.
--
Egerváry Gergely
From: christos@zoulas.com (Christos Zoulas)
To: =?ISO-8859-1?Q?Egerv=E1ry_Gergely?= <gergely@egervary.hu>,
gnats-bugs@NetBSD.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc:
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Sun, 9 Jun 2013 12:29:53 -0400
On Jun 9, 6:26pm, gergely@egervary.hu (=?ISO-8859-1?Q?Egerv=E1ry_Gergely?=) wrote:
-- Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
| > Good debugging. Please see:
| >
| >
| > PR/47886: Dr. Wolfgang Stukenbrock: IPSEC_NAT_T enabled kernels may access
| > outdated pointers and pass ESP data to UPD-sockets.
| > While here, simplify the code and remove the IPSEC_NAT_T option; always
| > compile nat-traversal in so that it does not bitrot.
|
| Now I have a NetBSD-6.99.21 sandbox with yesterday's CURRENT.
| No difference - still UDP/500...
|
| You can find my racoon debug log here:
| http://pastebin.com/sNyxeA4V
|
Can you send me your configuration so I can replicate your setup?
thanks,
christos
From: =?ISO-8859-2?Q?Egerv=E1ry_Gergely?= <gergely@egervary.hu>
To: gnats-bugs@NetBSD.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Cc:
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Fri, 06 Jun 2014 11:16:27 +0200
Now I'm testing on NetBSD-6.99.43 (Tue Jun 3 23:27:56 CEST 2014) and
still does not work when client is behind NAT, get udp/500 instead of
udp/4500:
Jun 3 23:58:25 sandbox racoon: INFO: respond new phase 1 negotiation:
server.ip.address[500]<=>client.ip.address[500]
Jun 3 23:58:25 sandbox racoon: INFO: begin Identity Protection mode.
Jun 3 23:58:25 sandbox racoon: INFO: received broken Microsoft ID: MS
NT5 ISAKMPOAKLEY
Jun 3 23:58:25 sandbox racoon: INFO: received Vendor ID: RFC 3947
Jun 3 23:58:25 sandbox racoon: INFO: received Vendor ID:
draft-ietf-ipsec-nat-t-ike-02
Jun 3 23:58:25 sandbox racoon: INFO: received Vendor ID: FRAGMENTATION
Jun 3 23:58:25 sandbox racoon: [client.ip.address] INFO: Selected NAT-T
version: RFC 3947
Jun 3 23:58:25 sandbox racoon: ERROR: invalid DH group 20.
Jun 3 23:58:25 sandbox racoon: ERROR: invalid DH group 19.
Jun 3 23:58:25 sandbox racoon: [server.ip.address] INFO: Hashing
server.ip.address[500] with algo #2
Jun 3 23:58:25 sandbox racoon: INFO: NAT-D payload #0 verified
Jun 3 23:58:25 sandbox racoon: [client.ip.address] INFO: Hashing
client.ip.address[500] with algo #2
Jun 3 23:58:25 sandbox racoon: INFO: NAT-D payload #1 doesn't match
Jun 3 23:58:25 sandbox racoon: INFO: NAT detected: PEER
Jun 3 23:58:25 sandbox racoon: [client.ip.address] INFO: Hashing
client.ip.address[500] with algo #2
Jun 3 23:58:25 sandbox racoon: [server.ip.address] INFO: Hashing
server.ip.address[500] with algo #2
Jun 3 23:58:25 sandbox racoon: INFO: Adding remote and local NAT-D
payloads.
Jun 3 23:58:25 sandbox racoon: INFO: NAT-T: ports changed to:
client.ip.address[4500]<->server.ip.address[4500]
Jun 3 23:58:25 sandbox racoon: INFO: KA list add:
server.ip.address[4500]->client.ip.address[4500]
Jun 3 23:58:25 sandbox racoon: INFO: ISAKMP-SA established
server.ip.address[4500]-client.ip.address[4500]
spi:b7055991cbd8c99c:7633ebfe9ba94261
Jun 3 23:58:25 sandbox racoon: INFO: respond new phase 2 negotiation:
server.ip.address[4500]<=>client.ip.address[4500]
Jun 3 23:58:25 sandbox racoon: INFO: Adjusting my encmode
UDP-Transport->Transport
Jun 3 23:58:25 sandbox racoon: INFO: Adjusting peer's encmode
UDP-Transport(4)->Transport(2)
Jun 3 23:58:25 sandbox racoon: INFO: IPsec-SA established:
ESP/Transport server.ip.address[500]->client.ip.address[500]
spi=166530160(0x9ed0c70)
Jun 3 23:58:25 sandbox racoon: INFO: IPsec-SA established:
ESP/Transport server.ip.address[500]->client.ip.address[500]
spi=1453915857(0x56a8fed1)
Any ideas how to fix this issue?
Thank you.
--
Gergely EGERVARY
From: christos@zoulas.com (Christos Zoulas)
To: =?ISO-8859-2?Q?Egerv=E1ry_Gergely?= <gergely@egervary.hu>,
gnats-bugs@NetBSD.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc:
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Fri, 6 Jun 2014 09:01:48 -0400
On Jun 6, 11:16am, gergely@egervary.hu (=?ISO-8859-2?Q?Egerv=E1ry_Gergely?=) wrote:
-- Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
| Now I'm testing on NetBSD-6.99.43 (Tue Jun 3 23:27:56 CEST 2014) and
| still does not work when client is behind NAT, get udp/500 instead of
| udp/4500:
Can you please add IPSEC_DEBUG in your kernel. I really want to fix this
but I don't have a working setup.
christos
From: SUENAGA Hiroki <hsuenaga@openbsd.org>
To: gnats-bugs@NetBSD.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, gergely@egervary.hu
Cc:
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Fri, 13 Jun 2014 21:13:09 +0900
Hi Gergely, thank you for your report.
(2014/06/06 19:35), Egerváry Gergely wrote:
> Now I'm testing on NetBSD-6.99.43 (Tue Jun 3 23:27:56 CEST 2014) and
> still does not work when client is behind NAT, get udp/500 instead of
> udp/4500:
I think your kernel receive the NAT-T port information correctly.
But your kernel doen't respond to racoon with the received NAT-T
information, so the racoon's log is broken.
Please check your SAD using setkey command:
# setkey -D
You can find UDP information like:
10.0.0.1[37905] 10.255.255.1[37905]
esp-udp mode=transport spi=264954840(0x0fcae3d8) reqid=0(0x00000000)
Soooo sorry, there is a byte-order bug... ntohs(37905) = 4500.
I think there are more BUGs. I will continue to debug, please wait.
Hi Christos, here is my patch. please check.
cvs diff: Diffing netipsec
Index: netipsec/key.c
===================================================================
RCS file: /cvs/cvsroot/src/sys/netipsec/key.c,v
retrieving revision 1.90
diff -u -w -p -r1.90 key.c
--- netipsec/key.c 5 Jun 2014 17:18:19 -0000 1.90
+++ netipsec/key.c 13 Jun 2014 11:54:57 -0000
@@ -5612,11 +5612,14 @@ key_getmsgbuf_x1(struct mbuf *m, const s
panic("key_getmsgbuf_x1: NULL pointer is passed");
/* create new sadb_msg to reply. */
- n = key_gather_mbuf(m, mhp, 1, 9, SADB_EXT_RESERVED,
+ n = key_gather_mbuf(m, mhp, 1, 15, SADB_EXT_RESERVED,
SADB_EXT_SA, SADB_X_EXT_SA2,
SADB_EXT_ADDRESS_SRC, SADB_EXT_ADDRESS_DST,
SADB_EXT_LIFETIME_HARD, SADB_EXT_LIFETIME_SOFT,
- SADB_EXT_IDENTITY_SRC, SADB_EXT_IDENTITY_DST);
+ SADB_EXT_IDENTITY_SRC, SADB_EXT_IDENTITY_DST,
+ SADB_X_EXT_NAT_T_TYPE, SADB_X_EXT_NAT_T_SPORT,
+ SADB_X_EXT_NAT_T_DPORT, SADB_X_EXT_NAT_T_OAI,
+ SADB_X_EXT_NAT_T_OAR, SADB_X_EXT_NAT_T_FRAG);
if (!n)
return NULL;
--
SUENAGA Hiroki <hsuenaga@netbsd.org>
facebook.com/hiroki.suenaga
PGP: 66B3 8939 6758 20BA F243 89EC 557A 8CFB ABA9 5E92
From: =?UTF-8?B?RWdlcnbDoXJ5IEdlcmdlbHk=?= <gergely@egervary.hu>
To: gnats-bugs@NetBSD.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Cc:
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Fri, 13 Jun 2014 16:44:32 +0200
Hi Hiroki,
Thank you for your help. Without your sadb_msg patch I get these
messages from racoon on the server (public IP) side:
racoon: INFO: IPsec-SA established: ESP/Transport
193.225.174.14[500]->193.225.174.1[500] spi=142774671(0x882918f)
racoon: INFO: IPsec-SA established: ESP/Transport
193.225.174.14[500]->193.225.174.1[500] spi=156259458(0x9505482)
... and these from racoon on the client (behind NAT) side:
racoon: INFO: IPsec-SA established: ESP/Transport
10.0.0.20[500]->193.225.174.14[500] spi=60514983(0x39b62a7)
racoon: INFO: IPsec-SA established: ESP/Transport
10.0.0.20[500]->193.225.174.14[500] spi=188879077(0xb4210e5)
with your patch, on the server side:
racoon: INFO: IPsec-SA established: ESP/Transport
193.225.174.14[4500]->193.225.174.1[21230] spi=214723282(0xccc6ad2)
racoon: INFO: IPsec-SA established: ESP/Transport
193.225.174.14[4500]->193.225.174.1[21230] spi=17298023(0x107f267)
and on the client side:
racoon: INFO: IPsec-SA established: ESP/Transport
10.0.0.20[4500]->193.225.174.14[4500] spi=17298023(0x107f267)
racoon: INFO: IPsec-SA established: ESP/Transport
10.0.0.20[4500]->193.225.174.14[4500] spi=214723282(0xccc6ad2)
looks a bit better. setkey -D on the server side:
193.225.174.14[4500] 193.225.174.1[21230]
esp-udp mode=transport spi=17298023(0x0107f267) reqid=0(0x00000000)
...
193.225.174.1[21230] 193.225.174.14[4500]
esp-udp mode=transport spi=214723282(0x0ccc6ad2) reqid=0(0x00000000)
and on the client side:
10.0.0.20[4500] 193.225.174.14[4500]
esp-udp mode=transport spi=214723282(0x0ccc6ad2) reqid=0(0x00000000)
193.225.174.14[4500] 10.0.0.20[4500]
esp-udp mode=transport spi=17298023(0x0107f267) reqid=0(0x00000000)
IP reference:
Client internal (NAT) address: 10.0.0.20
NAT box external address: 193.225.174.1
Server external address: 193.115.174.14
btw, I do not see endianness issues here.
--
Egerváry Gergely
<gergely@egervary.hu>
From: SUENAGA Hiroki <hsuenaga@iij.ad.jp>
To: gnats-bugs@NetBSD.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org,
gergely@egervary.hu
Cc: hsuenaga@iij.ad.jp
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Mon, 16 Jun 2014 11:36:19 +0900
Hi Gergely, thank you for your test.
(2014/06/13 23:45), Egerváry Gergely wrote:
> 193.225.174.14[4500] 193.225.174.1[21230]
> esp-udp mode=transport spi=17298023(0x0107f267) reqid=0(0x00000000)
> ...
> 193.225.174.1[21230] 193.225.174.14[4500]
> esp-udp mode=transport spi=214723282(0x0ccc6ad2) reqid=0(0x00000000)
>
> and on the client side:
> 10.0.0.20[4500] 193.225.174.14[4500]
> esp-udp mode=transport spi=214723282(0x0ccc6ad2) reqid=0(0x00000000)
> 193.225.174.14[4500] 10.0.0.20[4500]
> esp-udp mode=transport spi=17298023(0x0107f267) reqid=0(0x00000000)
OK, the SA is correct.
I found a BUG that there was no ESP header in UDP encapsulated ESP packet
on my local environment.
setkey says:
# setkey -D
192.168.187.11[4500] 192.168.187.1[4500]
esp-udp mode=transport spi=262330893(0x0fa2da0d) reqid=0(0x00000000)
E: null 01020304 05060708
seq=0x00000000 replay=4 flags=0x00000000 state=mature
created: Jun 16 11:23:29 2014 current: Jun 16 11:24:27 2014
diff: 58(s) hard: 1402885409(s) soft: 5616830(s)
last: Jun 13 17:12:07 2014 hard: 0(s) soft: 0(s)
current: 0(bytes) hard: 0(bytes) soft: 0(bytes)
allocated: 0 hard: 0 soft: 0
sadb_seq=0 pid=10078 refcnt=1
=> SPI is 0x0fa2da0d.
but tcpdump says:
# tcpdump -n -i wm0 -s 1500 -x -vvvv udp port 4500
tcpdump: listening on wm0, link-type EN10MB (Ethernet), capture size 1500 bytes
11:23:29.569166 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 60)
192.168.187.11.4500 > 192.168.187.1.4500: [udp sum ok] UDP-encap: ESP(spi=0x01020304,seq=0x5060708), length 32
=> SPI and SEQ seem head of payload. It's wrong.
I'm analyzing the problem now.
How about your application?
For your interest, I put my test code on ftp.netbsd.org.
ftp://ftp.netbsd.org/pub/NetBSD/misc/hsuenaga/pfkey_test.tar.gz
The program creates dummy NAT-T SA and send UDP packet. Your application and
SP settings may cause other problems.
> IP reference:
> Client internal (NAT) address: 10.0.0.20
> NAT box external address: 193.225.174.1
> Server external address: 193.115.174.14
>
> btw, I do not see endianness issues here.
Oops, my test code itself had a endianness issue... thank you.
--
Internet Initiative Japan Inc.
Device Engineering Section,
Core Product Development Department,
Product Division,
Technology Unit
SUENAGA Hiroki <hsuenaga@iij.ad.jp>
PGP: 66B3 8939 6758 20BA F243 89EC 557A 8CFB ABA9 5E92
From: SUENAGA Hiroki <hsuenaga@openbsd.org>
To: gnats-bugs@NetBSD.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, gergely@egervary.hu
Cc:
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Mon, 16 Jun 2014 12:11:18 +0900
(2014/06/16 11:36), SUENAGA Hiroki wrote:
> # tcpdump -n -i wm0 -s 1500 -x -vvvv udp port 4500
>
> tcpdump: listening on wm0, link-type EN10MB (Ethernet), capture size 1500 bytes
> 11:23:29.569166 IP (tos 0x0, ttl 64, id 0, offset 0, flags [none], proto UDP (17), length 60)
> 192.168.187.11.4500 > 192.168.187.1.4500: [udp sum ok] UDP-encap: ESP(spi=0x01020304,seq=0x5060708), length 32
oops again, I looked irrelevant pakcets...
UDP encapsulation works fine on my environment.
Let me see your ipsec statistics.
# netstat -s -p ipsec
Thank you,
--
--
SUENAGA Hiroki <hsuenaga@netbsd.org>
facebook.com/hiroki.suenaga
PGP: 66B3 8939 6758 20BA F243 89EC 557A 8CFB ABA9 5E92
From: =?UTF-8?B?RWdlcnbDoXJ5IEdlcmdlbHk=?= <gergely@egervary.hu>
To: gnats-bugs@NetBSD.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Cc: hsuenaga@netbsd.org
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Mon, 16 Jun 2014 15:25:13 +0200
> Let me see your ipsec statistics.
> # netstat -s -p ipsec
On the server:
--->8 ---- cut here ---- 8< ---
(Fast) IPsec:
0 no SA found (output)
0 no memory available (output)
0 no route available (output)
0 generic errors (output)
0 bundled SA processed (output)
469 SPD cache lookups
469 SPD cache misses
IPsec ah:
0 ah input packets processed
0 ah output packets processed
0 ah headers too short
0 ah headers for unsupported address family
0 ah packets with no SA
0 ah packets dropped by crypto returning NULL mbuf
0 ah packets with bad authentication
0 ah packets with no xform
0 ah packets dropped due to queue full
0 ah packets dropped for replay counter wrap
0 ah packets dropped for possible replay
0 ah packets dropped for bad authenticator length
0 ah packets with an invalid SA
0 ah packets too big
0 ah packets blocked due to policy
0 ah failed crypto requests
0 ah tunnel sanity check failures
ah histogram:
ah packets with hmac-sha1: 8
0 ah bytes received
0 ah bytes transmitted
IPsec esp:
4 esp input packets processed
4 esp output packets processed
0 esp headers too short
0 esp headers for unsupported address family
0 esp packets with no SA
0 esp packets dropped by crypto returning NULL mbuf
0 esp packets dropped due to queue full
0 esp packets with no xform
0 esp packets with bad ilen
0 esp packets with bad encryption
0 esp packets with bad authentication
0 esp packets dropped for replay counter wrap
0 esp packets dropped for possible replay
0 esp packets with an invalid SA
0 esp packets too big
0 esp packets blocked due to policy
0 esp failed crypto requests
--->8 ---- cut here ---- 8< ---
on the client:
--->8 ---- cut here ---- 8< ---
(Fast) IPsec:
1 no SA found (output)
0 no memory available (output)
0 no route available (output)
0 generic errors (output)
0 bundled SA processed (output)
294 SPD cache lookups
281 SPD cache misses
IPsec ah:
0 ah input packets processed
0 ah output packets processed
0 ah headers too short
0 ah headers for unsupported address family
0 ah packets with no SA
0 ah packets dropped by crypto returning NULL mbuf
0 ah packets with bad authentication
0 ah packets with no xform
0 ah packets dropped due to queue full
0 ah packets dropped for replay counter wrap
0 ah packets dropped for possible replay
0 ah packets dropped for bad authenticator length
0 ah packets with an invalid SA
0 ah packets too big
0 ah packets blocked due to policy
0 ah failed crypto requests
0 ah tunnel sanity check failures
ah histogram:
ah packets with hmac-sha1: 14
0 ah bytes received
0 ah bytes transmitted
IPsec esp:
7 esp input packets processed
7 esp output packets processed
0 esp headers too short
0 esp headers for unsupported address family
0 esp packets with no SA
0 esp packets dropped by crypto returning NULL mbuf
0 esp packets dropped due to queue full
0 esp packets with no xform
0 esp packets with bad ilen
0 esp packets with bad encryption
0 esp packets with bad authentication
0 esp packets dropped for replay counter wrap
0 esp packets dropped for possible replay
0 esp packets with an invalid SA
0 esp packets too big
0 esp packets blocked due to policy
0 esp failed crypto requests
0 esp tunnel sanity check failures
esp histogram:
esp packets with aes-cbc: 14
224 esp bytes received
308 esp bytes transmitted
IPsec ipip:
0 ipip total input packets
0 ipip total output packets
0 ipip packets too short for header length
0 ipip packets dropped due to queue full
0 ipip packets blocked due to policy
0 ipip IP spoofing attempts
0 ipip protocol family mismatched
0 ipip missing tunnel-endpoint address
0 ipip input bytes received
0 ipip output bytes processed
IPsec ipcomp:
0 ipcomp packets too short for header length
0 ipcomp protocol family not supported
0 ipcomp packets with no SA
0 ipcomp packets dropped by crypto returning NULL mbuf
0 ipcomp queue full
0 ipcomp no support for transform
0 ipcomp packets dropped for replay counter wrap
0 ipcomp input IPcomp packets
0 ipcomp output IPcomp packets
0 ipcomp packets with an invalid SA
0 ipcomp packets decompressed as too big
0 ipcomp packets too short to be compressed
0 ipcomp packet for which compression was useless
0 ipcomp packets blocked due to policy
0 ipcomp failed crypto requests
IPcomp histogram:
0 ipcomp input bytes
0 ipcomp output bytes
--->8 ---- cut here ---- 8< ---
--
Egerváry Gergely
<gergely@egervary.hu>
From: =?UTF-8?B?RWdlcnbDoXJ5IEdlcmdlbHk=?= <gergely@egervary.hu>
To: gnats-bugs@NetBSD.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Cc: hsuenaga@netbsd.org
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Mon, 16 Jun 2014 15:28:15 +0200
>> # netstat -s -p ipsec
>
> On the server:
Ooops here too, nasty `head -n 50' was applied on the server.
Now in full version:
--- >8 ---- cut here ---- 8< ---
(Fast) IPsec:
0 no SA found (output)
0 no memory available (output)
0 no route available (output)
0 generic errors (output)
0 bundled SA processed (output)
522 SPD cache lookups
522 SPD cache misses
IPsec ah:
0 ah input packets processed
0 ah output packets processed
0 ah headers too short
0 ah headers for unsupported address family
0 ah packets with no SA
0 ah packets dropped by crypto returning NULL mbuf
0 ah packets with bad authentication
0 ah packets with no xform
0 ah packets dropped due to queue full
0 ah packets dropped for replay counter wrap
0 ah packets dropped for possible replay
0 ah packets dropped for bad authenticator length
0 ah packets with an invalid SA
0 ah packets too big
0 ah packets blocked due to policy
0 ah failed crypto requests
0 ah tunnel sanity check failures
ah histogram:
ah packets with hmac-sha1: 18
0 ah bytes received
0 ah bytes transmitted
IPsec esp:
9 esp input packets processed
9 esp output packets processed
0 esp headers too short
0 esp headers for unsupported address family
0 esp packets with no SA
0 esp packets dropped by crypto returning NULL mbuf
0 esp packets dropped due to queue full
0 esp packets with no xform
0 esp packets with bad ilen
0 esp packets with bad encryption
0 esp packets with bad authentication
0 esp packets dropped for replay counter wrap
0 esp packets dropped for possible replay
0 esp packets with an invalid SA
0 esp packets too big
0 esp packets blocked due to policy
0 esp failed crypto requests
0 esp tunnel sanity check failures
esp histogram:
esp packets with aes-cbc: 18
432 esp bytes received
180 esp bytes transmitted
IPsec ipip:
0 ipip total input packets
0 ipip total output packets
0 ipip packets too short for header length
0 ipip packets dropped due to queue full
0 ipip packets blocked due to policy
0 ipip IP spoofing attempts
0 ipip protocol family mismatched
0 ipip missing tunnel-endpoint address
0 ipip input bytes received
0 ipip output bytes processed
IPsec ipcomp:
0 ipcomp packets too short for header length
0 ipcomp protocol family not supported
0 ipcomp packets with no SA
0 ipcomp packets dropped by crypto returning NULL mbuf
0 ipcomp queue full
0 ipcomp no support for transform
0 ipcomp packets dropped for replay counter wrap
0 ipcomp input IPcomp packets
0 ipcomp output IPcomp packets
0 ipcomp packets with an invalid SA
0 ipcomp packets decompressed as too big
0 ipcomp packets too short to be compressed
0 ipcomp packet for which compression was useless
0 ipcomp packets blocked due to policy
0 ipcomp failed crypto requests
IPcomp histogram:
0 ipcomp input bytes
0 ipcomp output bytes
--- >8 ---- cut here ---- 8< ---
--
Egerváry Gergely
<gergely@egervary.hu>
From: SUENAGA Hiroki <hsuenaga@iij.ad.jp>
To: gnats-bugs@NetBSD.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org,
gergely@egervary.hu
Cc: hsuenaga@iij.ad.jp
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Fri, 20 Jun 2014 16:24:04 +0900
Hi Gergely,
I found a problem about checksum update of transport mode NAT-T IPsec.
A NAT box can rewrite a address in IP header, but it cannot rewrite a
checksum field in TCP/UDP header. The header is protected by ESP.
So IPsec stack must update the checksum instead of NAT BOX,
but there is no implementation about it.
IKE Protocol has NAT-OA(Origianl Address) payloads to do this. We must
handle the payload and update the checksum. There is some difficulity
around it. There is two NAT-OA payload for initiator side and responder
side. But our kernel doesn't know its side of nagotiation. racoon knows it,
but there is no API to send the side information to kernel.
Tunnel mode IPsec doesn't affected, because inner IP header is not
rewritten by NAT box.
I wrote a simple patch that recalculate TCP/UDP checksum instead of
differential update. This patch is not a goal, but we can confirm that
your problem really comes from checksum calculation.
Please try this patch.
Thank you,
Index: ipsec_input.c
===================================================================
RCS file: /cvsroot/src/sys/netipsec/ipsec_input.c,v
retrieving revision 1.32
diff -u -w -p -r1.32 ipsec_input.c
--- ipsec_input.c 8 Mar 2014 12:18:04 -0000 1.32
+++ ipsec_input.c 20 Jun 2014 07:07:43 -0000
@@ -70,6 +70,8 @@ __KERNEL_RCSID(0, "$NetBSD: ipsec_input.
#include <netinet/ip_var.h>
#include <netinet/in_var.h>
#include <netinet/in_proto.h>
+#include <netinet/udp.h>
+#include <netinet/tcp.h>
#include <netinet/ip6.h>
#ifdef INET6
@@ -118,6 +120,63 @@ do { \
} while (/*CONSTCOND*/0)
/*
+ * fixup TCP/UDP checksum
+ *
+ * XXX: if we have NAT-OA payload from IKE server,
+ * we must do the differential update of checksum.
+ *
+ * XXX: NAT-OAi/NAT-OAr drived from IKE initiator/responder.
+ * how to know the IKE side from kernel?
+ */
+static struct mbuf *
+ipsec4_fixup_checksum(struct mbuf *m)
+{
+ struct ip *ip;
+ struct tcphdr *th;
+ struct udphdr *uh;
+ int poff, off;
+ int plen;
+
+ if (m->m_len < sizeof(*ip))
+ m = m_pullup(m, sizeof(*ip));
+ ip = mtod(m, struct ip *);
+ poff = ip->ip_hl << 2;
+ plen = ntohs(ip->ip_len) - poff;
+
+ switch (ip->ip_p) {
+ case IPPROTO_TCP:
+ IP6_EXTHDR_GET(th, struct tcphdr *, m, poff, sizeof(*th));
+ if (th == NULL)
+ return NULL;
+ off = th->th_off << 2;
+ if (off < sizeof(*th) || off > plen) {
+ m_freem(m);
+ return NULL;
+ }
+ th->th_sum = 0;
+ th->th_sum = in4_cksum(m, IPPROTO_TCP, poff, plen);
+ break;
+ case IPPROTO_UDP:
+ IP6_EXTHDR_GET(uh, struct udphdr *, m, poff, sizeof(*uh));
+ if (uh == NULL)
+ return NULL;
+ off = sizeof(*uh);
+ if (off > plen) {
+ m_freem(m);
+ return NULL;
+ }
+ uh->uh_sum = 0;
+ uh->uh_sum = in4_cksum(m, IPPROTO_UDP, poff, plen);
+ break;
+ default:
+ /* no checksum */
+ return m;
+ }
+
+ return m;
+}
+
+/*
* ipsec_common_input gets called when an IPsec-protected packet
* is received by IPv4 or IPv6. It's job is to find the right SA
# and call the appropriate transform. The transform callback
@@ -324,6 +383,20 @@ ipsec4_common_input_cb(struct mbuf *m, s
ip->ip_len = htons(m->m_pkthdr.len);
prot = ip->ip_p;
+ /* Update TCP/UDP checksum */
+ m = ipsec4_fixup_checksum(m);
+ if (m == NULL) {
+ DPRINTF(("ipsec4_common_input_cb: processing failed "
+ "for SA %s/%08lx\n",
+ ipsec_address(&sav->sah->saidx.dst),
+ (u_long) ntohl(sav->spi)));
+ IPSEC_ISTAT(sproto, ESP_STAT_HDROPS, AH_STAT_HDROPS,
+ IPCOMP_STAT_HDROPS);
+ error = ENOBUFS;
+ goto bad;
+ }
+
+
/* IP-in-IP encapsulation */
if (prot == IPPROTO_IPIP) {
struct ip ipn;
--
Internet Initiative Japan Inc.
Device Engineering Section,
Core Product Development Department,
Product Division,
Technology Unit
SUENAGA Hiroki <hsuenaga@iij.ad.jp>
PGP: 66B3 8939 6758 20BA F243 89EC 557A 8CFB ABA9 5E92
From: =?UTF-8?B?RWdlcnbDoXJ5IEdlcmdlbHk=?= <gergely@egervary.hu>
To: SUENAGA Hiroki <hsuenaga@iij.ad.jp>, gnats-bugs@NetBSD.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc:
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Fri, 20 Jun 2014 10:15:13 +0200
> I found a problem about checksum update of transport mode NAT-T IPsec.
>
> A NAT box can rewrite a address in IP header, but it cannot rewrite a
> checksum field in TCP/UDP header. The header is protected by ESP.
> So IPsec stack must update the checksum instead of NAT BOX,
> but there is no implementation about it.
>
> IKE Protocol has NAT-OA(Origianl Address) payloads to do this. We must
> handle the payload and update the checksum. There is some difficulity
> around it. There is two NAT-OA payload for initiator side and responder
> side. But our kernel doesn't know its side of nagotiation. racoon knows it,
> but there is no API to send the side information to kernel.
Thank you for the patch, I will try it soon.
For your information: for testing only, all the three hosts are now
running NetBSD. (the server, the NAT box, and the client, too) but in
my real-life screnario, only the server runs NetBSD. (I don't know
anything about the client side, my users may use Windows, Linux, etc.
as clients, and SOHO wireless routers as NAT)
I'm using Linux (with racoon installed from Debian GNU/Linux package)
as IPSec/L2TP server for years with great success, all of my Windows,
Linux, etc. users are satisfied.
Now, I want to move the IPsec/L2TP service from my Linux server to my
NetBSD server with the same racoon.conf and settings.
You say:
"kernel doesn't know its side of negotiation. racoon knows it, but
there is no API to send the side information to kernel."
Probably you should look into the Linuxized racoon code (and the Linux
IPSec code, if required) how it is handled there.
Thank you again!
--
Egerváry Gergely
From: =?UTF-8?B?RWdlcnbDoXJ5IEdlcmdlbHk=?= <gergely@egervary.hu>
To: SUENAGA Hiroki <hsuenaga@iij.ad.jp>, gnats-bugs@NetBSD.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc:
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Fri, 20 Jun 2014 10:42:52 +0200
> You say:
> "kernel doesn't know its side of negotiation. racoon knows it, but
> there is no API to send the side information to kernel."
>
> Probably you should look into the Linuxized racoon code (and the Linux
> IPSec code, if required) how it is handled there.
This is the source of the Debianized racoon:
ftp://ftp.debian.org/debian/pool/main/i/ipsec-tools/ipsec-tools_0.8.2.orig.tar.gz
ftp://ftp.hu.debian.org/debian/pool/main/i/ipsec-tools/ipsec-tools_0.8.2-2.debian.tar.xz
It's the same as the NetBSD racoon in ~ 99 percent. The kernel part is
completely different, but the API should be the same.
Hope it helps.
--
Egerváry Gergely
From: SUENAGA Hiroki <hsuenaga@openbsd.org>
To: =?UTF-8?B?RWdlcnbDoXJ5IEdlcmdlbHk=?= <gergely@egervary.hu>,
gnats-bugs@NetBSD.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc:
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Fri, 20 Jun 2014 18:47:14 +0900
2014/06/20 17:42), Egerváry Gergely wrote:
> It's the same as the NetBSD racoon in ~ 99 percent. The kernel part is
> completely different, but the API should be the same.
Thank you for your help.
I will check the Linux kernel and read the IKE specification closely again...
--
SUENAGA Hiroki <hsuenaga@netbsd.org>
facebook.com/hiroki.suenaga
PGP: 66B3 8939 6758 20BA F243 89EC 557A 8CFB ABA9 5E92
From: =?UTF-8?B?RWdlcnbDoXJ5IEdlcmdlbHk=?= <gergely@egervary.hu>
To: SUENAGA Hiroki <hsuenaga@iij.ad.jp>, gnats-bugs@NetBSD.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc:
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Fri, 20 Jun 2014 13:39:38 +0200
> I wrote a simple patch that recalculate TCP/UDP checksum instead of
> differential update. This patch is not a goal, but we can confirm that
> your problem really comes from checksum calculation.
>
> Please try this patch.
YES! I can confirm, this patch makes it work! Brilliant!
--
Egerváry Gergely
<gergely@egervary.hu>
From: christos@zoulas.com (Christos Zoulas)
To: =?UTF-8?B?RWdlcnbDoXJ5IEdlcmdlbHk=?= <gergely@egervary.hu>,
SUENAGA Hiroki <hsuenaga@iij.ad.jp>, gnats-bugs@NetBSD.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc:
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Fri, 20 Jun 2014 12:00:47 -0400
On Jun 20, 10:42am, gergely@egervary.hu (=?UTF-8?B?RWdlcnbDoXJ5IEdlcmdlbHk=?=) wrote:
-- Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
| > You say:
| > "kernel doesn't know its side of negotiation. racoon knows it, but
| > there is no API to send the side information to kernel."
| >
| > Probably you should look into the Linuxized racoon code (and the Linux
| > IPSec code, if required) how it is handled there.
|
| This is the source of the Debianized racoon:
|
| ftp://ftp.debian.org/debian/pool/main/i/ipsec-tools/ipsec-tools_0.8.2.orig.tar.gz
| ftp://ftp.hu.debian.org/debian/pool/main/i/ipsec-tools/ipsec-tools_0.8.2-2.debian.tar.xz
|
| It's the same as the NetBSD racoon in ~ 99 percent. The kernel part is
| completely different, but the API should be the same.
I just looked at the patches (the xz file). There doesn't seem to be anything
interesting there. Except the EDNS stuff, which is done incorrectly (it should
be using the new resolver routines).
christos
From: SUENAGA Hiroki <hsuenaga@openbsd.org>
To: Christos Zoulas <christos@zoulas.com>, =?UTF-8?B?RWdlcnbDoXJ5IEdlcmdl?=
=?UTF-8?B?bHk=?= <gergely@egervary.hu>, gnats-bugs@NetBSD.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc:
Subject: Re: bin/47894: racoon w/NAT-T - pfkey update: wrong ports
Date: Tue, 24 Jun 2014 11:18:07 +0900
(2014/06/21 1:00), Christos Zoulas wrote:
> I just looked at the patches (the xz file). There doesn't seem to be anything
> interesting there. Except the EDNS stuff, which is done incorrectly (it should
> be using the new resolver routines).
I looked linux kernel and found it simply ignores the checksum.
esp4.c
331 /*
332 * 2) ignore UDP/TCP checksums in case
...
338 if (x->props.mode == XFRM_MODE_TRANSPORT)
339 skb->ip_summed = CHECKSUM_UNNECESSARY;
The linux kernel always works without NAT-OAs. This is one decision,
integrity of IPsec packets are guaranteed by ESP-Auth, AES-GCM, or AH
in most case. Some old Internet-Drafts recommend this behavior. But
our racoon supports NAT-OA payload to keep the checksum consistency.
It's a little better to use it correctly.
I found racoon tells only one of NAT-OAi or NAT-OAr to NetBSD kernel.
If it telled both of NAT-OAi and NAT-OAr, we could update the checksum
independently from negotiation side.
newsum = oldsum + ipsrc + ipdst - NAT-OAi - NAT-OAr
I will fix the racoon, and the kernel to do this. If there is no NAT-OA,
linux compatible behavior should be good.
--
SUENAGA Hiroki <hsuenaga@netbsd.org>
facebook.com/hiroki.suenaga
PGP: 66B3 8939 6758 20BA F243 89EC 557A 8CFB ABA9 5E92
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2007
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home