NetBSD Problem Report #59460

From el@assimilation2.clifford.lol  Fri Jun  6 21:44:29 2025
Return-Path: <el@assimilation2.clifford.lol>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id CC29F1A923A
	for <gnats-bugs@gnats.NetBSD.org>; Fri,  6 Jun 2025 21:44:29 +0000 (UTC)
Message-Id: <20250606214427.42E1A66F12@assimilation2.clifford.lol>
Date: Fri,  6 Jun 2025 22:44:27 +0100 (BST)
From: Eleanor Clifford <el@assimilation2.clifford.lol>
Reply-To: Eleanor Clifford <el-ops-netbsd@clifford.lol>
To: gnats-bugs@NetBSD.org
Subject: Xen netback sends data debian domU considers malicious
X-Send-Pr-Version: 3.95

>Number:         59460
>Category:       port-xen
>Synopsis:       Xen netback sends data debian domU considers malicious
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    port-xen-maintainer
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Fri Jun 06 21:45:00 +0000 2025
>Last-Modified:  Sat Jun 07 09:30:01 +0000 2025
>Originator:     Eleanor Clifford
>Release:        NetBSD 10.0
>Environment:
	System: NetBSD assimilation2.clifford.lol 10.0 NetBSD 10.0 (XEN3_DOM0) #0: Thu Mar 28 08:33:33 UTC 2024 mkrepro@mkrepro.NetBSD.org:/usr/src/sys/arch/xen/compile/XEN3_DOM0 amd64
	Architecture: x86_64
	Machine: amd64
	domU: 6.10.11-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.10.11-1 (2024-09-22) x86_64 GNU/Linux

>Description:
	I have a system with NetBSD 10.0 as the dom0, and a number of domUs
	including debian PVH (6.10.11-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.10.11-1 (2024-09-22) x86_64 GNU/Linux).
	Each time the debian domU is started, networking eventually fails, usually
	between a few hours and a few days after starting.

	I haven't observed similar problems on NetBSD or FreeBSD PVH domUs (but I
	assume that is because they aren't doing the same security checks). I have
	also observed issues with xen networking on an OpenBSD HVM domU, but I'm
	uncertain if that's related and it's less of an issue for me right now.

	Here is the debian domU config:

	name = "mondas"
	type="pvh"
	memory = 6144
	vcpus = 4
	kernel = "/domus/mondas/vmlinuz"
	ramdisk = "/domus/mondas/initrd.img"
	extra = "root=/dev/xvda1 init=/lib/systemd/systemd"
	vif = ['bridge=bridge0']
	disk = ['phy:/dev/root/mondas,xvda1,w'] # LVM Logical Volume

	Here are two examples of the logs from debian when networking fails:

	$ sudo journalctl -b -1 | grep 'kernel.*net'

	Jun 06 11:44:31 mondas kernel: audit: initializing netlink subsys (disabled)
	Jun 06 11:44:31 mondas kernel: xen_netfront: Initialising Xen virtual ethernet driver
	Jun 06 11:44:37 mondas kernel: bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
	Jun 06 11:44:37 mondas kernel: Initializing XFRM netlink socket
	Jun 06 15:42:22 mondas kernel: net enX0: Invalid extra type: 247
	Jun 06 15:42:22 mondas kernel: net enX0: Invalid extra type: 251
	Jun 06 15:42:22 mondas kernel: net enX0: Invalid extra type: 253
	Jun 06 15:42:22 mondas kernel: net enX0: Invalid extra type: 0
	Jun 06 15:42:22 mondas kernel: net enX0: Missing extra info
	Jun 06 15:44:36 mondas kernel: net enX0: Illegal number of responses 295

	$ sudo journalctl -b -5 | grep 'kernel.*net'

	Jun 04 11:04:28 mondas kernel: audit: initializing netlink subsys (disabled)
	Jun 04 11:04:29 mondas kernel: xen_netfront: Initialising Xen virtual ethernet driver
	Jun 04 11:04:34 mondas kernel: bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
	Jun 04 11:04:34 mondas kernel: Initializing XFRM netlink socket
	Jun 04 14:03:10 mondas kernel: net enX0: Invalid extra type: 38
	Jun 04 14:03:10 mondas kernel: net enX0: Invalid extra type: 43
	Jun 04 14:03:10 mondas kernel: net enX0: Invalid extra type: 47
	Jun 05 17:02:24 mondas kernel: net enX0: Missing extra info
	Jun 05 17:02:24 mondas kernel: net enX0: Need more slots
	Jun 05 17:03:26 mondas kernel: net enX0: Illegal number of responses 263

	When "Illegal number of responses" is hit, networking in debian stops
	working completely, until the domU is entirely restarted (I haven't been
	successful trying to only restart networking).

	I think the patch that adds this error to the linux kernel is this:

	https://patchwork.kernel.org/project/netdevbpf/patch/20210513100302.22027-8-jgross@suse.com/#24182397

	As far as I can tell, it is there to protect against malicious backends,
	and when the "Illegal number of responses" error is hit, the network
	interface is marked as broken and prevented from working.

	I am not familiar enough with the NetBSD kernel (or Xen, or the linux
	kernel) to know exactly what is going wrong, but I suppose their might be
	something wrong in sys/arch/xen/xen/xennetback_xenbus.c, perhaps
	xennetback_tx_response or xennetback_tx_check_packet?

	Of course it's also possible it's actually the Linux kernel's fault and I'm
	sending this to the wrong bug tracker...

>How-To-Repeat:
	1. Set up NetBSD 10 as dom0
	2. Set up debian domU as PVH with bridged network (as above)
	3. Wait
>Fix:
	workarounds:
	* restart domU each time
	* patch the domU kernel to remove the security check, I suppose

>Audit-Trail:
From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: port-xen/59460: Xen netback sends data debian domU considers malicious
Date: Sat, 7 Jun 2025 09:26:53 -0000 (UTC)

 el@assimilation2.clifford.lol (Eleanor Clifford) writes:

 >	Jun 06 11:44:31 mondas kernel: xen_netfront: Initialising Xen virtual ethernet driver
 >	Jun 06 11:44:37 mondas kernel: bridge: filtering via arp/ip/ip6tables is no longer available by default. Update your scripts to load br_netfilter if you need this.
 >	Jun 06 11:44:37 mondas kernel: Initializing XFRM netlink socket
 >	Jun 06 15:42:22 mondas kernel: net enX0: Invalid extra type: 247
 >	Jun 06 15:42:22 mondas kernel: net enX0: Invalid extra type: 251
 >	Jun 06 15:42:22 mondas kernel: net enX0: Invalid extra type: 253
 >	Jun 06 15:42:22 mondas kernel: net enX0: Invalid extra type: 0
 >	Jun 06 15:42:22 mondas kernel: net enX0: Missing extra info
 >	Jun 06 15:44:36 mondas kernel: net enX0: Illegal number of responses 295


 As a data point, I don't see anything like that when running
 a debian 12 guest with a NetBSD-current XEN3_DOM0.

 root@debian:~# journalctl -b -1 | grep 'kernel.*net'
 Jun 07 04:15:51 debian kernel: audit: initializing netlink subsys (disabled)
 Jun 07 04:15:51 debian kernel: xen_netfront: Initialising Xen virtual ethernet driver

 The domain config is:

 name = "debian"
 type = "pvh"
 memory = 1024
 vcpus = 2
 vif = [ 'mac=xx:xx:xx:xx:xx:xx,bridge=bridge0' ]

 kernel = "/home/xen/debian/vmlinuz"
 ramdisk = "/home/xen/debian/initrd.img"
 extra = "root=/dev/xvda1p1 console=tty1 console=hvc0"

 disk = [
   'file:/home/xen/debian/disk.img,xvda1,w',
   'file:/home/xen/debian/debian-12.11.0-amd64-netinst.iso,xvdb,ro,devtype=cdrom'
 ] 

Home	PR Database Search