NetBSD Problem Report #29750

From www@netbsd.org  Sun Mar 20 22:22:45 2005
Return-Path: <www@netbsd.org>
Received: by narn.netbsd.org (Postfix, from userid 31301)
	id C4C9363B116; Sun, 20 Mar 2005 22:22:45 +0000 (UTC)
Message-Id: <20050320222245.C4C9363B116@narn.netbsd.org>
Date: Sun, 20 Mar 2005 22:22:45 +0000 (UTC)
From: manu@netbsd.org
Reply-To: manu@netbsd.org
To: gnats-bugs@netbsd.org
Subject: send/sendto/sendmsg don't block when no buffer space is available
X-Send-Pr-Version: www-1.0

>Number:         29750
>Category:       kern
>Synopsis:       send/sendto/sendmsg don't block when no buffer space is available
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sun Mar 20 22:23:00 +0000 2005
>Last-Modified:  Fri Jun 08 00:45:01 +0000 2012
>Originator:     Emmanuel Dreyfus
>Release:        NetBSD-current (3.99.1)
>Organization:
The NetBSD Project
>Environment:
>Description:
NetBSD man page and Single Unix Specification say that if there is no kernel buffer space available, send/sendto/sendmsg should block if the socket was not set for non blocking I/O. 

Currently, we don't block, we return ENOBUFS, regardless if non blocking I/O was set. This breaks the standard.
>How-To-Repeat:
Use this test program with the numeric IP of a host near you as the first argument (NB: this will flood the network to get out of buffers. At mine it fails after sending 317 kB) 

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <fcntl.h>
#include <string.h>
#include <errno.h>
#include <netdb.h>
#include <time.h>

#include <sys/types.h>
#include <sys/param.h>
#include <sys/socket.h>

#include <arpa/inet.h>
#include <netinet/in.h>

int
main(ac, av) 
        int ac;
        char **av;
{
        int sd;
        int i;
        struct sockaddr_in src_addr;
        struct sockaddr_in send_addr;
        char *packet;
        size_t packet_size;

        if (ac != 2) {
                printf("Usage: test remote-host-ip\n");
                return 1;
        }

        if ((sd = socket(PF_INET, SOCK_DGRAM, IPPROTO_UDP)) == -1) {
                printf("Cannot create UDP/IP socket: %s\n", strerror(errno));
                exit(-1);
        }

        (void)memset(&src_addr, 0, sizeof(src_addr));
        src_addr.sin_family = AF_INET;
        src_addr.sin_len = sizeof(src_addr);
        src_addr.sin_addr.s_addr = htonl(0x00000000);
        src_addr.sin_port = htons(5002);

        (void)memset(&send_addr, 0, sizeof(send_addr));
        send_addr.sin_family = AF_INET;
        send_addr.sin_len = sizeof(send_addr);
        if (inet_aton(av[1], &send_addr.sin_addr) != 1) {
                printf("inet_aton: %s\n", strerror(errno));
                exit(-1);
        }
        send_addr.sin_port = htons(5002);

        if ((bind(sd, (struct sockaddr *)&src_addr, sizeof(src_addr))) == -1) {
                printf("Cannot bind to UDP port 5002: %s\n", strerror(errno));
                exit(-1);
        }

        packet_size = 1024;
        if ((packet = malloc(packet_size)) == NULL) {
                perror("Cannot allocate buffer");
                exit(-1);
        }
        bzero(packet, sizeof(*packet));

        for (i = 0; i < (1024 * 1024); i++) {
                if (sendto(sd, packet, packet_size, 0, 
                    (struct sockaddr *)&send_addr, sizeof(send_addr)) == -1) {
                        if (errno == ENOBUFS) {
                                printf("TEST FAILED (%d packets)\n", i);
                                exit(1);
                        } else {
                                perror("Send failed");
                        }
                }
        }

        printf("TEST PASSED\n");
        return 0;
}
>Fix:
None yet. Maybe in src/sys/kern/uipc_syscalls.c:sendit() loop around so->so_send(), sleeping when we get ENOBUFS? Of course that require the code that frees the buffer to wake us up.


>Audit-Trail:
From: manu@netbsd.org (Emmanuel Dreyfus)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/29750: send/sendto/sendmsg don't block when no buffer space is available
Date: Sun, 20 Mar 2005 23:31:43 +0100

 The probler is larger: poll() is not able to block until data is ready

 i.e: Adding this before sendto() will not make it work.

         struct pollfd pfd;

         pfd.fd = sd;
         pfd.events = POLLOUT;

         if (poll(&pfd, 1, INFTIM) == -1)
                 err(1, "poll");
         if ((pfd.revents & POLLOUT) == 0)
                 warnx("poll returned %x", pfd.revents);


 -- 
 Emmanuel Dreyfus
 Le cahier de l'admin BSD 2eme ed. est dans toutes les bonnes librairies
 http://www.eyrolles.com/Informatique/Livre/9782212114638/livre-bsd.php
 manu@netbsd.org

From: manu@netbsd.org (Emmanuel Dreyfus)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/29750: send/sendto/sendmsg don't block when no buffer space is available
Date: Sun, 20 Mar 2005 23:42:54 +0100

 1.6.2 passes the test
 2.0 fails to pass the test 

 -- 
 Emmanuel Dreyfus
 http://hcpnet.free.fr/pubz
 manu@netbsd.org

From: manu@netbsd.org (Emmanuel Dreyfus)
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/29750: send/sendto/sendmsg don't block when no buffer space is available
Date: Wed, 23 Mar 2005 00:33:22 +0100

 In-depth analysis of the problem:

 sosend() checks that there is enough buffer space and sleep awaiting for
 buffer space if it gets scarce. That code works: the problem is not
 caused because buffer space get depleted.

 If there is buffer space, sosend() calls lower layer output functions.
 Here udp_output(), then ip_output(), and ether_output().

 ether_output() uses IFQ_ENQUEUE() to queue the packet. That macro checks
 that the queue is not full through IF_QFULL(), and return ENOBUFS if the
 interface queue is full. This is what happens here. 

 ENOBUFS is returned up to send() and causes the problem. 


 Other systems:
 FreeBSD 4.11-RELEASE i386 also fails the test
 Linux passes the test

 -- 
 Emmanuel Dreyfus
 Le cahier de l'admin BSD 2eme ed. est dans toutes les bonnes librairies
 http://www.eyrolles.com/Informatique/Livre/9782212114638/livre-bsd.php
 manu@netbsd.org

From: Jason Thorpe <thorpej@shagadelic.org>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org,
	netbsd-bugs@netbsd.org
Subject: Re: kern/29750: send/sendto/sendmsg don't block when no buffer space is available
Date: Tue, 22 Mar 2005 15:42:19 -0800

 On Mar 22, 2005, at 3:34 PM, Emmanuel Dreyfus wrote:

 >  If there is buffer space, sosend() calls lower layer output  
 > functions.
 >  Here udp_output(), then ip_output(), and ether_output().

 By this time, the data is properly enqueued in the socket buffer, right?

 >  ether_output() uses IFQ_ENQUEUE() to queue the packet. That macro  
 > checks
 >  that the queue is not full through IF_QFULL(), and return ENOBUFS  
 > if the
 >  interface queue is full. This is what happens here.
 >
 >  ENOBUFS is returned up to send() and causes the problem.

 Yuck.  Arguably, what should happen is that this ENOBUFS should get  
 translated to 0 (no error) in sosend().  However, we still need some  
 way to kick the socket into calling udp_output() again once ifqueue  
 space becomes available.

 Or we could just say "screw it!" and get rid of ifqeueue max lengths  
 all together.

 -- thorpej

From: Gary Thorpe <gathorpe79@yahoo.com>
To: gnats-bugs@netbsd.org
Cc: 
Subject: Re: kern/29750: send/sendto/sendmsg don't block when no buffer space is available
Date: Wed, 25 Jul 2007 12:09:12 -0400 (EDT)

 Hi,

 I also encountered this bug, but with UNIX domain sockets and datagrams
 (NetBSD 3.0). It fails when using sendmsg() and using select() does not
 help: it returns that the descriptor is ready but attempting to send
 returns ENOBUFS. This error requires busy-waiting (not very efficient)
 as a work-around when ENOBUFS occurs.

 Bug seems to affect datagram services in general and it is probably not
 just IFQ_ENQUEUE because there is no IP or ethernet involved for my
 case.

 Hope this helps.


       Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail at http://mrd.mail.yahoo.com/try_beta?.intl=ca

From: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
To: gnats-bugs@netbsd.org
Cc: Hauke Fath <hauke@Espresso.Rhein-Neckar.DE>
Subject: Re: kern/29750: send/sendto/sendmsg don't block when no buffer
 space is available
Date: Thu, 7 Jun 2012 18:43:34 +0200

 I get a lot of these

 Jun  7 18:22:09 pizza vpnc[570]: udp sendto: No buffer space available
 Jun  7 18:23:54 pizza vpnc[570]: udp sendto: No buffer space available
 Jun  7 18:25:54 pizza last message repeated 7 times
 Jun  7 18:32:53 pizza last message repeated 21 times
 Jun  7 18:36:55 pizza vpnc[570]: udp sendto: No buffer space available
 Jun  7 18:36:55 pizza vpnc[570]: HMAC mismatch in ESP mode
 Jun  7 18:37:54 pizza vpnc[570]: udp sendto: No buffer space available
 Jun  7 18:37:54 pizza last message repeated 2 times
 Jun  7 18:38:15 pizza vpnc[570]: keepalive sendto: No buffer space available

 from a vpnc ipsec tunnel. Would that be related to this bug?

 	hauke


 --
 "It's never straight up and down"     (DEVO)


From: manu@netbsd.org (Emmanuel Dreyfus)
To: gnats-bugs@NetBSD.org, kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc: 
Subject: Re: kern/29750: send/sendto/sendmsg don't block when no buffer space is available
Date: Fri, 8 Jun 2012 02:41:50 +0200

 Hauke Fath <hauke@Espresso.Rhein-Neckar.DE> wrote:

 >  from a vpnc ipsec tunnel. Would that be related to this bug?

 Possibly. At least it looks similar.

 -- 
 Emmanuel Dreyfus
 http://hcpnet.free.fr/pubz
 manu@netbsd.org

Home	PR Database Search