NetBSD Problem Report #50367
From www@NetBSD.org Sun Oct 25 02:20:03 2015
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
by mollari.NetBSD.org (Postfix) with ESMTPS id 2D057A5858
for <gnats-bugs@gnats.NetBSD.org>; Sun, 25 Oct 2015 02:20:03 +0000 (UTC)
Message-Id: <20151025022001.F4019A65BB@mollari.NetBSD.org>
Date: Sun, 25 Oct 2015 02:20:01 +0000 (UTC)
From: schaecsn@gmx.net
Reply-To: schaecsn@gmx.net
To: gnats-bugs@NetBSD.org
Subject: libc resolver library does not resolve host names with underscores ("_")
X-Send-Pr-Version: www-1.0
>Number: 50367
>Category: lib
>Synopsis: libc resolver library does not resolve host names with underscores ("_")
>Confidential: no
>Severity: non-critical
>Priority: medium
>Responsible: lib-bug-people
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Sun Oct 25 02:25:00 +0000 2015
>Closed-Date: Sun Aug 18 04:29:38 +0000 2019
>Last-Modified: Sun Aug 18 04:29:38 +0000 2019
>Originator: Stefan Schaeckeler
>Release: NetBSD 7.0
>Organization:
>Environment:
NetBSD t60p 7.0 NetBSD 7.0 (GENERIC) #0: Sat Sep 26 20:21:41 PDT 2015 root@t60p:/usr/obj/sys/arch/i386/compile/GENERIC i386
>Description:
Several internet radio stations use host names with underscores, e.g.
http://dradio_mp3_dlf_m.akacast.akamaistream.net/7/249/142684/v1/gnl.akacast.akamaistream.net/dradio_mp3_dlf_m
http://dradio_mp3_dkultur_m.akacast.akamaistream.net/7/530/142684/v1/gnl.akacast.akamaistream.net/dradio_mp3_dkultur_m
http://dradio_mp3_dwissen_m.akacast.akamaistream.net/7/728/142684/v1/gnl.akacast.akamaistream.net/dradio_mp3_dwissen_m
Such host names are not resolved:
$ mpg123 -yC http://dradio_mp3_dlf_m.akacast.akamaistream.net/7/249/142684/v1/gnl.akacast.akamaistream.net/dradio_mp3_dlf_m
High Performance MPEG 1.0/2.0/2.5 Audio Player for Layers 1, 2 and 3
version 1.21.0; written and copyright by Michael Hipp and others
free software (LGPL) without any warranty but with best wishes
[resolver.c:289] error: Resolving dradio_mp3_dlf_m.akacast.akamaistream.net:80: Non-recoverable failure in name resolution
[httpget.c:592] error: Unable to establish connection to dradio_mp3_dlf_m.akacast.akamaistream.net
[mpg123.c:611] error: Access to http resource http://dradio_mp3_dlf_m.akacast.akamaistream.net/7/249/142684/v1/gnl.akacast.akamaistream.net/dradio_mp3_dlf_m failed.
Other libcs such as Linux' and FreeBSD's libc resolve such host names. Apparently, NetBSD's resolver library filters out underscores. Provided patch fixes the too restrictive filtering.
>How-To-Repeat:
mpg123 -yC http://dradio_mp3_dlf_m.akacast.akamaistream.net/7/249/142684/v1/gnl.akacast.akamaistream.net/dradio_mp3_dlf_m
or
$ getaddrinfo dradio_mp3_dlf_m.akacast.akamaistream.net
getaddrinfo: Non-recoverable failure in name resolution
>Fix:
--- lib/libc/resolv/res_comp.c 2015-10-24 19:05:43.000000000 -0700
+++ lib/libc/resolv/res_comp.c.new 2015-10-24 18:13:24.000000000 -0700
@@ -163,6 +163,7 @@
*/
#define PERIOD 0x2e
#define hyphenchar(c) ((c) == 0x2d)
+#define underschar(c) ((c) == 0x5f)
#define bslashchar(c) ((c) == 0x5c)
#define periodchar(c) ((c) == PERIOD)
#define asterchar(c) ((c) == 0x2a)
@@ -171,7 +172,7 @@
#define digitchar(c) ((c) >= 0x30 && (c) <= 0x39)
#define borderchar(c) (alphachar(c) || digitchar(c))
-#define middlechar(c) (borderchar(c) || hyphenchar(c))
+#define middlechar(c) (borderchar(c) || hyphenchar(c) || underschar(c))
#define domainchar(c) ((c) > 0x20 && (c) < 0x7f)
int
>Release-Note:
>Audit-Trail:
From: Brett Lymn <blymn@internode.on.net>
To: gnats-bugs@NetBSD.org
Cc: lib-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: lib/50367: libc resolver library does not resolve host names with underscores ("_")
Date: Sun, 25 Oct 2015 14:13:11 +1030
On Sun, Oct 25, 2015 at 02:25:00AM +0000, schaecsn@gmx.net wrote:
>
> Such host names are not resolved:
>
> $ mpg123 -yC http://dradio_mp3_dlf_m.akacast.akamaistream.net/7/249/142684/v1/gnl.akacast.akamaistream.net/dradio_mp3_dlf_m
> High Performance MPEG 1.0/2.0/2.5 Audio Player for Layers 1, 2 and 3
> version 1.21.0; written and copyright by Michael Hipp and others
> free software (LGPL) without any warranty but with best wishes
>
> [resolver.c:289] error: Resolving dradio_mp3_dlf_m.akacast.akamaistream.net:80: Non-recoverable failure in name resolution
>
That is actually the correct behaviour, underscores are not legal in
hostnames.
--
Brett Lymn
Let go, or be dragged - Zen proverb.
From: schaecsn <schaecsn@gmx.net>
To: gnats-bugs@NetBSD.org
Cc: lib-bug-people@netbsd.org,gnats-admin@netbsd.org,netbsd-bugs@netbsd.org
Subject: Re: lib/50367: libc resolver library does not resolve host names with underscores ("_")
Date: Sat, 24 Oct 2015 21:33:01 -0700 (PDT)
Oh, you are right!
By enforcing the standard, NetBSD is excluding parts of the WWW. Other OSes don't do that.
From: Robert Elz <kre@munnari.OZ.AU>
To: Brett Lymn <blymn@internode.on.net>
Cc: gnats-bugs@NetBSD.org, lib-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: lib/50367: libc resolver library does not resolve host names with underscores ("_")
Date: Sun, 25 Oct 2015 11:36:52 +0700
Date: Sun, 25 Oct 2015 14:13:11 +1030
From: Brett Lymn <blymn@internode.on.net>
Message-ID: <20151025034311.GA1222@internode.on.net>
| That is actually the correct behaviour, underscores are not legal in
| hostnames.
That's correct, for hostnames being entered into hostnames.txt (a file
that hasn't existed in any meaningful form for 30 years.)
There are no constraints on names that can appear in the DNS (only a
recommendation on what was good to use, in the mid 80's, for compat
with old software), and hence there should be no constraints on what
can be resolved (just adding '_' to a magic list of acceptable chars
is wrong.)
e-mail domain names are a whole other issue of course, similarly names
used with other protocols that might have various limitations, and as are
the set of chars that NetBSD (or any other OS) permits for the
sethostname(2) syscall.
kre
From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@NetBSD.org, lib-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc:
Subject: Re: lib/50367: libc resolver library does not resolve host names with underscores ("_")
Date: Sun, 25 Oct 2015 06:14:47 -0400
On Oct 25, 2:25am, schaecsn@gmx.net (schaecsn@gmx.net) wrote:
-- Subject: lib/50367: libc resolver library does not resolve host names with
| >Number: 50367
| >Category: lib
| >Synopsis: libc resolver library does not resolve host names with underscores ("_")
| >Confidential: no
| >Severity: non-critical
| >Priority: medium
| >Responsible: lib-bug-people
| >State: open
| >Class: sw-bug
| >Submitter-Id: net
| >Arrival-Date: Sun Oct 25 02:25:00 +0000 2015
| >Originator: Stefan Schaeckeler
| >Release: NetBSD 7.0
| >Organization:
| >Environment:
| NetBSD t60p 7.0 NetBSD 7.0 (GENERIC) #0: Sat Sep 26 20:21:41 PDT 2015 root@t60p:/usr/obj/sys/arch/i386/compile/GENERIC i386
| >Description:
| Several internet radio stations use host names with underscores, e.g.
|
| http://dradio_mp3_dlf_m.akacast.akamaistream.net/7/249/142684/v1/gnl.akacast.akamaistream.net/dradio_mp3_dlf_m
| http://dradio_mp3_dkultur_m.akacast.akamaistream.net/7/530/142684/v1/gnl.akacast.akamaistream.net/dradio_mp3_dkultur_m
| http://dradio_mp3_dwissen_m.akacast.akamaistream.net/7/728/142684/v1/gnl.akacast.akamaistream.net/dradio_mp3_dwissen_m
|
|
| Such host names are not resolved:
Underscores are not allowed in hostnames:
https://en.wikipedia.org/wiki/Hostname
christos
From: schaecsn <schaecsn@gmx.net>
To: gnats-bugs@NetBSD.org
Cc: lib-bug-people@netbsd.org,gnats-admin@netbsd.org,netbsd-bugs@netbsd.org
Subject: Re: lib/50367: libc resolver library does not resolve host names with underscores ("_")
Date: Sun, 25 Oct 2015 11:45:07 -0700 (PDT)
I start to understand that you are right. But by enforcing the RFC, NetBSD is excluding parts of the Internet while other OSes don't (I tested Linux, FreeBSD, MacOS and MS Windows 2000 + XP).
Also, should enforcing sound hostnames be the resonsible of the resolver library ... even if contacted DNS servers would be willing to map them to IP addresses? Perhaps, it should be up to the DNS servers to enforce sound hostnames?
From: christos@zoulas.com (Christos Zoulas)
To: schaecsn@gmx.net, gnats-bugs@NetBSD.org
Cc: lib-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: lib/50367: libc resolver library does not resolve host names with underscores ("_")
Date: Sun, 25 Oct 2015 16:15:36 -0400
On Oct 25, 11:45am, schaecsn@gmx.net (schaecsn) wrote:
-- Subject: Re: lib/50367: libc resolver library does not resolve host names
| I start to understand that you are right. But by enforcing the RFC, NetBSD is excluding parts of the Internet while other OSes don't (I tested Linux, FreeBSD, MacOS and MS Windows 2000 + XP).
|
| Also, should enforcing sound hostnames be the resonsible of the resolver library ... even if contacted DNS servers would be willing to map them to IP addresses? Perhaps, it should be up to the DNS servers to enforce sound hostnames?
I understand. It is a difficult question to answer. And if you fix the
resolver, do you go and fix the cacheing nameserver
(src/external/bsd/bind/dist/lib/dns/name.c)?
Do you accept it without warn, warn and accept, or reject?
My inclination is to have a new "strict" resolver option, default to off
and allow '_' in hostnames... If CDNs are putting _'s in hostnames there
is not much we can do; either we pretend to be the internet fighters for
chastity, or accept that the world is changing adjust course and move on.
christos
From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/50367 CVS commit: src/lib/libc/net
Date: Mon, 26 Oct 2015 10:48:04 -0400
Module Name: src
Committed By: christos
Date: Mon Oct 26 14:48:04 UTC 2015
Modified Files:
src/lib/libc/net: getaddrinfo.c
Log Message:
PR/50367: Stefan Schaeckeler: libc resolver library does not resolve host
names with underscores ("_"). According to resolv.conf(5) RES_NOCHECKNAME
is on by default; well, it is and gethostbyname(3) obeys it
(gethnamaddr.c:maybe_ok), but getaddrinfo(3) up till this commit not.
XXX: pullup-7, pullup-6
To generate a diff of this commit:
cvs rdiff -u -r1.109 -r1.110 src/lib/libc/net/getaddrinfo.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: schaecsn <schaecsn@gmx.net>
To: gnats-bugs@NetBSD.org
Cc: lib-bug-people@netbsd.org,gnats-admin@netbsd.org,netbsd-bugs@netbsd.org
Subject: Re: PR/50367 CVS commit: src/lib/libc/net
Date: Mon, 26 Oct 2015 10:36:50 -0700 (PDT)
Shouldn't the sister file getnetnamadr.c be changed as well? Something along this line:
Index: lib/libc/net/getnetnamadr.c
===================================================================
RCS file: /cvsroot/src/lib/libc/net/getnetnamadr.c,v
retrieving revision 1.42
diff -u -p -r1.42 getnetnamadr.c
--- lib/libc/net/getnetnamadr.c 13 Mar 2012 21:13:41 -0000 1.42
+++ lib/libc/net/getnetnamadr.c 26 Oct 2015 17:28:21 -0000
@@ -77,6 +77,12 @@ __weak_alias(getnetbyaddr,_getnetbyaddr)
__weak_alias(getnetbyname,_getnetbyname)
#endif
+#define maybe_ok(res, nm, ok) (((res)->options & RES_NOCHECKNAME) != 0U || \
+ (ok)(nm) != 0)
+#define maybe_hnok(res, hn) maybe_ok((res), (hn), res_hnok)
+#define maybe_dnok(res, dn) maybe_ok((res), (dn), res_dnok)
+
+
extern int _net_stayopen;
#define BYADDR 0
@@ -105,7 +111,7 @@ static struct netent net_entry;
static char *net_aliases[MAXALIASES];
static int parse_reversed_addr(const char *, in_addr_t *);
-static struct netent *getnetanswer(querybuf *, int, int);
+static struct netent *getnetanswer(res_state, querybuf *, int, int);
static int _files_getnetbyaddr(void *, void *, va_list);
static int _files_getnetbyname(void *, void *, va_list);
static int _dns_getnetbyaddr(void *, void *, va_list);
@@ -159,7 +165,7 @@ parse_reversed_addr(const char *str, in_
}
static struct netent *
-getnetanswer(querybuf *answer, int anslen, int net_i)
+getnetanswer(res_state res,querybuf *answer, int anslen, int net_i)
{
static char n_name[MAXDNAME];
static char netbuf[PACKETSZ];
@@ -172,6 +178,7 @@ getnetanswer(querybuf *answer, int ansle
char *in, *bp, **ap, *ep;
_DIAGASSERT(answer != NULL);
+ _DIAGASSERT(res != NULL);
/*
* find first satisfactory answer
@@ -216,7 +223,7 @@ getnetanswer(querybuf *answer, int ansle
n_name[0] = '\0';
while (--ancount >= 0 && cp < eom) {
n = dn_expand(answer->buf, eom, cp, bp, (int)(ep - bp));
- if ((n < 0) || !res_dnok(bp))
+ if ((n < 0) || !maybe_dnok(res, bp))
break;
cp += n;
(void)strlcpy(n_name, bp, sizeof(n_name));
@@ -226,7 +233,7 @@ getnetanswer(querybuf *answer, int ansle
GETSHORT(n, cp);
if (class == C_IN && type == T_PTR) {
n = dn_expand(answer->buf, eom, cp, bp, (int)(ep - bp));
- if ((n < 0) || !res_hnok(bp)) {
+ if ((n < 0) || !maybe_hnok(res, bp)) {
cp += n;
return NULL;
}
@@ -359,7 +366,7 @@ _dns_getnetbyaddr(void *cbrv, void *cbda
return NS_NOTFOUND;
}
__res_put_state(res);
- np = getnetanswer(buf, anslen, BYADDR);
+ np = getnetanswer(res, buf, anslen, BYADDR);
free(buf);
if (np) {
/* maybe net should be unsigned? */
@@ -471,7 +478,7 @@ _dns_getnetbyname(void *cbrv, void *cbda
return NS_NOTFOUND;
}
__res_put_state(res);
- np = getnetanswer(buf, anslen, BYNAME);
+ np = getnetanswer(res, buf, anslen, BYNAME);
free(buf);
if (np != NULL) {
From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/50367 CVS commit: src/lib/libc/net
Date: Mon, 26 Oct 2015 15:28:53 -0400
Module Name: src
Committed By: christos
Date: Mon Oct 26 19:28:53 UTC 2015
Modified Files:
src/lib/libc/net: getnetnamadr.c
Log Message:
PR/50367: Stefan Schaeckeler: Apply fix to obey RES_CHECKNAME to getnetbyaddr
and getnetbyname.
To generate a diff of this commit:
cvs rdiff -u -r1.42 -r1.43 src/lib/libc/net/getnetnamadr.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: christos@zoulas.com (Christos Zoulas)
To: schaecsn@gmx.net, gnats-bugs@NetBSD.org
Cc: lib-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org
Subject: Re: PR/50367 CVS commit: src/lib/libc/net
Date: Mon, 26 Oct 2015 15:29:38 -0400
On Oct 26, 10:36am, schaecsn@gmx.net (schaecsn) wrote:
-- Subject: Re: PR/50367 CVS commit: src/lib/libc/net
| Shouldn't the sister file getnetnamadr.c be changed as well? Something along
| this line:
Yes, applied.
christos
From: "Jeff Rizzo" <riz@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/50367 CVS commit: [netbsd-7-0] src
Date: Sun, 8 Nov 2015 00:35:21 +0000
Module Name: src
Committed By: riz
Date: Sun Nov 8 00:35:21 UTC 2015
Modified Files:
src/external/gpl3/binutils/dist/binutils [netbsd-7-0]: arsup.c
src/lib/libc/net [netbsd-7-0]: getnetnamadr.c
Log Message:
Pull up following revision(s) (requested by christos in ticket #1012):
lib/libc/net/getnetnamadr.c: revision 1.43
lib/libc/net/getnetnamadr.c: revision 1.44
external/gpl3/binutils/dist/binutils/arsup.c: revision 1.2
put the state back after it is used.
PR/50367: Stefan Schaeckeler: Apply fix to obey RES_CHECKNAME to getnetbyaddr
and getnetbyname.
use asprintf(3)
To generate a diff of this commit:
cvs rdiff -u -r1.1.1.2 -r1.1.1.2.22.1 \
src/external/gpl3/binutils/dist/binutils/arsup.c
cvs rdiff -u -r1.42 -r1.42.12.1 src/lib/libc/net/getnetnamadr.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Jeff Rizzo" <riz@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/50367 CVS commit: [netbsd-7] src
Date: Sun, 8 Nov 2015 00:35:45 +0000
Module Name: src
Committed By: riz
Date: Sun Nov 8 00:35:45 UTC 2015
Modified Files:
src/external/gpl3/binutils/dist/binutils [netbsd-7]: arsup.c
src/lib/libc/net [netbsd-7]: getnetnamadr.c
Log Message:
Pull up following revision(s) (requested by christos in ticket #1012):
lib/libc/net/getnetnamadr.c: revision 1.43
lib/libc/net/getnetnamadr.c: revision 1.44
external/gpl3/binutils/dist/binutils/arsup.c: revision 1.2
put the state back after it is used.
PR/50367: Stefan Schaeckeler: Apply fix to obey RES_CHECKNAME to getnetbyaddr
and getnetbyname.
use asprintf(3)
To generate a diff of this commit:
cvs rdiff -u -r1.1.1.2 -r1.1.1.2.20.1 \
src/external/gpl3/binutils/dist/binutils/arsup.c
cvs rdiff -u -r1.42 -r1.42.10.1 src/lib/libc/net/getnetnamadr.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: schaecsn <schaecsn@gmx.net>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: PR/50367 CVS commit: [netbsd-7-0] src
Date: Mon, 9 Nov 2015 13:03:03 -0800 (PST)
Something might have gone wrong here.
It has been committed:
src/external/gpl3/binutils/dist/binutils/arsup.c
src/lib/libc/net/getnetnamadr.c
What should have been committed, instead:
src/lib/libc/net/getaddrinfo.c
src/lib/libc/net/getnetnamadr.c
Stefan
From: "Manuel Bouyer" <bouyer@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/50367 CVS commit: [netbsd-6] src/lib/libc/net
Date: Tue, 17 Nov 2015 09:25:03 +0000
Module Name: src
Committed By: bouyer
Date: Tue Nov 17 09:25:03 UTC 2015
Modified Files:
src/lib/libc/net [netbsd-6]: getnetnamadr.c
Log Message:
Pull up following revision(s) (requested by christos in ticket #1340):
lib/libc/net/getnetnamadr.c: revision 1.43 via patch
lib/libc/net/getnetnamadr.c: revision 1.44 via patch
put the state back after it is used.
PR/50367: Stefan Schaeckeler: Apply fix to obey RES_CHECKNAME to getnetbyaddr
and getnetbyname.
To generate a diff of this commit:
cvs rdiff -u -r1.41 -r1.41.22.1 src/lib/libc/net/getnetnamadr.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: "Manuel Bouyer" <bouyer@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/50367 CVS commit: [netbsd-6] src/lib/libc/net
Date: Tue, 17 Nov 2015 09:28:05 +0000
Module Name: src
Committed By: bouyer
Date: Tue Nov 17 09:28:05 UTC 2015
Modified Files:
src/lib/libc/net [netbsd-6]: getaddrinfo.c
Log Message:
Pull up following revision(s) (requested by christos in ticket #1340):
lib/libc/net/getnetnamadr.c: revision 1.43 via patch
lib/libc/net/getnetnamadr.c: revision 1.44 via patch
put the state back after it is used.
PR/50367: Stefan Schaeckeler: Apply fix to obey RES_CHECKNAME to getnetbyaddr
and getnetbyname.
To generate a diff of this commit:
cvs rdiff -u -r1.96.4.2 -r1.96.4.3 src/lib/libc/net/getaddrinfo.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: open->closed
State-Changed-By: maya@NetBSD.org
State-Changed-When: Sun, 18 Aug 2019 04:29:38 +0000
State-Changed-Why:
From the discussion, it looks like everything is applied and backported. Let me know if that impression is wrong.
Thanks for the report, and thanks to christos for fixing it.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.43 2018/01/16 07:36:43 maya Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2017
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.