NetBSD Problem Report #48926
From buhrow@lothlorien.nfbcal.org Thu Jun 19 08:51:59 2014
Return-Path: <buhrow@lothlorien.nfbcal.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
(Client CN "mail.netbsd.org", Issuer "Postmaster NetBSD.org" (verified OK))
by mollari.NetBSD.org (Postfix) with ESMTPS id E74D0A6526
for <gnats-bugs@gnats.NetBSD.org>; Thu, 19 Jun 2014 08:51:59 +0000 (UTC)
Message-Id: <201406190851.s5J8pvbE014177@lothlorien.nfbcal.org>
Date: Thu, 19 Jun 2014 01:51:57 -0700 (PDT)
From: buhrow@nfbcal.org
Reply-To: buhrow@nfbcal.org
To: gnats-bugs@gnats.NetBSD.org
Subject: gethostbyname(3) doesn't appear to work under NetBSD-5.2 and NetBSD-6.1.x when hostnames appear only in /etc/hosts
X-Send-Pr-Version: 3.95
>Number: 48926
>Category: lib
>Synopsis: Hostnames only appearing in /etc/hosts are not reachable using gethostbyname(3) or gethostbyname2(3)
>Confidential: no
>Severity: serious
>Priority: high
>Responsible: lib-bug-people
>State: closed
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Thu Jun 19 08:55:00 +0000 2014
>Closed-Date: Wed Jun 22 08:42:20 +0000 2016
>Last-Modified: Wed Jun 22 08:42:20 +0000 2016
>Originator: Brian Buhrow
>Release: NetBSD 5.2_STABLE, NetBSD-6.1.x and probably Netbsd-current
>Organization:
NFB of California
>Environment:
>Description:
hello. It appears that gethostbyname(3) in NetBSD-5.2 and 6.1.x is
broken if the name you're trying to fetch is listed in /etc/hosts and
nowhere else. The problem appears to be with this change:
RCS file: /cvsroot/src/lib/libc/net/gethnamaddr.c
revision 1.73.18.3
date: 2013-12-23 23:12:44 +0000; author: riz; state: Exp; lines: +10 -7; commitid: QLuYhA2iZpQzpjix;
Apply patch (requested by bad in ticket #1887):
src/lib/libc/net/getaddrinfo.c r1.106:
this is supposed to be re-entrant, call don't call __hostalias that uses
a static buffer.
src/lib/libc/net/gethnamaddr.c r1.85:
- don't clobber hp in the RES_USE_INET6 case
- increment naddrs in the yp case
- don't use __hostalias(), it is not thread-safe.
This should finish addressing PR lib/46454
----------------------------
Here are some scripts that demonstrate the problem. Note I've only shown
the issue under NetbSD-5.1 and 5.2, but NetBSD-6.1.x exhibits the
NetBSD-5.2 broken behavior.
good script started on Wed Jun 18 23:35:48 2014
stats# uname -a
NetBSD stats 5.1_STABLE NetBSD 5.1_STABLE (X2200) #0: Mon Mar 18 11:05:41 PDT 2013 buhrow@lothlorien.nfbcal.org:/usr/src/sys/arch/i386/compile/X2200 i386
stats# ident /lib/libc.so.12.164 |grep 'gethnamaddr'
$NetBSD: gethnamaddr.c,v 1.73 2007/01/27 22:27:35 christos Exp $
stats# cat /etc/hosts |tail -1
192.168.3.1 this_host_only_in_etc_hosts.via.net this_host_only_in_etc_hosts
stats# getent hosts this_host_only_in_etc_hosts
192.168.3.1 this_host_only_in_etc_hosts.via.net this_host_only_in_etc_hosts
stats# exit
bad script started on Wed Jun 18 23:38:58 2014
mirkwood#
mirkwood# uname -a
NetBSD mirkwood.nfbcal.org 5.2_STABLE NetBSD 5.2_STABLE (RBL) #0: Mon Mar 24 21:42:05 PDT 2014 buhrow@lothlorien.nfbcal.org:/usr/src/sys/arch/i386/compile/RBL i386
mirkwood# ident /lib/libc.so.12.164 |grep 'gethnamaddr'
$NetBSD: gethnamaddr.c,v 1.73.18.3 2013/12/23 23:12:44 riz Exp $
mirkwood# cat /etc/hosts |tail -1
192.168.3.1 this_host_only_in_etc_hosts.via.net this_host_only_in_etc_hosts
mirkwood# getent hosts this_host_only_in_etc_hosts
mirkwood# exit
I'm not familiar enough with the libc resolver code to feel confident
enough to make a commitable fix. Has anyone else run into this behavior?
Even better, has anyone found a fix?
-thanks
-brian
>How-To-Repeat:
See above.
>Fix:
>Release-Note:
>Audit-Trail:
From: Brian Buhrow <buhrow@nfbcal.org>
To: gnats-bugs@NetBSD.org
Cc: buhrow@nfbcal.org
Subject: Re: lib/48926: gethostbyname(3) doesn't appear to work under NetBSD-5.2 and NetBSD-6.1.x when hostnames appear only in /etc/hosts
Date: Mon, 23 Jun 2014 01:14:04 -0700
Hello. Further investigation revealed this bug appears to
only affecte NetBSD-5.2 and anything on the netbsd-5 branch. A better
descripption of the bug is that only the first valid entry in the
/etc/hosts file will ever be seen by the system because the new file
parsing routines assume that fgetln() returns a nul terminated string, as
fgets() used to, and that it is safe to put NULL characters in the middle
of the returned string to help separate the fields of valid lines. The
side effect is that with the latest changes in this file, once the first
entry is parsed, subsequent lines of the file are efectively erased because
the beginning of each line is "Nulled out" from the parsing of the previous
line.
Below is a patch to the NetBSd-5 version of the gethnamaddr.c file
which fixes this problem and restores the NetBSd-5.1 and earlier behavior.
It's possible this patch will need to be applied in some form to the
NetBSD-6 branch, but I've not yet tested that. This patch is known to work
on NetBSD-5 systems without a problem.
I'll request a pullup shortly.
-thanks
-Brian
Index: gethnamaddr.c
===================================================================
RCS file: /cvsroot/src/lib/libc/net/gethnamaddr.c,v
retrieving revision 1.73.18.3
diff -u -r1.73.18.3 gethnamaddr.c
--- gethnamaddr.c 23 Dec 2013 23:12:44 -0000 1.73.18.3
+++ gethnamaddr.c 23 Jun 2014 07:27:02 -0000
@@ -1,4 +1,4 @@
-/* $NetBSD$ */
+/* $NetBSD: gethnamaddr.c,v 1.73.18.3 2013/12/23 23:12:44 riz Exp $ */
/*
* ++Copyright++ 1985, 1988, 1993
@@ -57,7 +57,7 @@
static char sccsid[] = "@(#)gethostnamadr.c 8.1 (Berkeley) 6/4/93";
static char rcsid[] = "Id: gethnamaddr.c,v 8.21 1997/06/01 20:34:37 vixie Exp ";
#else
-__RCSID("$NetBSD$");
+__RCSID("$NetBSD: gethnamaddr.c,v 1.73.18.3 2013/12/23 23:12:44 riz Exp $");
#endif
#endif /* LIBC_SCCS and not lint */
@@ -722,7 +722,7 @@
gethostent_r(FILE *hf, struct hostent *hent, char *buf, size_t buflen, int *he)
{
char *p, *name;
- char *cp, **q;
+ char *cp, **q, sav;
int af, len;
size_t llen, anum, i;
char *aliases[MAXALIASES];
@@ -742,12 +742,17 @@
goto again;
if (*p == '#')
goto again;
+ sav = p[llen];
p[llen] = '\0';
- if (!(cp = strpbrk(p, "#\n")))
+ if (!(cp = strpbrk(p, "#\n"))) {
+ p[llen] = sav;
goto again;
+ }
*cp = '\0';
- if (!(cp = strpbrk(p, " \t")))
+ if (!(cp = strpbrk(p, " \t"))) {
+ p[llen] = sav;
goto again;
+ }
*cp++ = '\0';
if (inet_pton(AF_INET6, p, &host_addr) > 0) {
af = AF_INET6;
@@ -766,13 +771,18 @@
}
__res_put_state(res);
} else {
+ p[llen] = sav;
goto again;
}
/* if this is not something we're looking for, skip it. */
- if (hent->h_addrtype != 0 && hent->h_addrtype != af)
+ if (hent->h_addrtype != 0 && hent->h_addrtype != af) {
+ p[llen] = sav;
goto again;
- if (hent->h_length != 0 && hent->h_length != len)
+ }
+ if (hent->h_length != 0 && hent->h_length != len) {
+ p[llen] = sav;
goto again;
+ }
while (*cp == ' ' || *cp == '\t')
cp++;
@@ -805,10 +815,12 @@
hent->h_aliases[anum] = NULL;
*he = NETDB_SUCCESS;
+ p[llen] = sav;
return hent;
nospc:
errno = ENOSPC;
*he = NETDB_INTERNAL;
+ p[llen] = sav;
return NULL;
}
From: Brian Buhrow <buhrow@nfbcal.org>
To: gnats-bugs@NetBSD.org, lib-bug-people@NetBSD.org, gnats-admin@NetBSD.org,
netbsd-bugs@NetBSD.org
Cc: buhrow@nfbcal.org
Subject: Re: lib/48926: gethostbyname(3) doesn't appear to work under NetBSD-5.2 and NetBSD-6.1.x when hostnames appear only in /etc/hosts
Date: Mon, 23 Jun 2014 08:09:43 -0700
hello. I should know never to submit the first working patch as the
final version. This one is much simpler and solves the problem much more
elegantly.
-thanks
-Brian
Index: gethnamaddr.c
===================================================================
RCS file: /cvsroot/src/lib/libc/net/gethnamaddr.c,v
retrieving revision 1.73.18.3
diff -u -r1.73.18.3 gethnamaddr.c
--- gethnamaddr.c 23 Dec 2013 23:12:44 -0000 1.73.18.3
+++ gethnamaddr.c 23 Jun 2014 14:46:25 -0000
@@ -742,10 +742,9 @@
goto again;
if (*p == '#')
goto again;
- p[llen] = '\0';
- if (!(cp = strpbrk(p, "#\n")))
- goto again;
- *cp = '\0';
+ p[llen - 1] = '\0';
+ if (cp = strpbrk(p, "#"))
+ *cp = '\0';
if (!(cp = strpbrk(p, " \t")))
goto again;
*cp++ = '\0';
From: christos@zoulas.com (Christos Zoulas)
To: Brian Buhrow <buhrow@nfbcal.org>, gnats-bugs@NetBSD.org,
lib-bug-people@NetBSD.org, gnats-admin@NetBSD.org,
netbsd-bugs@NetBSD.org
Cc:
Subject: Re: lib/48926: gethostbyname(3) doesn't appear to work under NetBSD-5.2 and NetBSD-6.1.x when hostnames appear only in /etc/hosts
Date: Mon, 23 Jun 2014 11:25:03 -0400
On Jun 23, 8:09am, buhrow@nfbcal.org (Brian Buhrow) wrote:
-- Subject: Re: lib/48926: gethostbyname(3) doesn't appear to work under NetB
| hello. I should know never to submit the first working patch as the
| final version. This one is much simpler and solves the problem much more
| elegantly.
| -thanks
| -Brian
Doesn't that eat the last character of a line if it does not end with '\n'?
christos
From: Joerg Sonnenberger <joerg@britannica.bec.de>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: lib/48926: gethostbyname(3) doesn't appear to work under
NetBSD-5.2 and NetBSD-6.1.x when hostnames appear only in /etc/hosts
Date: Mon, 23 Jun 2014 17:26:25 +0200
On Mon, Jun 23, 2014 at 03:10:13PM +0000, Brian Buhrow wrote:
> Index: gethnamaddr.c
> ===================================================================
> RCS file: /cvsroot/src/lib/libc/net/gethnamaddr.c,v
> retrieving revision 1.73.18.3
> diff -u -r1.73.18.3 gethnamaddr.c
> --- gethnamaddr.c 23 Dec 2013 23:12:44 -0000 1.73.18.3
> +++ gethnamaddr.c 23 Jun 2014 14:46:25 -0000
> @@ -742,10 +742,9 @@
> goto again;
> if (*p == '#')
> goto again;
> - p[llen] = '\0';
> - if (!(cp = strpbrk(p, "#\n")))
> - goto again;
> - *cp = '\0';
> + p[llen - 1] = '\0';
> + if (cp = strpbrk(p, "#"))
> + *cp = '\0';
> if (!(cp = strpbrk(p, " \t")))
> goto again;
> *cp++ = '\0';
>
What about checking whether p[len -1] is blankish first?
Joerg
From: christos@zoulas.com (Christos Zoulas)
To: gnats-bugs@NetBSD.org, lib-bug-people@netbsd.org,
gnats-admin@netbsd.org, netbsd-bugs@netbsd.org, buhrow@nfbcal.org
Cc:
Subject: Re: lib/48926: gethostbyname(3) doesn't appear to work under NetBSD-5.2 and NetBSD-6.1.x when hostnames appear only in /etc/hosts
Date: Mon, 23 Jun 2014 11:34:30 -0400
On Jun 23, 3:30pm, joerg@britannica.bec.de (Joerg Sonnenberger) wrote:
-- Subject: Re: lib/48926: gethostbyname(3) doesn't appear to work under NetB
| What about checking whether p[len -1] is blankish first?
And len != 0 for the paranoid.
christos
From: "D'Arcy J.M. Cain" <darcy@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: christos@zoulas.com, lib-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, buhrow@nfbcal.org
Subject: Re: lib/48926: gethostbyname(3) doesn't appear to work under
NetBSD-5.2 and NetBSD-6.1.x when hostnames appear only in /etc/hosts
Date: Mon, 23 Jun 2014 12:18:55 -0400
On Mon, 23 Jun 2014 15:35:00 +0000 (UTC)
christos@zoulas.com (Christos Zoulas) wrote:
> | What about checking whether p[len -1] is blankish first?
Especially since comment processing can make this situation more likely.
Consider "BlahBlahBlah#oops\n"
> And len != 0 for the paranoid.
Or do it all in one line.
while (len && isspace(p[len - 1])) p[--len] = 0
Also trims blanks at end of line.
--
D'Arcy J.M. Cain <darcy@NetBSD.org>
http://www.NetBSD.org/ IM:darcy@Vex.Net
From: Brian Buhrow <buhrow@nfbcal.org>
To: gnats-bugs@NetBSD.org, lib-bug-people@NetBSD.org, gnats-admin@NetBSD.org,
netbsd-bugs@NetBSD.org
Cc: buhrow@nfbcal.org
Subject: Re: lib/48926: gethostbyname(3) doesn't appear to work under NetBSD-5.2 and NetBSD-6.1.x when hostnames appear only in /etc/hosts
Date: Mon, 23 Jun 2014 09:58:03 -0700
Hello. There are two versions of this fix that I've provided.
Both restore working behavior as it existed in NetBSD-5.1 and before. Here
are some assumptions that I see in the historical code that has existed for
years:
1. It has been the case for a long time that lines not ending with a
newline character would not get parsed correctly.
2. The code assumes the fields on the lines it processes are divided up
into null terminated strings before it begins its work.
The bug introduced with the changes recently added to gethnamaddr.c in
the netbsd-5 branch is that prior to that change, the gethostent_r(),
formerly _gethtent(), used fgets() to fetch the lines from the hosts file.
Fgets() returns a null terminated string that contains only the line
requested. Fgetln(), which is now used in the current version, returns a
pointer to the first character of the requested line, but the string
potentially contains the contents of the rest of the file and certainly
more than just the current line. In order to deal with that, the new code
puts a NULL character at the end of the current line. Unfortunately, the
character after the end of the current line is the first character of the
next line of the file. So, there are 3 ways, as I see it, to fix this
problem:
1. Save and restore the first character of the next line before and after
processing. this is what my first patch does. The patch works fine but is
more invasive than the second patch.
2. The second patch eats the last character of the current line and avoids
having to save and restore any values. By the time we get to the place
where my patch takes effect in the code, we know we're dealing with a line
that's not just a comment line, and the code assumes the line either ends
with a "#" or a newline character, and if it doesn't end with either of
those two things, the line is ignored. The second patch I provided still
does comment processing, assuming the comment begins with a "#" character.
It does assume the last character of the current line is a "newline", but
that's not a change from the existing code.
3. One could create a temporary string by using something like strndup()
to copy the current line into it and then process that, but my feeling is
that this would be an expensive operation for not much gain, plus, the
potential for memory leaks is increased.
So, patch 1 addresses Christos's objection that patch 2 eats the last
character of the current line. However, I don't see that as a big problem
since if the line doesn't end in a newline or a "#" character, it won't get
processed anyway. I could live with either patch, but I think patch 2 is
easier to understand and doesn't break anything that wasn't already broken
in the pre-existing versions of the code. In fact, as I think about it,
fgetln() assumes a line ends with a "newline" and so won't even give back a
length value that's even close to correct if the lines it's working on
don't end in newline. and, just for completeness, I think that's also true
for fparseln(), but I could be mistaken on that.
-thanks
-Brian
From: Joerg Sonnenberger <joerg@britannica.bec.de>
To: gnats-bugs@NetBSD.org
Cc: lib-bug-people@netbsd.org, gnats-admin@netbsd.org,
netbsd-bugs@netbsd.org, buhrow@nfbcal.org
Subject: Re: lib/48926: gethostbyname(3) doesn't appear to work under
NetBSD-5.2 and NetBSD-6.1.x when hostnames appear only in /etc/hosts
Date: Mon, 23 Jun 2014 20:02:01 +0200
On Mon, Jun 23, 2014 at 05:00:00PM +0000, Brian Buhrow wrote:
> 2. The second patch eats the last character of the current line and avoids
> having to save and restore any values. By the time we get to the place
> where my patch takes effect in the code, we know we're dealing with a line
> that's not just a comment line, and the code assumes the line either ends
> with a "#" or a newline character, and if it doesn't end with either of
> those two things, the line is ignored.
The test for blankish would fix that as well. You don't need to loop,
just check if the last character can be replaced and otherwise keep it.
That just makes it work for files without trailing newline.
Joerg
State-Changed-From-To: open->pending-pullups
State-Changed-By: buhrow@NetBSD.org
State-Changed-When: Mon, 23 Jun 2014 23:02:54 +0000
State-Changed-Why:
Requested pullups to the NetBSD-5 and NetBSD-6 branches, since the current state of those
branches is affected by this bug.
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: lib/48926 (Hostnames only appearing in /etc/hosts are not
reachable using gethostbyname(3) or gethostbyname2(3))
Date: Sat, 27 Dec 2014 23:46:36 +0000
On Mon, Jun 23, 2014 at 11:02:55PM +0000, buhrow@NetBSD.org wrote:
> Synopsis: Hostnames only appearing in /etc/hosts are not reachable using gethostbyname(3) or gethostbyname2(3)
>
> State-Changed-From-To: open->pending-pullups
> State-Changed-By: buhrow@NetBSD.org
> State-Changed-When: Mon, 23 Jun 2014 23:02:54 +0000
> State-Changed-Why:
> Requested pullups to the NetBSD-5 and NetBSD-6 branches, since the current state of those
> branches is affected by this bug.
I see pullup-6 #1085 and pullup-5 #1910, which haven't been processed
yet. Is that all of it?
--
David A. Holland
dholland@netbsd.org
From: Brian Buhrow <buhrow@nfbcal.org>
To: gnats-bugs@NetBSD.org, lib-bug-people@NetBSD.org, gnats-admin@NetBSD.org,
netbsd-bugs@NetBSD.org
Cc: buhrow@nfbcal.org
Subject: Re: lib/48926 (Hostnames only appearing in /etc/hosts are not reachable using gethostbyname(3) or gethostbyname2(3))
Date: Mon, 29 Dec 2014 00:17:14 -0800
hello. Yes, just the two pullup requests. I haven't checked
NetBSD-7, but I think it got fixed before it branched.
-thanks
-Brian
On Dec 27, 11:50pm, David Holland wrote:
} Subject: Re: lib/48926 (Hostnames only appearing in /etc/hosts are not rea
} The following reply was made to PR lib/48926; it has been noted by GNATS.
}
} From: David Holland <dholland-bugs@netbsd.org>
} To: gnats-bugs@NetBSD.org
} Cc:
} Subject: Re: lib/48926 (Hostnames only appearing in /etc/hosts are not
} reachable using gethostbyname(3) or gethostbyname2(3))
} Date: Sat, 27 Dec 2014 23:46:36 +0000
}
} On Mon, Jun 23, 2014 at 11:02:55PM +0000, buhrow@NetBSD.org wrote:
} > Synopsis: Hostnames only appearing in /etc/hosts are not reachable using gethostbyname(3) or gethostbyname2(3)
} >
} > State-Changed-From-To: open->pending-pullups
} > State-Changed-By: buhrow@NetBSD.org
} > State-Changed-When: Mon, 23 Jun 2014 23:02:54 +0000
} > State-Changed-Why:
} > Requested pullups to the NetBSD-5 and NetBSD-6 branches, since the current state of those
} > branches is affected by this bug.
}
} I see pullup-6 #1085 and pullup-5 #1910, which haven't been processed
} yet. Is that all of it?
}
} --
} David A. Holland
} dholland@netbsd.org
}
>-- End of excerpt from David Holland
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: lib/48926 (Hostnames only appearing in /etc/hosts are not
reachable using gethostbyname(3) or gethostbyname2(3))
Date: Mon, 29 Dec 2014 09:24:29 +0000
On Mon, Dec 29, 2014 at 08:20:02AM +0000, Brian Buhrow wrote:
> Yes, just the two pullup requests.
Ok, good to know for sure. I have no idea what the holdup is; maybe
you or I should ping releng.
> I haven't checked
> NetBSD-7, but I think it got fixed before it branched.
If those are the ones, then yes.
--
David A. Holland
dholland@netbsd.org
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: lib/48926 (Hostnames only appearing in /etc/hosts are not
reachable using gethostbyname(3) or gethostbyname2(3))
Date: Sun, 12 Jun 2016 03:30:35 +0000
On Sat, Dec 27, 2014 at 11:50:01PM +0000, David Holland wrote:
> I see pullup-6 #1085 and pullup-5 #1910, which haven't been processed
> yet. Is that all of it?
pullup-6 #1085 still hasn't been done, and there's nothing in the
ticket to suggest why...
--
David A. Holland
dholland@netbsd.org
From: "Soren Jacobsen" <snj@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/48926 CVS commit: [netbsd-6] src/lib/libc/net
Date: Tue, 21 Jun 2016 19:39:25 +0000
Module Name: src
Committed By: snj
Date: Tue Jun 21 19:39:25 UTC 2016
Modified Files:
src/lib/libc/net [netbsd-6]: gethnamaddr.c
Log Message:
Pull up following revision(s) (requested by buhrow in ticket #1085):
src/lib/libc/net/gethnamaddr.c: patch
Fix parsing of /etc/hosts. PR lib/48926.
To generate a diff of this commit:
cvs rdiff -u -r1.77.4.2 -r1.77.4.3 src/lib/libc/net/gethnamaddr.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: pending-pullups->closed
State-Changed-By: snj@NetBSD.org
State-Changed-When: Wed, 22 Jun 2016 08:42:20 +0000
State-Changed-Why:
Pulled up.
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.39 2013/11/01 18:47:49 spz Exp $
$NetBSD: gnats_config.sh,v 1.8 2006/05/07 09:23:38 tsutsui Exp $
Copyright © 1994-2014
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.