NetBSD Problem Report #42917

From ken@mitana.nanohz.org  Wed Mar  3 21:54:33 2010
Return-Path: <ken@mitana.nanohz.org>
Received: from mail.netbsd.org (mail.netbsd.org [204.152.190.11])
	by www.NetBSD.org (Postfix) with ESMTP id 1728363B873
	for <gnats-bugs@gnats.NetBSD.org>; Wed,  3 Mar 2010 21:54:33 +0000 (UTC)
Message-Id: <20100303203545.18DB4A8BD@mitana.nanohz.org>
Date: Wed,  3 Mar 2010 14:35:45 -0600 (CST)
From: kamada@nanohz.org
Reply-To: kamada@nanohz.org
To: gnats-bugs@gnats.NetBSD.org
Cc: kamada@nanohz.org
Subject: ftp(1) fails to preserve HTTP modfication time in non-C locales
X-Send-Pr-Version: 3.95

>Number:         42917
>Category:       bin
>Synopsis:       ftp(1) fails to preserve HTTP modfication time in non-C locales
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    releng
>State:          closed
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Wed Mar 03 21:55:00 +0000 2010
>Closed-Date:    Sat Jun 16 11:54:13 +0000 2012
>Last-Modified:  Fri Oct 31 18:50:04 +0000 2014
>Originator:     KAMADA Ken'ichi
>Release:        NetBSD 5.0 and -current
>Organization:
>Environment:
NetBSD mitana.nanohz.org 5.99.24 NetBSD 5.99.24 (MITANA) #200: Mon Mar  1 15:10:27 CST 2010  ken@mitana.nanohz.org:/usr/src/sys/arch/i386/compile/MITANA i386

>Description:
ftp(1) fails to parse Last-Modified HTTP header fields
when it runs in non-C (non-English, more precisely) locales.
This is because the representation of %a of strptime() depends on locales.
Patch attached.

FWIW, while fixing this, I found that rfc2822time() in util.c generates
locale-dependent day of the week in contrast to its name, "RFC 2822".
However, further reading of codes revealed that the function is
only used for human interaction regardless of its name, "RFC 2822" :-)
So I did not touch it.

>How-To-Repeat:
LANG=ja_JP.eucJP /usr/bin/ftp http://something-that-returns-Last-Modified
and look at the mtime of the resulting file.

>Fix:
The patch is against 5.0 and also applied cleanly to the current.

Index: extern.h
===================================================================
RCS file: /cvsroot/src/usr.bin/ftp/extern.h,v
retrieving revision 1.75
diff -u -r1.75 extern.h
--- extern.h	10 May 2008 00:05:31 -0000	1.75
+++ extern.h	3 Mar 2010 19:50:47 -0000
@@ -166,6 +166,7 @@
 void	opts(int, char **);
 void	newer(int, char **);
 void	page(int, char **);
+const char *parse_httpdate(struct tm *parsed, const char *httpdate);
 int	parserate(int, char **, int);
 char   *prompt(void);
 void	proxabort(int);
Index: fetch.c
===================================================================
RCS file: /cvsroot/src/usr.bin/ftp/fetch.c,v
retrieving revision 1.185
diff -u -r1.185 fetch.c
--- fetch.c	28 Apr 2008 20:24:13 -0000	1.185
+++ fetch.c	3 Mar 2010 19:50:48 -0000
@@ -931,21 +931,11 @@

 			} else if (match_token(&cp, "Last-Modified:")) {
 				struct tm parsed;
-				char *t;
+				const char *t;

 				memset(&parsed, 0, sizeof(parsed));
-							/* RFC1123 */
-				if ((t = strptime(cp,
-						"%a, %d %b %Y %H:%M:%S GMT",
-						&parsed))
-							/* RFC0850 */
-				    || (t = strptime(cp,
-						"%a, %d-%b-%y %H:%M:%S GMT",
-						&parsed))
-							/* asctime */
-				    || (t = strptime(cp,
-						"%a, %b %d %H:%M:%S %Y",
-						&parsed))) {
+				t = parse_httpdate(&parsed, cp);
+				if (t != NULL) {
 					parsed.tm_isdst = -1;
 					if (*t == '\0')
 						mtime = timegm(&parsed);
Index: util.c
===================================================================
RCS file: /cvsroot/src/usr.bin/ftp/util.c,v
retrieving revision 1.148
diff -u -r1.148 util.c
--- util.c	13 Aug 2008 04:59:13 -0000	1.148
+++ util.c	3 Mar 2010 19:50:49 -0000
@@ -85,6 +85,7 @@
 #include <signal.h>
 #include <libgen.h>
 #include <limits.h>
+#include <locale.h>
 #include <netdb.h>
 #include <stdio.h>
 #include <stdlib.h>
@@ -791,6 +792,31 @@
 }

 /*
+ * Parse HTTP-date as per RFC 2616.
+ * Return a pointer to the next character of the consumed date string,
+ * or NULL if failed.
+ */
+const char *
+parse_httpdate(struct tm *parsed, const char *httpdate)
+{
+	const char *t;
+	const char *curlocale;
+
+	/* The representation of %a depends on the current locale. */
+	curlocale = setlocale(LC_TIME, NULL);
+	(void)setlocale(LC_TIME, "C");
+								/* RFC1123 */
+	if ((t = strptime(httpdate, "%a, %d %b %Y %H:%M:%S GMT", parsed)) ||
+								/* RFC0850 */
+	    (t = strptime(httpdate, "%a, %d-%b-%y %H:%M:%S GMT", parsed)) ||
+								/* asctime */
+	    (t = strptime(httpdate, "%a, %b %d %H:%M:%S %Y", parsed)))
+		;			/* do nothing */
+	(void)setlocale(LC_TIME, curlocale);
+	return t;
+}
+
+/*
  * Update global `localcwd', which contains the state of the local cwd
  */
 void

>Release-Note:

>Audit-Trail:

Responsible-Changed-From-To: bin-bug-people->lukem
Responsible-Changed-By: lukem@NetBSD.org
Responsible-Changed-When: Thu, 04 Mar 2010 07:11:33 +0000
Responsible-Changed-Why:
I'll look after this


State-Changed-From-To: open->feedback
State-Changed-By: lukem@NetBSD.org
State-Changed-When: Thu, 04 Mar 2010 21:53:08 +0000
State-Changed-Why:
A variation of your fix has been applied to -current.
Are you able to verify that it works correclty ?


From: KAMADA Ken'ichi <kamada@nanohz.org>
To: gnats-bugs@NetBSD.org
Cc: lukem@NetBSD.org,
	kamada@nanohz.org
Subject: Re: bin/42917 (ftp(1) fails to preserve HTTP modfication time in non-C locales)
Date: Fri, 05 Mar 2010 11:11:15 -0600

 lukem wrote:
 > 
 > A variation of your fix has been applied to -current.
 > Are you able to verify that it works correclty ?

 Worked fine, thank you.

State-Changed-From-To: feedback->pending-pullups
State-Changed-By: lukem@NetBSD.org
State-Changed-When: Sat, 10 Apr 2010 14:41:28 +0000
State-Changed-Why:
pullups requeseted


Responsible-Changed-From-To: lukem->releng
Responsible-Changed-By: lukem@NetBSD.org
Responsible-Changed-When: Sat, 16 Jun 2012 06:38:25 +0000
Responsible-Changed-Why:
.


State-Changed-From-To: pending-pullups->closed
State-Changed-By: martin@NetBSD.org
State-Changed-When: Sat, 16 Jun 2012 11:54:13 +0000
State-Changed-Why:
All pullups have already been processd.
(Hint: if you move a PR to penging-pullups state, list the pullup request numbers, so the PR can easily be identified when the requests are processed)


From: "S.P.Zeidler" <spz@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/42917 CVS commit: pkgsrc/net/tnftp/files
Date: Fri, 31 Oct 2014 18:47:26 +0000

 Module Name:	pkgsrc
 Committed By:	spz
 Date:		Fri Oct 31 18:47:26 UTC 2014

 Update of /cvsroot/pkgsrc/net/tnftp/files
 In directory ivanova.netbsd.org:/tmp/cvs-serv2789

 Log Message:
 Fri Oct 31 04:07:38 UTC 2014    lukem

         * Release as "tnftp 20141031".

         * Merge NetBSD usr.bin/ftp from 20130220 to 20141026:
                 - Don't pay attention to special characters if they don't
                   come from the command line (from jmcneill).
                   Fixes CVE-2014-8517.
                 - pr/34796: Hauke Fath: ftp does not timeout on http fetches.

 Sun May  5 13:51:47 UTC 2013    lukem

         * Release as "tnftp 20130505"

         * Implement --enable-ssl (and --with-openssl) to enable
           https:// fetch support.

         * Merge NetBSD ftp from 20090520 to 20130220.  Changes:
                 - https:// support.
                   NetBSD problem report 47276 from NONAKA Kimihiro.
                 - Allow -R to restart non-existent ftp:// URIs.
                 - Don't assume AF_INET support is available.
                   FreeBSD problem report 162661.
                 - Parse HTTP 'Date' entries in the `C' locale rather than the
                   user's.
                   NetBSD problem report 42917 from KAMADA Ken'ichi.
                 - Improve error handling when parsing of URI scheme.
                 - Silence connection warnings to multi-homed hosts in
                   non-verbose mode.
                 - Fix compile warnings.
                 - In ftpvis(), prevent incomplete escape sequences at end of
                   dst, and ensure NUL-termination of dst.
                   Fix from Uwe Stuehler and Stefan Sperling, via Marc Balmer.
                 - When using the response to SYST to decide whether to
                   default to 'binary' be a lot less specific.

         * Replace glob with newer copy from NetBSD that does not suffer
           from DoS exhaustion attacks.
           Fix in NetBSD from Maksymilian Arciemowicz.  See CVE-2011-0418

 Tue Jan 12 06:58:15 UTC 2010    lukem

         * Release as "tnftp 20100108"

         * Rename onoff() argument "bool" to "val".

 Tue Jan  5 09:12:01 UTC 2010    lukem

         * If ARG_MAX isn't defined, use the result from sysconf(_SC_ARG_MAX).
           Fixes build when using newer glibc.

         * Add libnetbsd.la to the LIBADD for libedit.
           Fix provided by Adam Sampson.

 Mon Jan  4 06:28:07 UTC 2010    lukem

         * Distribute various files not shipped by default automake rules,
           to use 'make dist' instead of 'cvs export'.

 Wed Dec 30 00:12:47 UTC 2009    lukem

         * Release as "tnftp 20091122"

 Sun Nov 15 10:14:44 UTC 2009    lukem

         * Merge NetBSD ftp from 20090520 to 20090915.  Change:
                 - Rename internal getline() to get_line() to avoid
                   conflict with libc with former.
                 - Avoid a NULL dereference in an error message.

 Sat Nov 14 09:21:19 UTC 2009    lukem

         * Convert to automake & libtool.

 Sat Jun  6 07:17:38 UTC 2009    lukem

         * Release as "tnftp 20090606"

 Fri May 22 01:11:15 UTC 2009    lukem

         * configure fixes:
           - Add the time.h headers to accheck_includes, for the strptime check.
           - Remove the check for el_init in libedit; we're always replacing
             the library and the presence of strvis() in some versions
             confuses other checks.

 Wed May 20 13:47:43 UTC 2009    lukem

         * Release as "tnftp 20090520"

         * Merge NetBSD ftp from 20070722 to 20090520.  Changes:
             - Only attempt to el_parse() a command unknown by the default
               parser if editing is enabled.
               Fixes pr 38589.
             - Turn off the alarmtimer before resetting the SIGALRM handler
               back to SIG_DFL.
               Fixes pr 35630.
             - Add epsv6 and epsv to disable extended passive mode for ipv6 or
               both ipv4 and ipv6 respectively.  This hack is due to our
               friends a Juniper Networks who break epsv in ipv6.
               Should be fixed in ScreenOS 6.2.X.
             - Improve parsing of chunked transfer chunks per RFC2616:
               - more stringent chunk-size parsing
               - ignore optional trailing ';chunk-ext' stuff, instead of barfing
               - detect EOF before final \r\n.
             - Use the service name to getaddrinfo() (along with the host
               name), so that features such as DNS Service Discovery have a
               better chance of working.
               Display the service name in various status & error messages.
             - Don't getservbyname() the :port component of a URL; RFC 3986
               says it's just an unsigned number, not a service name.
             - Fix numerous WARNS=4 issues (-Wcast-qual -Wsign-compare).
             - Fix -Wshadow issues
             - Update copyrights
             - Remove clause 3 and 4 from TNF licenses
             - Rename HAVE_STRUCT_SOCKADDR_SA_LEN to
               HAVE_STRUCT_SOCKADDR_IN_SIN_LEN to accurately reflect the
               structure member being used.
             - Use AF_INET instead of AF_UNSPEC as the default family if
               !defined(INET6).

         * configure improvements:
           - Style tweaks.
           - Use AC_LANG_PROGRAM() instead of AC_LANG_SOURCE()
           - Add a check for strptime() requiring separators between
             conversions, and use our replacement one if it does.

 Sat Dec 20 15:28:24 UTC 2008    lukem

         * configure improvements:
           - Move IPv6 check from tnftp.h to configure.ac (as per tnftpd).
           - Rework option descriptions.
           - Highlight when tests are for a specific option.
           - Move configuration results to the end of the file.
           - Display $prefix in configure results.

 Fri Aug 15 03:03:36 UTC 2008    lukem

         * Add a "Configuration results" display at the end of configure.
           Cosmetic tweaks.

 Fri Feb 29 09:45:56 UTC 2008    lukem

         * Support @EXEEXT@ for Cygwin (etc).

 Status:

 Vendor Tag:	tnftp
 Release Tags:	tnftp-20141031

 C pkgsrc/net/tnftp/files/ChangeLog
 N pkgsrc/net/tnftp/files/Makefile.am
 C pkgsrc/net/tnftp/files/todo
 U pkgsrc/net/tnftp/files/README
 C pkgsrc/net/tnftp/files/tnftp.h
 C pkgsrc/net/tnftp/files/aclocal.m4
 U pkgsrc/net/tnftp/files/INSTALL
 C pkgsrc/net/tnftp/files/THANKS
 U pkgsrc/net/tnftp/files/NEWS
 N pkgsrc/net/tnftp/files/tnftp_config.h.in
 C pkgsrc/net/tnftp/files/Makefile.in
 C pkgsrc/net/tnftp/files/configure
 C pkgsrc/net/tnftp/files/COPYING
 C pkgsrc/net/tnftp/files/configure.ac
 N pkgsrc/net/tnftp/files/buildaux/lt~obsolete.m4
 N pkgsrc/net/tnftp/files/buildaux/config.guess
 N pkgsrc/net/tnftp/files/buildaux/ltmain.sh
 N pkgsrc/net/tnftp/files/buildaux/install-sh
 N pkgsrc/net/tnftp/files/buildaux/ax_check_openssl.m4
 N pkgsrc/net/tnftp/files/buildaux/ltoptions.m4
 N pkgsrc/net/tnftp/files/buildaux/ltversion.m4
 N pkgsrc/net/tnftp/files/buildaux/ltsugar.m4
 N pkgsrc/net/tnftp/files/buildaux/missing
 N pkgsrc/net/tnftp/files/buildaux/depcomp
 N pkgsrc/net/tnftp/files/buildaux/config.sub
 N pkgsrc/net/tnftp/files/buildaux/libtool.m4
 C pkgsrc/net/tnftp/files/src/progressbar.c
 N pkgsrc/net/tnftp/files/src/Makefile.am
 C pkgsrc/net/tnftp/files/src/extern.h
 C pkgsrc/net/tnftp/files/src/util.c
 C pkgsrc/net/tnftp/files/src/domacro.c
 C pkgsrc/net/tnftp/files/src/ftp.c
 U pkgsrc/net/tnftp/files/src/ruserpass.c
 C pkgsrc/net/tnftp/files/src/version.h
 N pkgsrc/net/tnftp/files/src/ssl.c
 C pkgsrc/net/tnftp/files/src/fetch.c
 C pkgsrc/net/tnftp/files/src/progressbar.h
 N pkgsrc/net/tnftp/files/src/ssl.h
 C pkgsrc/net/tnftp/files/src/Makefile.in
 C pkgsrc/net/tnftp/files/src/cmds.c
 C pkgsrc/net/tnftp/files/src/ftp_var.h
 C pkgsrc/net/tnftp/files/src/ftp.1
 C pkgsrc/net/tnftp/files/src/cmdtab.c
 C pkgsrc/net/tnftp/files/src/complete.c
 C pkgsrc/net/tnftp/files/src/main.c
 U pkgsrc/net/tnftp/files/libnetbsd/getnameinfo.c
 U pkgsrc/net/tnftp/files/libnetbsd/strerror.c
 N pkgsrc/net/tnftp/files/libnetbsd/Makefile.am
 U pkgsrc/net/tnftp/files/libnetbsd/fseeko.c
 U pkgsrc/net/tnftp/files/libnetbsd/inet_ntop.c
 U pkgsrc/net/tnftp/files/libnetbsd/strlcpy.c
 U pkgsrc/net/tnftp/files/libnetbsd/timegm.c
 U pkgsrc/net/tnftp/files/libnetbsd/inet_pton.c
 C pkgsrc/net/tnftp/files/libnetbsd/strvis.c
 U pkgsrc/net/tnftp/files/libnetbsd/strdup.c
 U pkgsrc/net/tnftp/files/libnetbsd/strunvis.c
 U pkgsrc/net/tnftp/files/libnetbsd/snprintf.c
 U pkgsrc/net/tnftp/files/libnetbsd/err.c
 C pkgsrc/net/tnftp/files/libnetbsd/strptime.c
 C pkgsrc/net/tnftp/files/libnetbsd/utimes.c
 U pkgsrc/net/tnftp/files/libnetbsd/getaddrinfo.c
 C pkgsrc/net/tnftp/files/libnetbsd/sl_init.c
 U pkgsrc/net/tnftp/files/libnetbsd/ftpvis.h
 C pkgsrc/net/tnftp/files/libnetbsd/setprogname.c
 C pkgsrc/net/tnftp/files/libnetbsd/glob.c
 C pkgsrc/net/tnftp/files/libnetbsd/dirname.c
 U pkgsrc/net/tnftp/files/libnetbsd/mkstemp.c
 U pkgsrc/net/tnftp/files/libnetbsd/strtoll.c
 C pkgsrc/net/tnftp/files/libnetbsd/ftpglob.h
 U pkgsrc/net/tnftp/files/libnetbsd/strlcat.c
 C pkgsrc/net/tnftp/files/libnetbsd/usleep.c
 C pkgsrc/net/tnftp/files/libnetbsd/Makefile.in
 U pkgsrc/net/tnftp/files/libnetbsd/strsep.c
 C pkgsrc/net/tnftp/files/libnetbsd/fgetln.c
 C pkgsrc/net/tnftp/files/libedit/filecomplete.h
 N pkgsrc/net/tnftp/files/libedit/Makefile.am
 U pkgsrc/net/tnftp/files/libedit/hist.h
 U pkgsrc/net/tnftp/files/libedit/refresh.c
 U pkgsrc/net/tnftp/files/libedit/chared.c
 U pkgsrc/net/tnftp/files/libedit/hist.c
 U pkgsrc/net/tnftp/files/libedit/tokenizer.c
 U pkgsrc/net/tnftp/files/libedit/sys.h
 U pkgsrc/net/tnftp/files/libedit/el.c
 U pkgsrc/net/tnftp/files/libedit/prompt.h
 U pkgsrc/net/tnftp/files/libedit/makelist.in
 U pkgsrc/net/tnftp/files/libedit/search.h
 U pkgsrc/net/tnftp/files/libedit/key.h
 U pkgsrc/net/tnftp/files/libedit/sig.c
 U pkgsrc/net/tnftp/files/libedit/search.c
 U pkgsrc/net/tnftp/files/libedit/term.c
 U pkgsrc/net/tnftp/files/libedit/term.h
 U pkgsrc/net/tnftp/files/libedit/editrc.5
 U pkgsrc/net/tnftp/files/libedit/chared.h
 C pkgsrc/net/tnftp/files/libedit/filecomplete.c
 U pkgsrc/net/tnftp/files/libedit/key.c
 C pkgsrc/net/tnftp/files/libedit/readline.c
 U pkgsrc/net/tnftp/files/libedit/sig.h
 C pkgsrc/net/tnftp/files/libedit/Makefile.in
 U pkgsrc/net/tnftp/files/libedit/parse.c
 U pkgsrc/net/tnftp/files/libedit/common.c
 U pkgsrc/net/tnftp/files/libedit/parse.h
 U pkgsrc/net/tnftp/files/libedit/prompt.c
 U pkgsrc/net/tnftp/files/libedit/emacs.c
 U pkgsrc/net/tnftp/files/libedit/el.h
 U pkgsrc/net/tnftp/files/libedit/history.c
 U pkgsrc/net/tnftp/files/libedit/tty.h
 U pkgsrc/net/tnftp/files/libedit/map.h
 U pkgsrc/net/tnftp/files/libedit/refresh.h
 U pkgsrc/net/tnftp/files/libedit/vi.c
 U pkgsrc/net/tnftp/files/libedit/map.c
 U pkgsrc/net/tnftp/files/libedit/editline.3
 U pkgsrc/net/tnftp/files/libedit/tty.c
 U pkgsrc/net/tnftp/files/libedit/histedit.h
 C pkgsrc/net/tnftp/files/libedit/read.h
 U pkgsrc/net/tnftp/files/libedit/read.c
 C pkgsrc/net/tnftp/files/libedit/readline/readline.h

 41 conflicts created by this import.
 Use the following command to help the merge:

 	cvs checkout -jtnftp:yesterday -jtnftp pkgsrc/net/tnftp/files

>Unformatted:

Home	PR Database Search