NetBSD Problem Report #58581
From www@netbsd.org Sun Aug 11 14:03:06 2024
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
key-exchange X25519 server-signature RSA-PSS (2048 bits)
client-signature RSA-PSS (2048 bits))
(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
by mollari.NetBSD.org (Postfix) with ESMTPS id 684451A9244
for <gnats-bugs@gnats.NetBSD.org>; Sun, 11 Aug 2024 14:03:06 +0000 (UTC)
Message-Id: <20240811140305.3370A1A9243@mollari.NetBSD.org>
Date: Sun, 11 Aug 2024 14:03:05 +0000 (UTC)
From: campbell+netbsd@mumble.net
Reply-To: campbell+netbsd@mumble.net
To: gnats-bugs@NetBSD.org
Subject: ftp(1) should allow specifying header fields in http requests
X-Send-Pr-Version: www-1.0
>Number: 58581
>Category: bin
>Synopsis: ftp(1) should allow specifying header fields in http requests
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: bin-bug-people
>State: closed
>Class: change-request
>Submitter-Id: net
>Arrival-Date: Sun Aug 11 14:05:00 +0000 2024
>Closed-Date: Sun Oct 13 20:31:16 +0000 2024
>Last-Modified: Sun Oct 13 20:31:16 +0000 2024
>Originator: Taylor R Campbell
>Release: current, 10, 9, ...
>Organization:
The X-NetBSD: Fetchation
>Environment:
>Description:
It would be nice if you could add a custom header field to an http request.
For example, the Instance Metadata Service version 2 in Oracle Compute Infrastructure requires adding a header field `Authorization: Bearer Oracle' in order to prevent SSRF attacks which might expose secret keys.
Similarly, the IMDSv2 in Amazon EC2 requires an X-aws-ec2-metadata-token header field, populated with a token retrieved by another request made with an X-aws-ec2-metadata-token-ttl-seconds field.
Although you can do this with fancier http clients like curl(1), we might want to use these in rc scripts at first boot like /etc/rc.d/ec2_init, and it would be good if that worked only with what's available in the base system.
>How-To-Repeat:
try to use a service that requires a custom header field
>Fix:
Add a `-H <headerfield>' option to ftp(1) like curl(1) has.
>Release-Note:
>Audit-Trail:
From: Sunil Nimmagadda <sunil@nimmagadda.net>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/58581: ftp(1) should allow specifying header fields in http
requests
Date: Sun, 11 Aug 2024 21:15:22 +0530
campbell+netbsd@mumble.net writes:
>>Number: 58581
>>Category: bin
>>Synopsis: ftp(1) should allow specifying header fields in http requests
>>Confidential: no
>>Severity: serious
>>Priority: medium
>>Responsible: bin-bug-people
>>State: open
>>Class: change-request
>>Submitter-Id: net
>>Arrival-Date: Sun Aug 11 14:05:00 +0000 2024
>>Originator: Taylor R Campbell
>>Release: current, 10, 9, ...
>>Organization:
> The X-NetBSD: Fetchation
>>Environment:
>>Description:
> It would be nice if you could add a custom header field to an http request.
>
> For example, the Instance Metadata Service version 2 in Oracle Compute
> Infrastructure requires adding a header field `Authorization: Bearer
> Oracle' in order to prevent SSRF attacks which might expose secret
> keys.
>
> Similarly, the IMDSv2 in Amazon EC2 requires an X-aws-ec2-metadata-token header field, populated with a token retrieved by another request made with an X-aws-ec2-metadata-token-ttl-seconds field.
>
> Although you can do this with fancier http clients like curl(1), we
> might want to use these in rc scripts at first boot like
> /etc/rc.d/ec2_init, and it would be good if that worked only with
> what's available in the base system.
>>How-To-Repeat:
> try to use a service that requires a custom header field
>>Fix:
> Add a `-H <headerfield>' option to ftp(1) like curl(1) has.
Initial attempt at adding a custom HTTP request header...
$ ftp -H 'Authorization: Bearer Oracle' https://example.com/foo
diff --git a/usr.bin/ftp/fetch.c b/usr.bin/ftp/fetch.c
--- a/usr.bin/ftp/fetch.c
+++ b/usr.bin/ftp/fetch.c
@@ -882,6 +882,9 @@
print_host(fin, ui);
fetch_printf(fin, "Accept: */*\r\n");
fetch_printf(fin, "Connection: close\r\n");
+ if (http_header) {
+ fetch_printf(fin, "%s\r\n", http_header);
+ }
if (restart_point) {
fputs(leading, ttyout);
fetch_printf(fin, "Range: bytes=" LLF "-\r\n",
diff --git a/usr.bin/ftp/ftp_var.h b/usr.bin/ftp/ftp_var.h
--- a/usr.bin/ftp/ftp_var.h
+++ b/usr.bin/ftp/ftp_var.h
@@ -255,6 +255,7 @@
GLOBAL int epsv6bad; /* EPSV doesn't work on the current server */
GLOBAL int editing; /* command line editing enabled */
GLOBAL int features[FEAT_max]; /* remote FEATures supported */
+GLOBAL const char *http_header; /* Custom HTTP Request header */
#ifndef NO_EDITCOMPLETE
GLOBAL EditLine *el; /* editline(3) status structure */
diff --git a/usr.bin/ftp/main.c b/usr.bin/ftp/main.c
--- a/usr.bin/ftp/main.c
+++ b/usr.bin/ftp/main.c
@@ -267,7 +267,7 @@
}
}
- while ((ch = getopt(argc, argv, ":46Aab:defginN:o:pP:q:r:Rs:tT:u:vVx:")) != -1) {
+ while ((ch = getopt(argc, argv, ":46Aab:defgH:inN:o:pP:q:r:Rs:tT:u:vVx:")) != -1) {
switch (ch) {
case '4':
family = AF_INET;
@@ -315,6 +315,10 @@
doglob = 0;
break;
+ case 'H':
+ http_header = ftp_strdup(optarg);
+ break;
+
case 'i':
interactive = 0;
break;
From: Sunil Nimmagadda <sunil@nimmagadda.net>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: bin/58581: ftp(1) should allow specifying header fields in http
requests
Date: Wed, 25 Sep 2024 09:43:37 +0530
gnats-admin@netbsd.org writes:
> The following reply was made to PR bin/58581; it has been noted by GNATS.
>
> From: Sunil Nimmagadda <sunil@nimmagadda.net>
> To: gnats-bugs@netbsd.org
> Cc:
> Subject: Re: bin/58581: ftp(1) should allow specifying header fields in http
> requests
> Date: Sun, 11 Aug 2024 21:15:22 +0530
>
> campbell+netbsd@mumble.net writes:
>
> >>Number: 58581
> >>Category: bin
> >>Synopsis: ftp(1) should allow specifying header fields in http requests
> >>Confidential: no
> >>Severity: serious
> >>Priority: medium
> >>Responsible: bin-bug-people
> >>State: open
> >>Class: change-request
> >>Submitter-Id: net
> >>Arrival-Date: Sun Aug 11 14:05:00 +0000 2024
> >>Originator: Taylor R Campbell
> >>Release: current, 10, 9, ...
> >>Organization:
> > The X-NetBSD: Fetchation
> >>Environment:
> >>Description:
> > It would be nice if you could add a custom header field to an http request.
> >
> > For example, the Instance Metadata Service version 2 in Oracle Compute
> > Infrastructure requires adding a header field `Authorization: Bearer
> > Oracle' in order to prevent SSRF attacks which might expose secret
> > keys.
> >
> > Similarly, the IMDSv2 in Amazon EC2 requires an X-aws-ec2-metadata-token header field, populated with a token retrieved by another request made with an X-aws-ec2-metadata-token-ttl-seconds field.
> >
> > Although you can do this with fancier http clients like curl(1), we
> > might want to use these in rc scripts at first boot like
> > /etc/rc.d/ec2_init, and it would be good if that worked only with
> > what's available in the base system.
> >>How-To-Repeat:
> > try to use a service that requires a custom header field
> >>Fix:
> > Add a `-H <headerfield>' option to ftp(1) like curl(1) has.
Updated diff with two changes...
- Support specifiying multiple -H <hdr> option.
- Document new -H option in manpage.
diff --git a/usr.bin/ftp/fetch.c b/usr.bin/ftp/fetch.c
index a2ccd8c3897a..f714bb6b6f2b 100644
--- a/usr.bin/ftp/fetch.c
+++ b/usr.bin/ftp/fetch.c
@@ -865,6 +865,7 @@ print_get(FETCH *fin, int hasleading, int isproxy, const struct urlinfo *oui,
const struct urlinfo *ui)
{
const char *leading = hasleading ? ", " : " (";
+ struct entry *np;
if (isproxy) {
if (verbose) {
@@ -882,6 +883,9 @@ print_get(FETCH *fin, int hasleading, int isproxy, const struct urlinfo *oui,
print_host(fin, ui);
fetch_printf(fin, "Accept: */*\r\n");
fetch_printf(fin, "Connection: close\r\n");
+ SLIST_FOREACH(np, &custom_headers, entries)
+ fetch_printf(fin, "%s\r\n", np->header);
+
if (restart_point) {
fputs(leading, ttyout);
fetch_printf(fin, "Range: bytes=" LLF "-\r\n",
diff --git a/usr.bin/ftp/ftp.1 b/usr.bin/ftp/ftp.1
index 9d2319884706..c0663f800266 100644
--- a/usr.bin/ftp/ftp.1
+++ b/usr.bin/ftp/ftp.1
@@ -67,6 +67,7 @@
.Nm
.Op Fl 46AadefginpRtVv\&?
.Op Fl b Ar bufsize
+.Op Fl H Ar hdr
.Op Fl N Ar netrc
.Op Fl o Ar output
.Op Fl P Ar port
@@ -223,6 +224,10 @@ or
proxies.
.It Fl g
Disables file name globbing.
+.It Fl H Ar hdr
+Include
+.Ar hdr
+string as a custom HTTP header for an HTTP request.
.It Fl i
Turns off interactive prompting during
multiple file transfers.
diff --git a/usr.bin/ftp/ftp_var.h b/usr.bin/ftp/ftp_var.h
index 1c7448dc0a75..2c12ae82d9ed 100644
--- a/usr.bin/ftp/ftp_var.h
+++ b/usr.bin/ftp/ftp_var.h
@@ -101,6 +101,7 @@
#endif
#include <sys/param.h>
+#include <sys/queue.h>
#include <netinet/in.h>
#include <arpa/inet.h>
@@ -165,6 +166,14 @@ enum {
FEAT_max
};
+/*
+ * Custom HTTP headers
+ */
+struct entry {
+ SLIST_ENTRY(entry) entries;
+ const char *header;
+};
+SLIST_HEAD(http_headers, entry);
/*
* Global defines
@@ -320,8 +329,9 @@ GLOBAL FILE *cin;
GLOBAL FILE *cout;
GLOBAL int data;
-extern struct cmd cmdtab[];
-extern struct option optiontab[];
+extern struct cmd cmdtab[];
+extern struct option optiontab[];
+extern struct http_headers custom_headers;
extern size_t ftp_buflen;
diff --git a/usr.bin/ftp/main.c b/usr.bin/ftp/main.c
index 82e0b6656fd7..54fc1ca3cf64 100644
--- a/usr.bin/ftp/main.c
+++ b/usr.bin/ftp/main.c
@@ -134,6 +134,8 @@ static int usage(void);
static int usage_help(void);
static void setupoption(const char *, const char *, const char *);
+struct http_headers custom_headers;
+
int
main(int volatile argc, char **volatile argv)
{
@@ -267,7 +269,8 @@ main(int volatile argc, char **volatile argv)
}
}
- while ((ch = getopt(argc, argv, ":46Aab:defginN:o:pP:q:r:Rs:tT:u:vVx:")) != -1) {
+ SLIST_INIT(&custom_headers);
+ while ((ch = getopt(argc, argv, ":46Aab:defgH:inN:o:pP:q:r:Rs:tT:u:vVx:")) != -1) {
switch (ch) {
case '4':
family = AF_INET;
@@ -315,6 +318,13 @@ main(int volatile argc, char **volatile argv)
doglob = 0;
break;
+ case 'H':
+ struct entry *p;
+ p = ftp_malloc(sizeof *p);
+ p->header = ftp_strdup(optarg);
+ SLIST_INSERT_HEAD(&custom_headers, p, entries);
+ break;
+
case 'i':
interactive = 0;
break;
From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/58581 CVS commit: src/usr.bin/ftp
Date: Wed, 25 Sep 2024 12:55:40 -0400
Module Name: src
Committed By: christos
Date: Wed Sep 25 16:55:40 UTC 2024
Modified Files:
src/usr.bin/ftp: fetch.c ftp.1 ftp_var.h main.c
Log Message:
PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.
To generate a diff of this commit:
cvs rdiff -u -r1.240 -r1.241 src/usr.bin/ftp/fetch.c
cvs rdiff -u -r1.155 -r1.156 src/usr.bin/ftp/ftp.1
cvs rdiff -u -r1.88 -r1.89 src/usr.bin/ftp/ftp_var.h
cvs rdiff -u -r1.131 -r1.132 src/usr.bin/ftp/main.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: open->needs-pullups
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Fri, 27 Sep 2024 16:34:22 +0000
State-Changed-Why:
Cool, thanks, Sunil! Would also be nice if we had automatic tests for
this, maybe with bozohttpd and a simple cgi script run under atf.
Anyway, let's get this pulled up to 10.
From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/58581 CVS commit: src/usr.bin/ftp
Date: Fri, 27 Sep 2024 16:31:36 +0000
Module Name: src
Committed By: riastradh
Date: Fri Sep 27 16:31:36 UTC 2024
Modified Files:
src/usr.bin/ftp: ftp.1
Log Message:
ftp(1): Nix trailing whitespace in man page.
No functional change intended.
PR bin/58581: ftp(1) should allow specifying header fields in http requests
To generate a diff of this commit:
cvs rdiff -u -r1.156 -r1.157 src/usr.bin/ftp/ftp.1
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
From: Sunil Nimmagadda <sunil@nimmagadda.net>
To: riastradh@NetBSD.org
Cc: gnats-admin@netbsd.org, netbsd-bugs@netbsd.org,
campbell+netbsd@mumble.net, gnats-bugs@netbsd.org
Subject: Re: bin/58581 (ftp(1) should allow specifying header fields in http
requests)
Date: Mon, 30 Sep 2024 13:10:23 +0530
--=-=-=
Content-Type: text/plain
riastradh@NetBSD.org writes:
> Synopsis: ftp(1) should allow specifying header fields in http requests
>
> State-Changed-From-To: open->needs-pullups
> State-Changed-By: riastradh@NetBSD.org
> State-Changed-When: Fri, 27 Sep 2024 16:34:22 +0000
> State-Changed-Why:
> Cool, thanks, Sunil! Would also be nice if we had automatic tests for
> this, maybe with bozohttpd and a simple cgi script run under atf.
>
> Anyway, let's get this pulled up to 10.
Sure, the attached diff has a testcase for the custom headers feature
using atf...
--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment; filename=custom_headers_tests.diff
diff --git a/tests/usr.bin/Makefile b/tests/usr.bin/Makefile
index d0456d50235b..3d30e6157de5 100644
--- a/tests/usr.bin/Makefile
+++ b/tests/usr.bin/Makefile
@@ -6,7 +6,7 @@
TESTSDIR= ${TESTSBASE}/usr.bin
TESTS_SUBDIRS= awk basename bzip2 cc cmp compress config cpio col cut \
- diff dirname error find fstat gdb grep gzip id indent \
+ diff dirname error find fstat ftp gdb grep gzip id indent \
infocmp jot ld locale m4 make mixerctl mkdep mtree nbperf \
netpgpverify patch pkill pr printf pwhash realpath rump_server \
shmif_dumpbus shmif_pcapin sdiff sed sort stat tar tmux tr \
diff --git a/tests/usr.bin/ftp/Makefile b/tests/usr.bin/ftp/Makefile
new file mode 100644
index 000000000000..b66e9fdeefde
--- /dev/null
+++ b/tests/usr.bin/ftp/Makefile
@@ -0,0 +1,8 @@
+# $NetBSD: Makefile,v 1.1 2020/06/24 10:05:07 jruoho Exp $
+
+.include <bsd.own.mk>
+
+TESTSDIR= ${TESTSBASE}/usr.bin/ftp
+TESTS_SH= t_custom_headers
+
+.include <bsd.test.mk>
diff --git a/tests/usr.bin/ftp/t_custom_headers.sh b/tests/usr.bin/ftp/t_custom_headers.sh
new file mode 100644
index 000000000000..03eddee22e54
--- /dev/null
+++ b/tests/usr.bin/ftp/t_custom_headers.sh
@@ -0,0 +1,34 @@
+atf_test_case custom_headers cleanup
+custom_headers_head()
+{
+ atf_require_prog ftp
+ atf_set "descr" "Check for custom HTTP headers"
+}
+
+HTTPD_PID=./.__httpd.pid
+custom_headers_body()
+{
+ # start httpd in daemon mode
+ atf_check -s exit:0 \
+ /usr/libexec/httpd -P $HTTPD_PID -I 8080 -b -C .sh /bin/sh \
+ -c $(atf_get_srcdir)/www/cgi-bin $(atf_get_srcdir)/www
+
+ atf_check -o inline:"example.com\n1000\n" \
+ -x "ftp -Vo- -H X-Origin:example.com \
+ -H X-Rate-Limit:1000 \
+ http://127.0.0.1:8080/cgi-bin/custom_headers.sh"
+}
+
+custom_headers_cleanup()
+{
+ if [ -f $HTTPD_PID ]; then
+ kill -9 $(cat $HTTPD_PID)
+ rm -f $HTTPD_PID
+ sleep 1
+ fi
+}
+
+atf_init_test_cases()
+{
+ atf_add_test_case custom_headers
+}
diff --git a/tests/usr.bin/ftp/www/cgi-bin/custom_headers.sh b/tests/usr.bin/ftp/www/cgi-bin/custom_headers.sh
new file mode 100755
index 000000000000..65428b79c352
--- /dev/null
+++ b/tests/usr.bin/ftp/www/cgi-bin/custom_headers.sh
@@ -0,0 +1,11 @@
+#! /bin/sh
+
+echo "Content-type: text/html; charset=utf-8"
+echo ""
+if [ "$HTTP_X_ORIGIN" ]; then
+ echo "$HTTP_X_ORIGIN"
+fi
+
+if [ "$HTTP_X_RATE_LIMIT" ]; then
+ echo "$HTTP_X_RATE_LIMIT"
+fi
--=-=-=--
From: RVP <rvp@SDF.ORG>
To: gnats-bugs@netbsd.org
Cc: Sunil Nimmagadda <sunil@nimmagadda.net>
Subject: Re: bin/58581 (ftp(1) should allow specifying header fields in http
requests)
Date: Mon, 30 Sep 2024 08:30:26 +0000 (UTC)
Some very minor nits, Sunil/Christos:
1. In ftp.1, the SYNOPSIS says `-h header' instead of `-H header'.
Also, there's a `.' missing before `It Fl' in the DESCRIPTION of that
option, so it's now taken to be part of the prev. option, `-g'.
And, it should be mentioned that multiple `-H's are allowed.
2. In ftp/main.c synopsis() should be updated for `-H'.
Thx,
-RVP
From: Sunil Nimmagadda <sunil@nimmagadda.net>
To: "RVP via gnats" <gnats-admin@NetBSD.org>
Cc: netbsd-bugs@netbsd.org, campbell+netbsd@mumble.net, gnats-bugs@netbsd.org
Subject: Re: bin/58581 (ftp(1) should allow specifying header fields in http
requests)
Date: Mon, 30 Sep 2024 15:07:01 +0530
--=-=-=
Content-Type: text/plain
"RVP via gnats" <gnats-admin@NetBSD.org> writes:
> The following reply was made to PR bin/58581; it has been noted by GNATS.
>
> From: RVP <rvp@SDF.ORG>
> To: gnats-bugs@netbsd.org
> Cc: Sunil Nimmagadda <sunil@nimmagadda.net>
> Subject: Re: bin/58581 (ftp(1) should allow specifying header fields in http
> requests)
> Date: Mon, 30 Sep 2024 08:30:26 +0000 (UTC)
>
> Some very minor nits, Sunil/Christos:
>
> 1. In ftp.1, the SYNOPSIS says `-h header' instead of `-H header'.
>
> Also, there's a `.' missing before `It Fl' in the DESCRIPTION of that
> option, so it's now taken to be part of the prev. option, `-g'.
>
> And, it should be mentioned that multiple `-H's are allowed.
>
> 2. In ftp/main.c synopsis() should be updated for `-H'.
>
> Thx,
>
> -RVP
>
Thanks, the markup fixes are attached...
--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment; filename=markup.diff
diff --git a/usr.bin/ftp/ftp.1 b/usr.bin/ftp/ftp.1
index e703e2e58cd8..4215d22e114f 100644
--- a/usr.bin/ftp/ftp.1
+++ b/usr.bin/ftp/ftp.1
@@ -67,7 +67,7 @@
.Nm
.Op Fl 46AadefginpRtVv\&?
.Op Fl b Ar bufsize
-.Op Fl h Ar header
+.Op Fl H Ar header
.Op Fl N Ar netrc
.Op Fl o Ar output
.Op Fl P Ar port
@@ -224,7 +224,7 @@ or
proxies.
.It Fl g
Disables file name globbing.
-It Fl H Ar header
+.It Fl H Ar header
Include the provided
.Ar header
string as a custom
@@ -232,6 +232,9 @@ string as a custom
header for an
.Th HTTP
request.
+Multiple headers may be specified via
+.Fl H
+option.
.It Fl i
Turns off interactive prompting during
multiple file transfers.
diff --git a/usr.bin/ftp/main.c b/usr.bin/ftp/main.c
index cc862615c50f..3095fe772a10 100644
--- a/usr.bin/ftp/main.c
+++ b/usr.bin/ftp/main.c
@@ -1080,7 +1080,7 @@ synopsis(FILE * stream)
const char * progname = getprogname();
fprintf(stream,
-"usage: %s [-46AadefginpRtVv] [-N NETRC] [-o OUTPUT] [-P PORT] [-q QUITTIME]\n"
+"usage: %s [-46AadefginpRtVv] [-H header] [-N NETRC] [-o OUTPUT] [-P PORT] [-q QUITTIME]\n"
" [-r RETRY] [-s SRCADDR] [-T DIR,MAX[,INC]] [-x XFERSIZE]\n"
" [[USER@]HOST [PORT]]\n"
" [[USER@]HOST:[PATH][/]]\n"
--=-=-=--
From: Taylor R Campbell <riastradh@NetBSD.org>
To: Sunil Nimmagadda <sunil@nimmagadda.net>
Cc: gnats-admin@NetBSD.org, netbsd-bugs@NetBSD.org,
gnats-bugs@NetBSD.org
Subject: Re: bin/58581 (ftp(1) should allow specifying header fields in http
requests)
Date: Tue, 1 Oct 2024 14:46:36 +0000
> Date: Mon, 30 Sep 2024 13:10:23 +0530
> From: Sunil Nimmagadda <sunil@nimmagadda.net>
>
> Sure, the attached diff has a testcase for the custom headers feature
> using atf...
That looks great! Since it's not a trivial change, can you please
provide a 2-clause BSD licence statement on the new files?
(There are lots of other examples you can derive from, like
tests/usr.bin/stat/t_stat.sh -- feel free to keep the copyright for
yourself or contribute it to TNF, as you see fit, as long as it's
permissively licensed.)
From: Taylor R Campbell <riastradh@NetBSD.org>
To: netbsd-bugs@NetBSD.org, gnats-bugs@NetBSD.org
Cc:
Subject: Re: bin/58581 (ftp(1) should allow specifying header fields in http
requests)
Date: Tue, 1 Oct 2024 17:57:21 +0000
Followups for pullup:
https://mail-index.netbsd.org/source-changes/2024/09/30/msg153623.html
https://mail-index.netbsd.org/source-changes/2024/09/30/msg153627.html
Module Name: src
Committed By: christos
Date: Mon Sep 30 13:03:37 UTC 2024
Modified Files:
src/usr.bin/ftp: ftp.1 main.c
Log Message:
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)
To generate a diff of this commit:
cvs rdiff -u -r1.157 -r1.158 src/usr.bin/ftp/ftp.1
cvs rdiff -u -r1.132 -r1.133 src/usr.bin/ftp/main.c
Module Name: src
Committed By: kre
Date: Mon Sep 30 19:23:31 UTC 2024
Modified Files:
src/usr.bin/ftp: ftp.1
Log Message:
Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
To generate a diff of this commit:
cvs rdiff -u -r1.158 -r1.159 src/usr.bin/ftp/ftp.1
From: Sunil Nimmagadda <sunil@nimmagadda.net>
To: Taylor R Campbell <riastradh@NetBSD.org>
Cc: gnats-admin@NetBSD.org, netbsd-bugs@NetBSD.org, gnats-bugs@NetBSD.org
Subject: Re: bin/58581 (ftp(1) should allow specifying header fields in http
requests)
Date: Wed, 02 Oct 2024 19:24:53 +0530
--=-=-=
Content-Type: text/plain
Taylor R Campbell <riastradh@NetBSD.org> writes:
>> Date: Mon, 30 Sep 2024 13:10:23 +0530
>> From: Sunil Nimmagadda <sunil@nimmagadda.net>
>>
>> Sure, the attached diff has a testcase for the custom headers feature
>> using atf...
>
> That looks great! Since it's not a trivial change, can you please
> provide a 2-clause BSD licence statement on the new files?
>
> (There are lots of other examples you can derive from, like
> tests/usr.bin/stat/t_stat.sh -- feel free to keep the copyright for
> yourself or contribute it to TNF, as you see fit, as long as it's
> permissively licensed.)
Sure, updated diff with licence for the new files...
--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment; filename=testcase.diff
Content-Description: ftp(1): Test for custom HTTP headers in a request
diff --git a/tests/usr.bin/Makefile b/tests/usr.bin/Makefile
index d0456d50235b..3d30e6157de5 100644
--- a/tests/usr.bin/Makefile
+++ b/tests/usr.bin/Makefile
@@ -6,7 +6,7 @@
TESTSDIR= ${TESTSBASE}/usr.bin
TESTS_SUBDIRS= awk basename bzip2 cc cmp compress config cpio col cut \
- diff dirname error find fstat gdb grep gzip id indent \
+ diff dirname error find fstat ftp gdb grep gzip id indent \
infocmp jot ld locale m4 make mixerctl mkdep mtree nbperf \
netpgpverify patch pkill pr printf pwhash realpath rump_server \
shmif_dumpbus shmif_pcapin sdiff sed sort stat tar tmux tr \
diff --git a/tests/usr.bin/ftp/Makefile b/tests/usr.bin/ftp/Makefile
new file mode 100644
index 000000000000..b66e9fdeefde
--- /dev/null
+++ b/tests/usr.bin/ftp/Makefile
@@ -0,0 +1,8 @@
+# $NetBSD: Makefile,v 1.1 2020/06/24 10:05:07 jruoho Exp $
+
+.include <bsd.own.mk>
+
+TESTSDIR= ${TESTSBASE}/usr.bin/ftp
+TESTS_SH= t_custom_headers
+
+.include <bsd.test.mk>
diff --git a/tests/usr.bin/ftp/t_custom_headers.sh b/tests/usr.bin/ftp/t_custom_headers.sh
new file mode 100644
index 000000000000..3609fb6ec6a8
--- /dev/null
+++ b/tests/usr.bin/ftp/t_custom_headers.sh
@@ -0,0 +1,62 @@
+# Copyright (c) 2024 The NetBSD Foundation, Inc.
+# All rights reserved.
+#
+# This code is derived from software contributed to The NetBSD Foundation
+# by Sunil Nimmagadda.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+# ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
+#
+
+atf_test_case custom_headers cleanup
+custom_headers_head()
+{
+ atf_require_prog ftp
+ atf_set "descr" "Check for custom HTTP headers"
+}
+
+HTTPD_PID=./.__httpd.pid
+custom_headers_body()
+{
+ # start httpd in daemon mode
+ atf_check -s exit:0 \
+ /usr/libexec/httpd -P $HTTPD_PID -I 8080 -b -C .sh /bin/sh \
+ -c $(atf_get_srcdir)/www/cgi-bin $(atf_get_srcdir)/www
+
+ atf_check -o inline:"example.com\n1000\n" \
+ -x "ftp -Vo- -H X-Origin:example.com \
+ -H X-Rate-Limit:1000 \
+ http://127.0.0.1:8080/cgi-bin/custom_headers.sh"
+}
+
+custom_headers_cleanup()
+{
+ if [ -f $HTTPD_PID ]; then
+ kill -9 $(cat $HTTPD_PID)
+ rm -f $HTTPD_PID
+ sleep 1
+ fi
+}
+
+atf_init_test_cases()
+{
+ atf_add_test_case custom_headers
+}
diff --git a/tests/usr.bin/ftp/www/cgi-bin/custom_headers.sh b/tests/usr.bin/ftp/www/cgi-bin/custom_headers.sh
new file mode 100755
index 000000000000..f448c4b49a0f
--- /dev/null
+++ b/tests/usr.bin/ftp/www/cgi-bin/custom_headers.sh
@@ -0,0 +1,39 @@
+#! /bin/sh
+
+# Copyright (c) 2024 The NetBSD Foundation, Inc.
+# All rights reserved.
+#
+# This code is derived from software contributed to The NetBSD Foundation
+# by Sunil Nimmagadda.
+#
+# Redistribution and use in source and binary forms, with or without
+# modification, are permitted provided that the following conditions
+# are met:
+# 1. Redistributions of source code must retain the above copyright
+# notice, this list of conditions and the following disclaimer.
+# 2. Redistributions in binary form must reproduce the above copyright
+# notice, this list of conditions and the following disclaimer in the
+# documentation and/or other materials provided with the distribution.
+#
+# THIS SOFTWARE IS PROVIDED BY THE NETBSD FOUNDATION, INC. AND CONTRIBUTORS
+# ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED
+# TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
+# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE FOUNDATION OR CONTRIBUTORS
+# BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR
+# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF
+# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
+# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN
+# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE)
+# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE
+# POSSIBILITY OF SUCH DAMAGE.
+#
+
+echo "Content-type: text/html; charset=utf-8"
+echo ""
+if [ "$HTTP_X_ORIGIN" ]; then
+ echo "$HTTP_X_ORIGIN"
+fi
+
+if [ "$HTTP_X_RATE_LIMIT" ]; then
+ echo "$HTTP_X_RATE_LIMIT"
+fi
--=-=-=--
From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/58581 CVS commit: src
Date: Sat, 12 Oct 2024 22:19:37 +0000
Module Name: src
Committed By: riastradh
Date: Sat Oct 12 22:19:37 UTC 2024
Modified Files:
src/distrib/sets/lists/tests: mi
src/etc/mtree: NetBSD.dist.tests
src/tests/usr.bin: Makefile
Added Files:
src/tests/usr.bin/ftp: Makefile custom_headers.sh t_custom_headers.sh
Log Message:
ftp(1): Add test for custom HTTP header fields.
Based on a patch from Sunil Nimmagadda <sunil@nimmagadda.net>.
PR bin/58581: ftp(1) should allow specifying header fields in http
requests
To generate a diff of this commit:
cvs rdiff -u -r1.1341 -r1.1342 src/distrib/sets/lists/tests/mi
cvs rdiff -u -r1.207 -r1.208 src/etc/mtree/NetBSD.dist.tests
cvs rdiff -u -r1.41 -r1.42 src/tests/usr.bin/Makefile
cvs rdiff -u -r0 -r1.1 src/tests/usr.bin/ftp/Makefile \
src/tests/usr.bin/ftp/custom_headers.sh \
src/tests/usr.bin/ftp/t_custom_headers.sh
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: needs-pullups->pending-pullups
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Sun, 13 Oct 2024 13:24:28 +0000
State-Changed-Why:
pullup-10 #970 https://releng.netbsd.org/cgi-bin/req-10.cgi?show=970
not gonna bring this up to 9
From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc:
Subject: PR/58581 CVS commit: [netbsd-10] src
Date: Sun, 13 Oct 2024 16:06:37 +0000
Module Name: src
Committed By: martin
Date: Sun Oct 13 16:06:37 UTC 2024
Modified Files:
src/distrib/sets/lists/tests [netbsd-10]: mi
src/etc/mtree [netbsd-10]: NetBSD.dist.tests
src/tests/usr.bin [netbsd-10]: Makefile
src/usr.bin/ftp [netbsd-10]: cmds.c complete.c fetch.c ftp.1 ftp.c
ftp_var.h main.c progressbar.c ruserpass.c ssl.c util.c
Added Files:
src/tests/usr.bin/ftp [netbsd-10]: Makefile custom_headers.sh
t_custom_headers.sh
Log Message:
Pull up following revision(s) (requested by riastradh in ticket #970):
tests/usr.bin/Makefile: revision 1.42
usr.bin/ftp/ruserpass.c: revision 1.34
usr.bin/ftp/main.c: revision 1.130
usr.bin/ftp/ssl.c: revision 1.17
usr.bin/ftp/main.c: revision 1.131
usr.bin/ftp/ssl.c: revision 1.18
usr.bin/ftp/main.c: revision 1.132
usr.bin/ftp/ssl.c: revision 1.19
usr.bin/ftp/main.c: revision 1.133
distrib/sets/lists/tests/mi: revision 1.1342
usr.bin/ftp/ftp.1: revision 1.151
usr.bin/ftp/ftp.1: revision 1.152
usr.bin/ftp/progressbar.c: revision 1.25
usr.bin/ftp/ftp.1: revision 1.153
usr.bin/ftp/progressbar.c: revision 1.26
usr.bin/ftp/ftp.1: revision 1.155
usr.bin/ftp/ftp.1: revision 1.156
usr.bin/ftp/fetch.c: revision 1.239
usr.bin/ftp/ftp.1: revision 1.157
usr.bin/ftp/ftp.1: revision 1.158
usr.bin/ftp/ftp.1: revision 1.159
usr.bin/ftp/ftp_var.h: revision 1.87
etc/mtree/NetBSD.dist.tests: revision 1.208
usr.bin/ftp/ftp_var.h: revision 1.88
usr.bin/ftp/ftp_var.h: revision 1.89
usr.bin/ftp/cmds.c: revision 1.142
usr.bin/ftp/util.c: revision 1.168
usr.bin/ftp/cmds.c: revision 1.143
tests/usr.bin/ftp/custom_headers.sh: revision 1.1
usr.bin/ftp/ssl.c: revision 1.20
usr.bin/ftp/complete.c: revision 1.48
tests/usr.bin/ftp/Makefile: revision 1.1
tests/usr.bin/ftp/t_custom_headers.sh: revision 1.1
usr.bin/ftp/fetch.c: revision 1.240
usr.bin/ftp/fetch.c: revision 1.241
usr.bin/ftp/ftp.c: revision 1.176
usr.bin/ftp/ftp.c: revision 1.177
(all via patch)
ftp(1): wording and formatting improvements
Fix grammar issue with "Support values" reported in private mail.
Document all file transfer types in "type" and cross-reference that.
Consistency fixes in describing file transfer parameters and types.
Fix some mandoc -Tlint issues (except "useless macro: Tn").
Add -b <buflen> to specify the buffer size.
ftp: bump FTPBUFLEN from 4kB to 16kB
sourceforge.net returns a 5kB content-security-policy.
Analyzed by mlelstv@ who reports usual limits are between 4kB and 48kB.
default is now 16K
ftp: improve -b documentation
Order -b bufsize in the synopsis.
Document the actual default value.
ftp: improve units used in comments and errors
Use "KiB" instead of "K" in errors.
Clarify related comments.
pass some lint.
PR/58581: Sunil Nimmagadda: Add flag to allow specifying extra http header
fields.
ftp(1): Nix trailing whitespace in man page.
No functional change intended.
PR bin/58581: ftp(1) should allow specifying header fields in http requests
fix markup (h -> H), explain about multiple headers, fix usage (from RVP)
Don't forget the dot, use the intended macro name (I think),
and improve the wording a little. (All related to the -H option.)
ftp(1): Add test for custom HTTP header fields.
Based on a patch from Sunil Nimmagadda.
PR bin/58581: ftp(1) should allow specifying header fields in http
requests
To generate a diff of this commit:
cvs rdiff -u -r1.1238.2.15 -r1.1238.2.16 src/distrib/sets/lists/tests/mi
cvs rdiff -u -r1.197.2.4 -r1.197.2.5 src/etc/mtree/NetBSD.dist.tests
cvs rdiff -u -r1.37.2.1 -r1.37.2.2 src/tests/usr.bin/Makefile
cvs rdiff -u -r0 -r1.1.2.2 src/tests/usr.bin/ftp/Makefile \
src/tests/usr.bin/ftp/custom_headers.sh \
src/tests/usr.bin/ftp/t_custom_headers.sh
cvs rdiff -u -r1.141 -r1.141.6.1 src/usr.bin/ftp/cmds.c
cvs rdiff -u -r1.47 -r1.47.10.1 src/usr.bin/ftp/complete.c
cvs rdiff -u -r1.235.2.2 -r1.235.2.3 src/usr.bin/ftp/fetch.c
cvs rdiff -u -r1.147.2.4 -r1.147.2.5 src/usr.bin/ftp/ftp.1
cvs rdiff -u -r1.174.2.1 -r1.174.2.2 src/usr.bin/ftp/ftp.c
cvs rdiff -u -r1.86 -r1.86.2.1 src/usr.bin/ftp/ftp_var.h
cvs rdiff -u -r1.128.2.1 -r1.128.2.2 src/usr.bin/ftp/main.c
cvs rdiff -u -r1.24 -r1.24.6.1 src/usr.bin/ftp/progressbar.c
cvs rdiff -u -r1.33 -r1.33.86.1 src/usr.bin/ftp/ruserpass.c
cvs rdiff -u -r1.12.2.4 -r1.12.2.5 src/usr.bin/ftp/ssl.c
cvs rdiff -u -r1.164.2.2 -r1.164.2.3 src/usr.bin/ftp/util.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
State-Changed-From-To: pending-pullups->closed
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Sun, 13 Oct 2024 20:31:16 +0000
State-Changed-Why:
fixed in HEAD, pulled up to 10, not worth it for 9
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.47 2022/09/11 19:34:41 kim Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2024
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home