NetBSD Problem Report #58656

From www@netbsd.org  Thu Aug 29 22:18:16 2024
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 298E41A923F
	for <gnats-bugs@gnats.NetBSD.org>; Thu, 29 Aug 2024 22:18:16 +0000 (UTC)
Message-Id: <20240829221815.1FDDF1A9241@mollari.NetBSD.org>
Date: Thu, 29 Aug 2024 22:18:15 +0000 (UTC)
From: rvp@SDF.ORG
Reply-To: rvp@SDF.ORG
To: gnats-bugs@NetBSD.org
Subject: libfetch: multiple issues with SSL+CONNECT
X-Send-Pr-Version: www-1.0

>Number:         58656
>Category:       lib
>Synopsis:       libfetch: multiple issues with SSL+CONNECT
>Confidential:   no
>Severity:       non-critical
>Priority:       low
>Responsible:    lib-bug-people
>State:          feedback
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Thu Aug 29 22:20:01 +0000 2024
>Closed-Date:    
>Last-Modified:  Tue Oct 08 22:06:25 +0000 2024
>Originator:     RVP
>Release:        NetBSD/amd64 10.99.12
>Organization:
>Environment:
NetBSD/amd64 10.99.12
>Description:
1. Fetching files via a proxy doesn't work for HTTPS when the proxy
   server (here, `polipo`) doesn't doesn't add any HTTP headers (as
   reqd. by the RFCs):

```
$ echo $http_proxy
http://localhost:8118/

$ time -p pkgin pc libfetch
00B88063247F0000:error:0A000126:SSL routines:ssl3_read_n:unexpected eof while reading:/usr/src/crypto/external/bsd/openssl/dist/ssl/record/rec_layer_s3.c:322:
pkg_info: can't find package `https://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All/libfetch-2.40.tgz', skipped
real 61.60
user 0.05
sys 0.08

$ no_proxy=\* time -p pkgin pc libfetch
Information for https://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All/libfetch-2.40.tgz:
Files:
/usr/pkg/include/fetch.h
/usr/pkg/lib/libfetch.a
/usr/pkg/man/man3/fetch.3
real         0.29
user         0.07
sys          0.08

$
```

This is caused by an extra read in the NetBSD version of libfetch for
headers which never arrive. The FreeBSD library removes trailing blanks,
then checks for a 0-length HTTP header before going to read a new line.

2. Even if you fix 1), Proxy-authentication isn't done for HTTPS URLs:

```
$ cat /tmp/fetchstat.c
#include <err.h>
#include <stdio.h>
#include <stdlib.h>
#include <fetch.h>
int main(int argc, char* argv[]) {
        struct url_stat us;
        if (argc != 2)
                exit(EXIT_FAILURE);
        if (fetchStatURL(argv[1], &us, "v") != 0)
                err(1, "fetchStat(%s) failed", argv[1]);
        printf("%zu bytes\n", us.size);
        return 0;
}

$ cc -o /tmp/f /tmp/fetchstat.c -L/tmp/fetch/lib -lfetch -lssl

$ HTTP_PROXY_AUTH='basic:*:rvp:not_very_secret' /tmp/f https://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All/libfetch-2.40.tgz
looking up localhost
connecting to localhost:8118
f: fetchStat(https://cdn.NetBSD.org/pub/pkgsrc/packages/NetBSD/x86_64/10.0/All/libfetch-2.40.tgz) failed: Undefined error: 0

$
```

Send Basic auth for proxy+SSL also.

3. Wrong test when reading HTTP headers in libfetch/http.c:http_connect():

```
    794                         }
    795                 } while (h < hdr_end);
    796         }
```

Surely that should be `while (h > hdr_end)` to discard headers (if any)
from the proxy server?
>How-To-Repeat:
As shown above.
>Fix:
Note this is for the libfetch in base.

---START patch---
diff -urN fetch.orig/dist/libfetch/http.c fetch/dist/libfetch/http.c
--- fetch.orig/dist/libfetch/http.c	2024-02-03 01:51:49.796872219 +0000
+++ fetch/dist/libfetch/http.c	2024-08-29 00:31:49.955206679 +0000
@@ -771,16 +771,20 @@
 				URL->host, URL->port);
 		http_cmd(conn, "Host: %s:%d\r\n",
 				URL->host, URL->port);
+		/* proxy authorization */
+		if (purl) {
+			if (*purl->user || *purl->pwd)
+				http_basic_auth(conn, "Proxy-Authorization",
+				    purl->user, purl->pwd);
+			else if ((p = getenv("HTTP_PROXY_AUTH")) != NULL && *p != '\0')
+				http_authorize(conn, "Proxy-Authorization", p);
+		}
 		http_cmd(conn, "\r\n");
 		if (http_get_reply(conn) != HTTP_OK) {
 			http_seterr(conn->err);
 			goto ouch;
 		}
-		/* Read and discard the rest of the proxy response */
-		if (fetch_getln(conn) < 0) {
-			fetch_syserr();
-			goto ouch;
-		}
+		/* Read and discard the rest of the proxy response (if any) */
 		do {
 			switch ((h = http_next_header(conn, &p))) {
 			case hdr_syserror:
@@ -792,7 +796,7 @@
 			default:
 				/* ignore */ ;
 			}
-		} while (h < hdr_end);
+		} while (h > hdr_end);
 	}
 	if (strcasecmp(URL->scheme, SCHEME_HTTPS) == 0 &&
 	    fetch_ssl(conn, URL, verbose) == -1) {
---END patch---

>Release-Note:

>Audit-Trail:
From: "Christos Zoulas" <christos@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/58656 CVS commit: src/external/bsd/fetch/dist/libfetch
Date: Sun, 1 Sep 2024 11:07:31 -0400

 Module Name:	src
 Committed By:	christos
 Date:		Sun Sep  1 15:07:31 UTC 2024

 Modified Files:
 	src/external/bsd/fetch/dist/libfetch: http.c

 Log Message:
 PR/58656: RVP: Fix proxy authentication for https and when proxy does not
 add any extra headers.


 To generate a diff of this commit:
 cvs rdiff -u -r1.5 -r1.6 src/external/bsd/fetch/dist/libfetch/http.c

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

State-Changed-From-To: open->feedback
State-Changed-By: riastradh@NetBSD.org
State-Changed-When: Tue, 08 Oct 2024 22:06:25 +0000
State-Changed-Why:
Fixed in HEAD?  Does this need pullups?


>Unformatted:

Home	PR Database Search