NetBSD Problem Report #59359

From www@netbsd.org  Sat Apr 26 15:22:21 2025
Return-Path: <www@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)
	 key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256
	 client-signature RSA-PSS (2048 bits) client-digest SHA256)
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id D852F1A923C
	for <gnats-bugs@gnats.NetBSD.org>; Sat, 26 Apr 2025 15:22:21 +0000 (UTC)
Message-Id: <20250426152220.4405C1A923E@mollari.NetBSD.org>
Date: Sat, 26 Apr 2025 15:22:20 +0000 (UTC)
From: campbell+netbsd@mumble.net
Reply-To: campbell+netbsd@mumble.net
To: gnats-bugs@NetBSD.org
Subject: static pies are broken
X-Send-Pr-Version: www-1.0

>Number:         59359
>Category:       lib
>Synopsis:       static pies are broken
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    lib-bug-people
>State:          open
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Apr 26 15:25:00 +0000 2025
>Last-Modified:  Sun Apr 27 14:05:01 +0000 2025
>Originator:     Taylor R Campbell
>Release:        current, 10, 9, ...
>Organization:
The StaticBSD Pie-in-the-sky
>Environment:
>Description:
$ cat hello.c
#include <stdio.h>

int
main(void)
{
	printf("Hello, world!\n");
	fflush(stdout);
	return ferror(stdout);
}
$ make hello.o hello DBG=-g\ -O2\ -Wall\ -Werror\ -fpie LDFLAGS=-pie\ -static
cc -g -O2 -Wall -Werror -fpie   -c hello.c -o hello.o
cc -g -O2 -Wall -Werror -fpie  -pie -static  -o hello hello.c 
$ ./hello
[1]   Segmentation fault (core dumped) ./hello

Same deal on mips and riscv.  Need to test and review on other platforms.  Presumably this happens because nothing relocates the GOT in static executables.

On mips, it crashes inside ___start (crt0-common.c) as soon as it tries to read something out of the GOT:

(gdb) x/i $pc
=> 0x2345c <___start+52>:       ld      s2,-32720(gp)
(gdb) print &_GLOBAL_OFFSET_TABLE_
$6 = (<variable (not text or data), no debug info> *) 0xf53f0
(gdb) print $gp - 0x7ff0
$8 = 0xe53f0

Note that these are off by 0x10000, the page size.  And so is $t9 (called procedure) versus the ___start symbol itself:

(gdb) print ___start
$9 = {void (void (*)(void), struct ps_strings *)} 0x23428 <___start>
(gdb) print $t9
$10 = 0x13428

But somehow control has reached the ___start symbol!

The executable itself has a LOAD command at 0, as shown by `readelf -l', which is rounded up by the kernel to the requested segment alignment, 0x10000:

Program Headers:
  Type           Offset             VirtAddr           PhysAddr
                 FileSiz            MemSiz              Flags  Align
  PHDR           0x0000000000000040 0x0000000000000040 0x0000000000000040
                 0x00000000000001c0 0x00000000000001c0  R      0x8
  ABIFLAGS       0x0000000000000230 0x0000000000000230 0x0000000000000230
                 0x0000000000000018 0x0000000000000018  R      0x8
  LOAD           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x00000000000cd45c 0x00000000000cd45c  R E    0x10000
  LOAD           0x00000000000cd460 0x00000000000dd460 0x00000000000dd460
                 0x0000000000009bc8 0x0000000000118700  RW     0x10000
  DYNAMIC        0x00000000000024a8 0x00000000000024a8 0x00000000000024a8
                 0x00000000000001e0 0x00000000000001e0  R      0x8
  NOTE           0x0000000000000200 0x0000000000000200 0x0000000000000200
                 0x000000000000002c 0x000000000000002c  R      0x4
  TLS            0x00000000000cd460 0x00000000000dd460 0x00000000000dd460
                 0x0000000000000a68 0x0000000000000a71  R      0x8
  NULL           0x0000000000000000 0x0000000000000000 0x0000000000000000
                 0x0000000000000000 0x0000000000000000         0x8

    128 static int
    129 elf_placedynexec(struct exec_package *epp, Elf_Ehdr *eh, Elf_Phdr *ph)
    130 {
    131 	Elf_Addr align, offset;
    132 	int i;
    133 
    134 	for (align = 1, i = 0; i < eh->e_phnum; i++)
    135 		if (ph[i].p_type == PT_LOAD && ph[i].p_align > align)
    136 			align = ph[i].p_align;
    137 
    138 	offset = (Elf_Addr)pax_aslr_exec_offset(epp, align);
    139 	if (offset < epp->ep_vm_minaddr)
    140 		offset = roundup(epp->ep_vm_minaddr, align);
    141 	if ((offset & (align - 1)) != 0) {
    142 		DPRINTF("bad offset=%#jx align=%#jx",
    143 		    (uintmax_t)offset, (uintmax_t)align);
    144 		return SET_ERROR(EINVAL);
    145 	}
    146 
    147 	for (i = 0; i < eh->e_phnum; i++)
    148 		ph[i].p_vaddr += offset;
    149 	epp->ep_entryoffset = offset;
    150 	eh->e_entry += offset;
    151 	return 0;
    152 }

https://nxr.netbsd.org/xref/src/sys/kern/exec_elf.c?r=1.107#128

How did control successfully transfer from __start (crt0.S) to ___start (crt0-common.c) with the wrong address in t9, when it works via jump to register t9?

     55 	PTR_L	t9,%call16(_C_LABEL(___start))(gp)
     56 	 move	a1, a3		/* ps_strings */
     57 	.reloc  1f,R_MIPS_JALR,___start
     58 1:	jr	t9

https://nxr.netbsd.org/xref/src/lib/csu/arch/mips/crt0.S?r=1.4#55

Disassembling crt0.o reveals why -- the assembler/linker has cleverly replaced  jr t9  by  b ___start:

0000000000013390 <__start>:
__start():
   13390:       03807825        move    t3,gp
   13394:       3c1c000e        lui     gp,0xe
   13398:       279ca050        addiu   gp,gp,-24496
   1339c:       0399e02d        daddu   gp,gp,t9
   133a0:       00a02025        move    a0,a1
   133a4:       df998020        ld      t9,-32736(gp)
   133a8:       00e02825        move    a1,a3
   133ac:       1000001e        b       13428 <___start>
   133b0:       00000000        nop

So the global offset table content is off by 0x10000, the amount by which the kernel adjusted the LOAD commands to avoid the zero page.  If I set a breakpoint on __start, gdb confirms this:

(gdb) disas
Dump of assembler code for function _start:
   0x0000000000023390 <+0>:     move    t3,gp
   0x0000000000023394 <+4>:     lui     gp,0xe
   0x0000000000023398 <+8>:     addiu   gp,gp,-24496
   0x000000000002339c <+12>:    daddu   gp,gp,t9
=> 0x00000000000233a0 <+16>:    move    a0,a1
   0x00000000000233a4 <+20>:    ld      t9,-32736(gp)
   0x00000000000233a8 <+24>:    move    a1,a3
   0x00000000000233ac <+28>:    b       0x23428 <___start>
   0x00000000000233b0 <+32>:    nop
End of assembler dump.
(gdb) print $gp
$5 = 0xfd3e0
(gdb) print &_GLOBAL_OFFSET_TABLE_
$6 = (<variable (not text or data), no debug info> *) 0xf53f0
(gdb) print $gp - 0x7ff0
$7 = 0xf53f0
(gdb) x/xg $gp - 32736
0xf5400:        0x0000000000013428
(gdb) print ___start
$8 = {void (void (*)(void), struct ps_strings *)} 0x23428 <___start>

Presumably there is nothing relocating the GOT like _rtld_start/_rtld_relocate_nonplt_self does on mips.  Likely the same issue on Alpha, but with explicit relocations instead of hard-coded logic to relocate each GOT entry; however, gdb is having more trouble on alpha, so I haven't confirmed this.

Guessing that in order to make this work, we need some CSU routine to do essentially the logic of _rtld_relocate_nonplt_self and relocate the GOT/.rel.dyn/.rela.dyn.
>How-To-Repeat:
as above
>Fix:
Yes, please!

>Release-Note:

>Audit-Trail:
From: =?UTF-8?Q?J=C3=B6rg_Sonnenberger?= <joerg@bec.de>
To: gnats-bugs@netbsd.org, kern-bug-people@netbsd.org,
 gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Cc: 
Subject: Re: kern/59359: static pies are broken
Date: Sun, 27 Apr 2025 00:18:25 +0200

 > Same deal on mips and riscv.  Need to test and review on other platforms.  Presumably this happens because nothing relocates the GOT in static executables.

 Support for static PIE is currently limited to x86. See crt0-common.c 
 HAS_RELOCATE_SELF.

 Joerg

Responsible-Changed-From-To: kern-bug-people->lib-bug-people
Responsible-Changed-By: riastradh@NetBSD.org
Responsible-Changed-When: Sun, 27 Apr 2025 03:02:58 +0000
Responsible-Changed-Why:
library issue (missing self-reloc in csu), not kernel issue


From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/59359 CVS commit: src/share/mk
Date: Sun, 27 Apr 2025 03:48:34 +0000

 Module Name:	src
 Committed By:	riastradh
 Date:		Sun Apr 27 03:48:34 UTC 2025

 Modified Files:
 	src/share/mk: bsd.own.mk

 Log Message:
 bsd.own.mk: If NOPIE is defined, set MKPIE=no.

 This way, on sun2, we don't wind up with the bizarre setting of
 MKPIC=no MKPIE=yes by default.

 Preparation for conditionalizing automatic tests for:

 PR lib/59359: static pies are broken


 To generate a diff of this commit:
 cvs rdiff -u -r1.1415 -r1.1416 src/share/mk/bsd.own.mk

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/59359 CVS commit: src
Date: Sun, 27 Apr 2025 04:09:35 +0000

 Module Name:	src
 Committed By:	riastradh
 Date:		Sun Apr 27 04:09:35 UTC 2025

 Modified Files:
 	src/distrib/sets/lists/debug: mi
 	src/distrib/sets/lists/tests: mi
 	src/tests/lib/csu: Makefile
 Added Files:
 	src/tests/lib/csu: h_hello.c t_hello.sh

 Log Message:
 lib/csu: Add tests of {static, dynamic} x {pie, non-pie}.

 Main goal here is to test static pies, but let's take advantage of
 this to test the same code several ways, since the csu logic has some
 differences for static vs dynamic builds too like the ordering of
 _libc_init calls.

 PR lib/59359: static pies are broken


 To generate a diff of this commit:
 cvs rdiff -u -r1.478 -r1.479 src/distrib/sets/lists/debug/mi
 cvs rdiff -u -r1.1371 -r1.1372 src/distrib/sets/lists/tests/mi
 cvs rdiff -u -r1.7 -r1.8 src/tests/lib/csu/Makefile
 cvs rdiff -u -r0 -r1.1 src/tests/lib/csu/h_hello.c \
     src/tests/lib/csu/t_hello.sh

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

From: "Taylor R Campbell" <riastradh@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/59359 CVS commit: src/tests/lib/csu
Date: Sun, 27 Apr 2025 14:04:19 +0000

 Module Name:	src
 Committed By:	riastradh
 Date:		Sun Apr 27 14:04:19 UTC 2025

 Modified Files:
 	src/tests/lib/csu: Makefile

 Log Message:
 tests/lib/csu: Build h_hello.o with -fPIE, not -fpie.

 Static libraries like libc may overflow the bounds assumed by -fpie.
 Resolves:

 h_hello.o: in function `main':
 h_hello.c:(.text.startup+0x30): relocation truncated to fit: R_SPARC_GOT13 against symbol `__sF' defined in .data.rel section in /tmp/build/2025.04.27.10.22.23-sparc64/destdir/usr/lib/libc.a(findfp.o)

 PR lib/59359: static pies are broken


 To generate a diff of this commit:
 cvs rdiff -u -r1.8 -r1.9 src/tests/lib/csu/Makefile

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

>Unformatted:

Home	PR Database Search