NetBSD Problem Report #55384

From tsutsui@ceres.dti.ne.jp  Sat Jun 13 14:21:56 2020
Return-Path: <tsutsui@ceres.dti.ne.jp>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id 267651A9219
	for <gnats-bugs@gnats.NetBSD.org>; Sat, 13 Jun 2020 14:21:56 +0000 (UTC)
Message-Id: <202006131421.05DELk61016314@ceres.dti.ne.jp>
Date: Sat, 13 Jun 2020 23:21:46 +0900 (JST)
From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
Reply-To: tsutsui@ceres.dti.ne.jp
To: gnats-bugs@NetBSD.org
Cc: tsutsui@ceres.dti.ne.jp
Subject: sysinst doesn't handle MD pm->ptstart value to reserve sectors
X-Send-Pr-Version: 3.95

>Number:         55384
>Category:       install
>Synopsis:       sysinst doesn't handle MD pm->ptstart value to reserve sectors
>Confidential:   no
>Severity:       critical
>Priority:       medium
>Responsible:    martin
>State:          feedback
>Class:          sw-bug
>Submitter-Id:   net
>Arrival-Date:   Sat Jun 13 14:25:00 +0000 2020
>Closed-Date:    
>Last-Modified:  Sun Oct 04 17:35:01 +0000 2020
>Originator:     Izumi Tsutsui
>Release:        NetBSD 9.0
>Organization:
>Environment:
System: NetBSD mirage 9.0 NetBSD 9.0 (GENERIC) #23: Wed May 13 04:08:16 JST 2020 tsutsui@mirage:/s/netbsd-9/src/sys/arch/i386/compile/GENERIC i386
Architecture: maybe all
Machine: ports on (pm->start != 0) in sysinst/arch/*/md.c
>Description:
In NetBSD 8.x days sysinst has a value "pm->ptstart" and it is set
in MD md_get_info() in src/usr.sbin/sysinst/arch/*/md.c.

It's used to reserve sectors at the top of the target disk
for bootstrap or to store MD partition info etc.

MI make_bsd_partitions() in src/usr.sbin/sysinst/bsddisklabel.c
adjust all partitions using the pm->ptstart to locate all partitions
after pt->ptstart:

 https://nxr.netbsd.org/xref/src/usr.sbin/sysinst/bsddisklabel.c?r=1.2.20.2#701
---
    701 		if (valid_part >= 0 && pm->oldlabel[valid_part].pi_offset < pm->ptstart) {
    702 			pm->oldlabel[valid_part].pi_offset = pm->ptstart;
    703 			pm->oldlabel[valid_part].pi_size -= pm->ptstart;
    704 		}

 :

    741 	if (layoutkind == LY_SETNEW)
    742 		get_ptn_sizes(partstart, ptend - partstart, no_swap);
---

However in NetBSD 9.0 pm->ptstart no longer used to adjust
start sector of partitions. It's just used to check
partition info returned from partition dependent(?)
get_part_info() functions.

 https://nxr.netbsd.org/xref/src/usr.sbin/sysinst/bsddisklabel.c?r=1.23.2.10#1039
---
   1039 		if (info.nat_type->generic_ptype != PT_swap &&
   1040 		    (info.start < ptstart ||
   1041 		    (info.start + info.size) > (ptstart+ptsize)))
   1042 			continue;
---

>How-To-Repeat:
Install NetBSD/x68k 9.0 (with fixes in PR/55187 and PR/55375).

The default partition settings show the root partition starts
from sector 0 so later newfs overwrite necessary partition info
and bootloaders.

Maybe other pm->ptstart != 0 ports (acorn32 and mvme68k?) are affected.

>Fix:
No idea which function should handle pm->ptstart restriction.

I've tried to fix fill_defaults(), apply_settings_to_partitions(),
sort_and_sync_parts(), merge_part_with_wanted() etc. but I get
no success. and give up to see partition independent abstraction layer.

---
Izumi Tsutsui

>Release-Note:

>Audit-Trail:
From: "Martin Husemann" <martin@netbsd.org>
To: gnats-bugs@gnats.NetBSD.org
Cc: 
Subject: PR/55384 CVS commit: src/usr.sbin/sysinst
Date: Sat, 3 Oct 2020 18:54:18 +0000

 Module Name:	src
 Committed By:	martin
 Date:		Sat Oct  3 18:54:18 UTC 2020

 Modified Files:
 	src/usr.sbin/sysinst: bsddisklabel.c disklabel.c gpt.c label.c mbr.c
 	    part_edit.c partitions.h

 Log Message:
 PR 55384: detangle pm->ptstart from the "install" flag (selecting a
 target partition). Instead introduce a new PTI_INSTALL_TARGET per partition
 flag and deal with it in the partitioning backends.

 Honour pm->ptstart when allocating new partitions - it is supposed to be
 the first sector usable by NetBSD.


 To generate a diff of this commit:
 cvs rdiff -u -r1.45 -r1.46 src/usr.sbin/sysinst/bsddisklabel.c
 cvs rdiff -u -r1.39 -r1.40 src/usr.sbin/sysinst/disklabel.c
 cvs rdiff -u -r1.18 -r1.19 src/usr.sbin/sysinst/gpt.c
 cvs rdiff -u -r1.25 -r1.26 src/usr.sbin/sysinst/label.c
 cvs rdiff -u -r1.33 -r1.34 src/usr.sbin/sysinst/mbr.c
 cvs rdiff -u -r1.17 -r1.18 src/usr.sbin/sysinst/part_edit.c
 cvs rdiff -u -r1.16 -r1.17 src/usr.sbin/sysinst/partitions.h

 Please note that diffs are not public domain; they are subject to the
 copyright notices on the relevant files.

Responsible-Changed-From-To: install-manager->martin
Responsible-Changed-By: martin@NetBSD.org
Responsible-Changed-When: Sat, 03 Oct 2020 18:59:25 +0000
Responsible-Changed-Why:
My bug


State-Changed-From-To: open->feedback
State-Changed-By: martin@NetBSD.org
State-Changed-When: Sat, 03 Oct 2020 18:59:25 +0000
State-Changed-Why:
Indeed, this was pretty intrusive.
Not sure I got them all - could you test on x68k ?


From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
To: gnats-bugs@netbsd.org
Cc: martin@netbsd.org, install-manager@netbsd.org, tsutsui@ceres.dti.ne.jp
Subject: Re: install/55384 (sysinst doesn't handle MD pm->ptstart value to reserve
	 sectors)
Date: Sun, 4 Oct 2020 20:38:03 +0900

 > State-Changed-From-To: open->feedback
 > State-Changed-By: martin@NetBSD.org
 > State-Changed-When: Sat, 03 Oct 2020 18:59:25 +0000
 > State-Changed-Why:
 > Indeed, this was pretty intrusive.
 > Not sure I got them all - could you test on x68k ?

 It looks still something wrong.

 (typed from XM6i emulator screen)
 ---

 You can now change the sizes for the system partitions.  The default is to allocate all the
 space to the root file system.  However, you may wish to have separate /usr (additional 
 system files), /var (log files etc) or /home (users' home directories) file systems.

 Free space will be added to the partition marked with a '+'.

       Size (sec)                           Filesystem
       ---------------------------------- - ---------------------
   a:  622592 (2097152)                   + /
   b:  0                                    <swap>
   c:  0                                    /tmp (mfs)
   d:  0                                    /usr
   e:  0                                    /var
       ---------------------------------- - ---------------------
   g:  Add a user defined partition
   h:  Clone external partition(s)
   i:  Change input units (sectors/cylinders/MB/GB)
   x:  Go on.  Free space 1474560 sec.

 ---

  We now have your disklabel partitions for sd0 below.  This is your last chance to change
  them.

  Flags: (I)nstall, (N)ewfs.  Total size: 512M, free: 512M

      Start (sec)    End (sec)   Size (sec)  FS type Flag Filesystem
     ------------ ------------ ------------ -------- ---- ----------------
  a:            0      1048575      1048576   unused
  b:            0      1048575      1048576 Whole disk
     ------------ ------------ ------------ -------- ---- ----------------
  d: Add a partition
  e: Change input units (sectors/cylinders/MB/GB)
  f: Edit name of the disk
  g: Clone external partition(s)
  h: Cancel
 >x: Partition sizes ok

 ---

 No NetBSD partition appears on default choices.

 If I explicitly specify swap partition in the
 "change the sizes for the system partitions" menu,
 a NetBSD partition appears, but it fails because the
 "unused" partition is defined as a: by default?

 ---

 You can now change the sizes for the system partitions.  The default is to allocate all the
 space to the root file system.  However, you may wish to have separate /usr (additional 
 system files), /var (log files etc) or /home (users' home directories) file systems.

 Free space will be added to the partition marked with a '+'.

       Size (sec)                           Filesystem
       ---------------------------------- - ---------------------
   a:  622592 (2097152)                   + /
   b:  65536                                <swap>
   c:  0                                    /tmp (mfs)
   d:  0                                    /usr
   e:  0                                    /var
       ---------------------------------- - ---------------------
   g:  Add a user defined partition
   h:  Clone external partition(s)
   i:  Change input units (sectors/cylinders/MB/GB)
   x:  Go on.  Free space 1409024 sec.

 ---

  We now have your disklabel partitions for sd0 below.  This is your last chance to change
  them.

  Flags: (I)nstall, (N)ewfs.  Total size: 512M, free: 512M

      Start (sec)    End (sec)   Size (sec)  FS type Flag Filesystem
     ------------ ------------ ------------ -------- ---- ----------------
  a:            0      1048575      1048576   unused
  b:            0      1048575      1048576 Whole disk
  c:           64       983103       983040   4.2BSD N
     ------------ ------------ ------------ -------- ---- ----------------
  d: Add a partition
  e: Change input units (sectors/cylinders/MB/GB)
  f: Edit name of the disk
  g: Clone external partition(s)
  h: Cancel
 >x: Partition sizes ok

 ---

 partitions 0 - 1048576 sec, unused and 64 - 983104 sec, 4.2BSD overlap.






                     Do you want to re-edit the disklabel partitions?    

                    >a: Re-edit                                          
                     b: Use this partitions anyway                       
                     c: Abort installation                               


 ---

 No idea what's going on the disklabel editor internals.

 ---
 Izumi Tsutsui

From: Izumi Tsutsui <tsutsui@ceres.dti.ne.jp>
To: gnats-bugs@netbsd.org
Cc: martin@netbsd.org, install-manager@netbsd.org, tsutsui@ceres.dti.ne.jp
Subject: Re: install/55384 (sysinst doesn't handle MD pm->ptstart value to reservesectors)
Date: Mon, 5 Oct 2020 02:34:34 +0900

 > It looks still something wrong.

 I've captured how partition editor behaves during installation on XM6i:
  https://www.youtube.com/watch?v=MbUoxN3rNQo

 ---
 Izumi Tsutsui

>Unformatted:

NetBSD Home
NetBSD PR Database Search

(Contact us) $NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020 The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.