NetBSD Problem Report #52888

From www@NetBSD.org  Tue Jan  2 19:12:51 2018
Return-Path: <www@NetBSD.org>
Received: from mail.netbsd.org (mail.netbsd.org [199.233.217.200])
	(using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits))
	(Client CN "mail.NetBSD.org", Issuer "mail.NetBSD.org CA" (not verified))
	by mollari.NetBSD.org (Postfix) with ESMTPS id A54D57A19A
	for <gnats-bugs@gnats.NetBSD.org>; Tue,  2 Jan 2018 19:12:51 +0000 (UTC)
Message-Id: <20180102191250.47F307A1DF@mollari.NetBSD.org>
Date: Tue,  2 Jan 2018 19:12:50 +0000 (UTC)
From: kusumi.tomohiro@gmail.com
Reply-To: kusumi.tomohiro@gmail.com
To: gnats-bugs@NetBSD.org
Subject: Add autofs for NetBSD
X-Send-Pr-Version: www-1.0

>Number:         52888
>Category:       kern
>Synopsis:       Add autofs for NetBSD
>Confidential:   no
>Severity:       serious
>Priority:       medium
>Responsible:    kern-bug-people
>State:          closed
>Class:          support
>Submitter-Id:   net
>Arrival-Date:   Tue Jan 02 19:15:00 +0000 2018
>Closed-Date:    Sun Jul 25 03:56:49 +0000 2021
>Last-Modified:  Sun Jul 25 03:56:49 +0000 2021
>Originator:     Tomohiro Kusumi
>Release:        NetBSD-current
>Organization:
>Environment:
Architecture independent.
# uname -r
8.99.9

>Description:
This is the initial patch submission for autofs filesystem on NetBSD ported from FreeBSD.
I'm aiming for this to get merged by NetBSD.

Autofs is a generic filesystem automounter filesystem, not limited to mounting NFS. It's been mentioned in the NetBSD project wiki as well.
https://wiki.netbsd.org/projects/project/new-automounter/

The initial patch is available from below. The first one is a Git format-patch style diff (I use Git mirror at https://github.com/NetBSD/src), and the second one is via diff -aNur. The patch is created on top of 8.99.9 as of around 2017.12.28. It also applies to 8.99.10, but recent src commit(s) broke buildworld on my environment. I didn't separate the patch into a series of patch sets, since most of the C code come from newly added subsystems, but I could if requested to do so.
https://leaf.dragonflybsd.org/~tkusumi/diff/netbsd/0001-NetBSD-autofs-v1.patch
https://leaf.dragonflybsd.org/~tkusumi/diff/netbsd/NetBSD-autofs-v1.patch

Online documentation and man pages for FreeBSD (which are also part of the patch) would/should work against NetBSD with a few exceptions. Below links are FreeBSD autofs info you can find on the web. Note that the FreeBSD autofs is not a port of Linux autofs which has existed for a long(er) time. There is no compatibility in configuration files and such, thus the same for NetBSD and Linux.
https://people.freebsd.org/~trasz/autofs.pdf
https://wiki.freebsd.org/Automounter
http://freebsdfoundation.blogspot.fi/2015/03/freebsd-from-trenches-using-autofs5-to_13.html

>How-To-Repeat:
How to use autofs

1. Add autofs_enable=YES to /etc/rc.conf.
2. Write configurations to /etc/auto_master. FreeBSD documentation should help, though configuration needed to do a simple NFS auto mounting is quite simple (1 line basically).
3. Reboot or restart autofs, have access to autofs(5) mount point, and see if auto mounting works. I've only tested with NFS (e.g. auto mount an exported FreeBSD directory via NFS via autofs), which is probably what majority of people use autofs for.

>Fix:
I'll write down some more details of newly added code.

---
Newly added subsystems

1. sys/fs/autofs - Autofs filesystem
Autofs(5) filesystem is a rewrite of the FreeBSD autofs(5) code. The autofs(5) filesystem catches some of the vnode operations from user processes, and notifies the user space daemon automountd(8) that auto mounting requests from user process have arrived. The character device /dev/autofs (with major# 343) is used as a communication channel between autofs(5) and automountd(8) daemon. Once automountd(8) receives the request from autofs(5) filesystem, it attempts to mount a target filesystem which corresponds to the autofs(5) mount point accessed by the user process, based on what auto_master(5) configuration file /etc/auto_master has. The actual mounting is done by a child process forked from automountd(8) using mount(8) and variants.

2. usr.sbin/autofs - Autofs user space command and daemons
User space code are basically the same as the original FreeBSD code (and DragonFlyBSD port) with minimum changes needed for NetBSD. The changes mostly come from difference in library/syscall interface, FreeBSD specific syscall, workaround code specific to NetBSD, etc. The directory contains followings.
* autmount(8) - A command to mount autofs(5) filesystem.
* automountd(8) - A daemon to auto mount target filesystems specified in /etc/auto_master.
* autounmountd(8) - A daemon to auto unmount the auto mounted filesystems if no longer used when the auto unmounting timer has expired.
These executable binaries actually consist of a single file, i.e. /usr/sbin/automountd and /usr/sbin/autounmountd are hardlinks to /usr/sbin/automount, and they do what they do by checking argv[0] in its main().

3. etc/auto_master and etc/autofs/* - Autofs configuration files
etc/auto_master is a configuration file which describes relations between autofs(5) mount points and target filesystems to auto mount. As mentioned above, automountd(8) mounts filesystems based on this file. The auto_master(5) man page explains details of the format. etc/autofs/* are collection of scripts used by auto_master(5) when pre-defined special mappings are used. These are just copied from FreeBSD unless a script contains FreeBSD specific stuff e.g. etc/autofs/special_media.

4. usr.sbin/fstyp - Filesystem type detection utility
Fstyp(8) is an utility to detect filesystem type of the given block device, and it was originally added to FreeBSD for autofs. It's been used by automountd(8) to detect filesystem type of block devices to auto mount when "-media" map is specified in auto_master(5). See auto_master(5) for details of "-media" map. Fstyp(8) supports filesystems such as UFS, ext2, ISO9660, FAT, NTFS. FreeBSD version also supports ZFS, and DragonFlyBSD version supports HAMMER, but these two are dropped in NetBSD. Note that auto mounting NFS doesn't require fstyp(8) or "-media" map.

---
Changes to the existing C code

1. Add MNT_AUTOMOUNTED to sys/sys/fstypes.h. This flag allows VFS to recognize auto mounted filesystems (i.e. not the autofs(5) itself, but filesystems mounted by automountd(8)). Filesystems mounted with this flag set are shown as "automounted" in mount(8) output.
2. Add MOUNT_AUTOFS to sys/sys/mount.h.
3. Add VT_AUTOFS to sys/sys/vnode.h.
4. Add -E option to usr.bin/showmount/showmount.c. This is used by "-hosts" map of autofs which is etc/autofs/special_hosts. This option exists in FreeBSD as well.
5. Add a warning (exporting of auto mounted fs isn't supported) to usr.sbin/mountd/mountd.c. This warning exists in FreeBSD as well.
6. Add EVFILT_FS event for kqueue(2) which gets triggered on mount and unmount of any filesystem. This is needed by autounmountd(8) daemon. EVFILT_FS has existed in FreeBSD for a long time, but not in NetBSD.

---
Notes

1. The autofs kernel code is located under sys/fs/autofs rather than sys/miscfs/autofs, since that's where FreeBSD autofs is located at. FreeBSD seems to be more or less aiming for compatibility with Solaris autofs, and Solaris autofs user space seems to assume autofs headers are under sys/fs/autofs. But I personally don't have any preference and free to change to sys/miscfs/autofs. DragonFlyBSD has it under sys/vfs/autofs as there is no sys/fs directory.
2. I've only added autofs to conf files for x86_64 (i.e. sys/arch/amd64/conf/...) which is the arch I used for porting autofs.
3. Changes to NetBSD configuration files, including above, are based on what other filesystems do, which may or may not be wrong in case of autofs. Any correction is appreciated.
4. "xxx first appeared in NetBSD <version>" in newly added man pages have "8.99.1" for <version> at the moment. There are several new man pages, such as autofs(5), auto_master(5), automount(8), automountd(8), autounmountd(8), fstyp(8). These man pages are all from FreeBSD as well.
5. The "-media" map is currently unsupported due to etc/autofs/special_media not being functional. The same script in FreeBSD uses GEOM sysctl, whereas there seems to be no easy way to achieve the same result on NetBSD. See the bottom part of etc/autofs/special_media for details. I'd apply if anyone knows how to do this on NetBSD, which is to list all possible block devices and partitions for fstyp to check filesystem type.

>Release-Note:

>Audit-Trail:
From: Thomas Klausner <wiz@NetBSD.org>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/52888: Add autofs for NetBSD
Date: Wed, 3 Jan 2018 10:41:35 +0100

 On Tue, Jan 02, 2018 at 07:15:00PM +0000, kusumi.tomohiro@gmail.com wrote:
 > >Number:         52888
 > >Category:       kern
 > >Synopsis:       Add autofs for NetBSD

 Thank you very much! That must have been a big piece of work.

 Some minor comments:

 > 4. usr.sbin/fstyp - Filesystem type detection utility
 > Fstyp(8) is an utility to detect filesystem type of the given block device, and it was originally added to FreeBSD for autofs. It's been used by automountd(8) to detect filesystem type of block devices to auto mount when "-media" map is specified in auto_master(5). See auto_master(5) for details of "-media" map. Fstyp(8) supports filesystems such as UFS, ext2, ISO9660, FAT, NTFS. FreeBSD version also supports ZFS, and DragonFlyBSD version supports HAMMER, but these two are dropped in NetBSD. Note that auto mounting NFS doesn't require fstyp(8) or "-media" map.

 NetBSD supports ZFS, so recognizing that would be useful.

 > 4. "xxx first appeared in NetBSD <version>" in newly added man pages have "8.99.1" for <version> at the moment. There are several new man pages, such as autofs(5), auto_master(5), automount(8), automountd(8), autounmountd(8), fstyp(8). These man pages are all from FreeBSD as well.

 We usually do not mention -current release numbers in man pages this
 way, so this should be "9.0" instead of "8.99.1".

 Cheers,
  Thomas

From: Kamil Rytarowski <n54@gmx.com>
To: gnats-bugs@NetBSD.org
Cc: 
Subject: Re: kern/52888: Add autofs for NetBSD
Date: Wed, 3 Jan 2018 11:40:16 +0100

 This is an OpenPGP/MIME signed message (RFC 4880 and 3156)
 --6GJqOMds2G1b6djie9ohxtKNWX2JJxWOm
 Content-Type: multipart/mixed; boundary="jDTjQ4vs3M89frR24iMDxJ3smjPpnGoc9";
  protected-headers="v1"
 From: Kamil Rytarowski <n54@gmx.com>
 To: gnats-bugs@NetBSD.org
 Message-ID: <97ff97fb-02a5-aa17-fa84-c17fde9f61bd@gmx.com>
 Subject: Re: kern/52888: Add autofs for NetBSD
 References: <pr-kern-52888@gnats.netbsd.org>
  <20180102191250.47F307A1DF@mollari.NetBSD.org>
  <20180102191500.25D397A1B9@mollari.NetBSD.org>
 In-Reply-To: <20180102191500.25D397A1B9@mollari.NetBSD.org>

 --jDTjQ4vs3M89frR24iMDxJ3smjPpnGoc9
 Content-Type: text/plain; charset=utf-8
 Content-Language: en-US
 Content-Transfer-Encoding: quoted-printable

 On 02.01.2018 20:15, kusumi.tomohiro@gmail.com wrote:
 > 1. Add autofs_enable=3DYES to /etc/rc.conf.

 I propose to keep the NetBSD style and call it autofs=3DYES.


 --jDTjQ4vs3M89frR24iMDxJ3smjPpnGoc9--

 --6GJqOMds2G1b6djie9ohxtKNWX2JJxWOm
 Content-Type: application/pgp-signature; name="signature.asc"
 Content-Description: OpenPGP digital signature
 Content-Disposition: attachment; filename="signature.asc"

 -----BEGIN PGP SIGNATURE-----

 iQJABAEBCAAqFiEELaxVpweEzw+lMDwuS7MI6bAudmwFAlpMsxgMHG41NEBnbXgu
 Y29tAAoJEEuzCOmwLnZsj+4P/17hdASdiVUBjXhiqPkUnvYRqds588cafj0kFic+
 2F7HrNSIO/Pz1QiYAXX6UKh0zvmVLHzCypdqnbQtDTQQsusuGO8ZEnLplcdvzSzu
 qRzTh0Wj8eTSB7fYSyg9hWweQ7bXU66qmBgjzmZUexWx1Xd+sAJ/o9Vmwg6JuIHi
 zRj2xn65eeA11WEoQ9x8mDZzVbuuiaEWVdSV9aBsV9sqfXXwLinVJ/LwV3uSQ6mX
 PVbHIpFqQXuF4iRfrYK4WVk/b0YVqTA2/3Wf3k+pL97YL1LukN9BvaSh2+nMpjKx
 gUTY1v7O6tKPr8CdRHJOtPpnvGta9nR1CUiZ7ON9u58tcSo4Yzc8mFvNCPo+Nq1U
 V75eN/o2sfEfe4C2hPleJCHcplK0nsJ26Wrdho/CdL8R7s7x9D2RZ1Q9b2WqInxF
 j1nD+ERM6yqXbedjmGL6DxtWNBce1u5jpq1HXcAj/7brW5mXF7aOJ4TxobRazQE5
 yZB6bWa53h0r9Y3L+fetqB+EP4Srqcw1ap4TBD0l+aILlS53m558x0nLPJlX6QGb
 /pJxUulItY2VCX72iQgmYDRyLs6yDO9pe9Vsq8SlJvUglJ3f+EsbyPCwrCxN4/zS
 JPqtH4Sbge7geEyX7Q/4giRtThfqRv5EAk8EsgNz5Glb2dAn1YxyLmH7XO5E4AQy
 zn/7
 =05qs
 -----END PGP SIGNATURE-----

 --6GJqOMds2G1b6djie9ohxtKNWX2JJxWOm--

From: Tomohiro Kusumi <kusumi.tomohiro@gmail.com>
To: gnats-bugs@netbsd.org
Cc: kern-bug-people@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/52888: Add autofs for NetBSD
Date: Sun, 7 Jan 2018 14:40:01 +0200

 Hi
 Thanks for the comments.
 I've uploaded -v2 patch which applies to 8.99.11 as of 2018.1.5.

 diff -aNur
 https://leaf.dragonflybsd.org/~tkusumi/diff/netbsd/NetBSD-autofs-v2.patch
 git format-patch
 https://leaf.dragonflybsd.org/~tkusumi/diff/netbsd/0001-NetBSD-autofs-v2.pa=
 tch

 Changes are as follows.
 1. Add ZFS support to fstyp(8) which gets compiled if ${MKZFS} !=3D
 "no". The reason for checking ${MKZFS} is because it requires ZFS
 libraries rather than just headers defining ondisk format (that are
 visible via -I${NETBSDSRCDIR}/external/cddl/... regardless of
 ${MKZFS}). This mechanism is pretty much the same with FreeBSD
 fstyp(8).
 2. Change the first-appeared-version in newly added man pages to 9.0.
 3. Change autofs_enable=3D to autofs=3D.
 4. Bring in recent updates from FreeBSD for etc/autofs/* scripts.

 I'll dogfood it for another few weeks, and then drop -v3 patch if any updat=
 e.
 Meanwhile any comments are appreciated.


 2018-01-03 11:45 GMT+02:00 Thomas Klausner <wiz@netbsd.org>:
 > The following reply was made to PR kern/52888; it has been noted by GNATS=
 .
 >
 > From: Thomas Klausner <wiz@NetBSD.org>
 > To: gnats-bugs@NetBSD.org
 > Cc:
 > Subject: Re: kern/52888: Add autofs for NetBSD
 > Date: Wed, 3 Jan 2018 10:41:35 +0100
 >
 >  On Tue, Jan 02, 2018 at 07:15:00PM +0000, kusumi.tomohiro@gmail.com wrot=
 e:
 >  > >Number:         52888
 >  > >Category:       kern
 >  > >Synopsis:       Add autofs for NetBSD
 >
 >  Thank you very much! That must have been a big piece of work.
 >
 >  Some minor comments:
 >
 >  > 4. usr.sbin/fstyp - Filesystem type detection utility
 >  > Fstyp(8) is an utility to detect filesystem type of the given block de=
 vice, and it was originally added to FreeBSD for autofs. It's been used by =
 automountd(8) to detect filesystem type of block devices to auto mount when=
  "-media" map is specified in auto_master(5). See auto_master(5) for detail=
 s of "-media" map. Fstyp(8) supports filesystems such as UFS, ext2, ISO9660=
 , FAT, NTFS. FreeBSD version also supports ZFS, and DragonFlyBSD version su=
 pports HAMMER, but these two are dropped in NetBSD. Note that auto mounting=
  NFS doesn't require fstyp(8) or "-media" map.
 >
 >  NetBSD supports ZFS, so recognizing that would be useful.
 >
 >  > 4. "xxx first appeared in NetBSD <version>" in newly added man pages h=
 ave "8.99.1" for <version> at the moment. There are several new man pages, =
 such as autofs(5), auto_master(5), automount(8), automountd(8), autounmount=
 d(8), fstyp(8). These man pages are all from FreeBSD as well.
 >
 >  We usually do not mention -current release numbers in man pages this
 >  way, so this should be "9.0" instead of "8.99.1".
 >
 >  Cheers,
 >   Thomas
 >

State-Changed-From-To: open->closed
State-Changed-By: dholland@NetBSD.org
State-Changed-When: Sun, 25 Jul 2021 03:56:49 +0000
State-Changed-Why:
my tree has autofs in it now, must have been merged :-)


>Unformatted:

Home	PR Database Search