NetBSD Problem Report #48449
From dholland@netbsd.org Sun Dec 15 07:21:53 2013
Return-Path: <dholland@netbsd.org>
Received: from mail.netbsd.org (mail.netbsd.org [149.20.53.66])
(using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits))
(Client CN "mail.NetBSD.org", Issuer "Postmaster NetBSD.org" (verified OK))
by mollari.NetBSD.org (Postfix) with ESMTPS id 86D05A5677
for <gnats-bugs@gnats.NetBSD.org>; Sun, 15 Dec 2013 07:21:53 +0000 (UTC)
Message-Id: <20131215072153.1786A14A2C9@mail.netbsd.org>
Date: Sun, 15 Dec 2013 07:21:53 +0000 (UTC)
From: dholland@NetBSD.org
Reply-To: dholland@NetBSD.org
To: gnats-bugs@gnats.NetBSD.org
Subject: y2038 shortcomings of ffsv1
X-Send-Pr-Version: 3.95
>Number: 48449
>Category: kern
>Synopsis: y2038 shortcomings of ffsv1
>Confidential: no
>Severity: serious
>Priority: medium
>Responsible: dholland
>State: open
>Class: sw-bug
>Submitter-Id: net
>Arrival-Date: Sun Dec 15 07:25:00 +0000 2013
>Last-Modified: Thu Jun 17 00:05:01 +0000 2021
>Originator: David A. Holland
>Release: NetBSD 6.99.28 (20131214)
>Organization:
>Environment:
n/a
>Description:
ffsv1 stores 32-bit times on disk. This is not fixable (solution is to
use ffsv2) but we ought to have a scheme in place where the 32-bit
times that are read and written are interpreted as current.
>How-To-Repeat:
Make an ffsv1 volume, set the date to 2040 and create some files. See
PR 48437 for an example seen in practice.
>Fix:
When 32-bit times are read in, instead of just sign-extending them as
currently appears to happen, paste on the top half of the current
64-bit time and then possibly adjust up or down by 0x100000000. There
are then several possible ways to pick the adjustment; one can match
on bits 30/31 of both times, or pick the version that's nearest the
current time; or assume that since file timestamps are rarely in the
future any time that would be more than a month or so in the future
should be interpreted in the past instead.
(This is basically the same as the y2k problem of deciding how to
interpret 2-digit years based on the current year, and I think a
standard scheme was eventually chosen for that but I forget what it
is; it should probably be replicated in binary for y2038.)
>Release-Note:
>Audit-Trail:
Responsible-Changed-From-To: kern-bug-people->dholland
Responsible-Changed-By: dholland@NetBSD.org
Responsible-Changed-When: Sun, 15 Dec 2013 07:35:00 +0000
Responsible-Changed-Why:
this is basically an entry in my todo queue; but if I store it in gnats,
maybe someone else will attend to it for me :-)
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@NetBSD.org
Cc:
Subject: Re: kern/48449: y2038 shortcomings of ffsv1
Date: Thu, 10 Jun 2021 05:26:00 +0000
On Sun, Dec 15, 2013 at 07:25:00AM +0000, dholland@NetBSD.org wrote:
> >Fix:
>
> When 32-bit times are read in, instead of just sign-extending them as
> currently appears to happen, paste on the top half of the current
> 64-bit time and then possibly adjust up or down by 0x100000000.
Or, much more simply, store a 64-bit epoch in the superblock (this
lets you pick it at newfs time) and add the 32-bit times to it. This
requires adding a superblock field but we can probably afford that.
--
David A. Holland
dholland@netbsd.org
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/48449: y2038 shortcomings of ffsv1
Date: Thu, 10 Jun 2021 18:25:27 +0700
Date: Thu, 10 Jun 2021 05:30:02 +0000 (UTC)
From: David Holland <dholland-bugs@netbsd.org>
Message-ID: <20210610053002.E2EEC1A923B@mollari.NetBSD.org>
| Or, much more simply, store a 64-bit epoch in the superblock (this
| lets you pick it at newfs time) and add the 32-bit times to it. This
| requires adding a superblock field but we can probably afford that.
No, that's is almost the worst of all possible solutions (even including
just doing nothing). It is simple and easy to implement in the steady
state, but the transition mechanism is horrid. If you really meant *only*
at newfs time, then it is a waste of effort, anyone doing a newfs can set -O2
and the issue doesn't arise. The problem is with all the existing FFSv1
filesystems, not new ones (and doing a dump/restore to handle the problem
is way too invasive, and risky).
While it is possible to set up a method whereby the offset can be added to
(or changed in, which is the same thing) an existing filesystem, doing it
is far more complex than is needed, as maintaining constant timestamps
means that if the offset alters, so must every timestamp in every inode,
and as that can't be done instantaneously, there's the possibility that the
filesystem might get into a state where some inodes have been converted,
and others not, which means inodes would need a state bit (old or new offset)
and the superblock would need two new offset fields. All a giant mess.
Further, no matter what is done, a FFSv1 cannot store a range of timestamps
that spans more than 140 years (approx), so any solution needs to live with
that - which really means that we need to make FFSv1 vanish by 2100 or so.
That is aside from perhaps access to some ancient filesystems in read only
(or at least no time update) mode.
Creating an environment where we can live with new FFSv1 filesystems created
hundreds of years from now, because they can have a base offset at or near
that time (and so would work fine) should not be an objective - assuming that
FFS filesystems (in any form) will survive that long in common usage is
something of a stretch, planning to make it possible is absurd - even if we
do still want them then, the extra overhead of FFSv2 (or successors) will be
irrelevant (it already is with today's storage sizes, compared with when it
was created, and it was OK then).
The "paste in the top half of the current time" is one solution that is even
worse than this suggestion, as it is really just the same thing with a
dynamically varying offset, and no flexibility at all.
A very simple solution which will give us time to phase out (most, see below
for the workaround) FFSv1 completely (within another 80/90 years from now) is
to add a single extra bit to the superblock, which, when set, indicates that
all times stored in the filesystem are unsigned. That alters the range of
times from 1907..2038 to 1970..2100 (give or take) instantly.
To that we add a tool that scans the filesystem (or an option to fsck more
likely) which determines if it is safe to set the bit (ie: there are no
times with the top bit set anywhere) which should be true if no-one has
been cheating (no-one really ever created/modified/accessed a FFS filesystem
file before 1970 or after 2037) and if so, simply set the bit and that
filesystem becomes safe to continue using until 2100 (after which, because
of the range issue above, nothing can really extend it by more than a few
years). The kernel just needs one change to deal with this - when converting
FFSv1 timestamps to time_t, check the bit in the superblock (copied to
the mount struct) and if set, zero extend rather than sign extend the value.
Nothing needs to change going the other way (except possibly defeating some
compiler sign/range warnings, but that's probably already done).
Simple, safe, and works. Closer to 2100 we can drop support in the
kernel for mounting FFSv1 filesystem at all (see below).
For filesystems with times before 1970 or after 2037 (ie: anything with the
top bit currently set) we can offer to fix them (set the value to something
else of the user's choosing) and until that is fixed, hopefully sometime
in the next 15 years, simply leave the bit clear, and the filesystem works
the same (for that period) as it does now.
In 2038 (or the years just before) we change the kernel to refuse to mount
FFSv1 filesystems with the bit clear (see below) at least in rw mode (ro
mode mounts can continue as long as the kernel retains FFSv1 support).
There's a slightly more general method we could use if there are users
with a legitimate reason for files with timestamps that predate 1970, but
only perhaps for a decade or two or three (ie: files from older, pre-unix,
systems, but created in the computer age ... seems doubtful to me, but
perhaps someone has such things).
For that, we just need to recognise that while the 140 year range is all
that FFSv1 can possibly deal with, nothing says that the 0 value must
occur in the centre of that range. We could allow timestamps that range
from 1950..2080 (approx) for example (and the range can vary per filesystem).
This requires a (32 bits is enough) field in the superblock, not to store
the origin, but to store the minimum (or max, makes no real difference) value
of the range, and then we simply convert between those values and time_t.
Once again time_t -> FFS time is a no-op conversion (just truncate the
value to the low 32 bits) but this time FFS->time_t requires a comparison,
if the (unsigned) FFS value is less than N (the unsigned version of the
minimum (maybe maximum, they turn out to be almost the same thing) value
of the supported range, for that filesystem) then treat the value as signed,
sign extend it to become a time_t. If it is greater (or equal), then zero
extend instead. Or some arithmetic just like that (the precise formula can
come later, if we need this, but it is that simple). This becomes the same
as the "one bit" version when the range is set to be 1970..2100 (or whatever)
ie: all values become zero extended as the other side of the test can never
be true. Again, when the current time passes the end of the range, the
kernel refuses to (rw) mount that filesystem any more.
To deal with old filesystems into the distant future (this is the "below"
referred to above) we need a user level (FUSE I assume) FFSv1 filesystem
converter which can be configured via mount options to convert timestamps
however is appropriate (there could even be a range of such things for
different types of conversion, especially when allowing writes, and deciding
how to store timestamps that are out of the range of what the filesystem
could normally handle ... since performance doesn't matter any more, and
special cases are definitely allowed, all kinds of possibilities exist
for what to do). Fortunately we have some decades before this will
really be required (and by then, it might not be).
Just please, no "variable time" nonsense in the filesystems, an inode
with a timestamp that currently converts as Thu Jun 10 11:01:00 UTC 2021
must always convert as exactly that (over the range, 1970-2038),
never anything else, or not from the kernel anyway (what a user level hack
might do is up to it, and if the user is content, that's good enough).
kre
From: David Holland <dholland@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/48449: y2038 shortcomings of ffsv1
Date: Fri, 11 Jun 2021 22:31:10 +0000
On Thu, Jun 10, 2021 at 11:30:02AM +0000, Robert Elz wrote:
> | Or, much more simply, store a 64-bit epoch in the superblock (this
> | lets you pick it at newfs time) and add the 32-bit times to it. This
> | requires adding a superblock field but we can probably afford that.
>
> No, that's is almost the worst of all possible solutions (even
> including just doing nothing). It is simple and easy to implement
> in the steady state, but the transition mechanism is horrid. If
> you really meant *only* at newfs time, then it is a waste of
> effort, anyone doing a newfs can set -O2 and the issue doesn't
> arise. The problem is with all the existing FFSv1 filesystems, not
> new ones (and doing a dump/restore to handle the problem is way too
> invasive, and risky).
No. Unless ffsv1 is deprecated entirely, some way to store a wider
range of dates needs to be hacked in -- it is not reasonable to try to
use a filesystem that can't store the current date.
Your argument seems to be that ffsv1 should be deprecated entirely;
that's fine if we can pull it off, but given that among other things
there are machines with ffs-reading firmware that don't understand
ffsv2, probably isn't realistic.
Conversely, I was assuming that only new volumes matter and that's
probably also not true.
> While it is possible to set up a method whereby the offset can be
> added to (or changed in, which is the same thing) an existing
> filesystem, doing it is far more complex than is needed, as
> maintaining constant timestamps means that if the offset alters, so
> must every timestamp in every inode, and as that can't be done
> instantaneously, there's the possibility that the filesystem might
> get into a state where some inodes have been converted, and others
> not, which means inodes would need a state bit (old or new offset)
> and the superblock would need two new offset fields. All a giant
> mess.
It's not hard to do this and do it in a way that's adequately
recoverable/restartable. It's nowhere near as complicated as
resize_ffs.
> Further, no matter what is done, a FFSv1 cannot store a range of timestamps
> that spans more than 140 years (approx), so any solution needs to live with
> that - which really means that we need to make FFSv1 vanish by 2100 or so.
> That is aside from perhaps access to some ancient filesystems in read only
> (or at least no time update) mode.
Maybe. It isn't clear that large timespans are necessary in practice,
especially for system volumes, where ordinarily nothing is older than
the original OS install. (And it doesn't matter if unpacking some
200-year-old 3rd-party source tarball to build it gives it all
timestamps of INT_MIN.)
Maybe in two hundred years people will have heirloom dirs full of
their grandparents' MS Word files, but I wouldn't be surprised if
most people didn't care in practice about timestamps older than they
are.
> A very simple solution which will give us time to phase out (most,
> see below for the workaround) FFSv1 completely (within another
> 80/90 years from now) is to add a single extra bit to the
> superblock, which, when set, indicates that all times stored in the
> filesystem are unsigned. That alters the range of times from
> 1907..2038 to 1970..2100 (give or take) instantly.
If one is going to change the on-disk format at all, it's best to not
do it halfassedly. In the computer biz doing things halfassedly is
almost always a mistake, especially in the long term.
The biggest costs of either of these approaches (or any other) are (a)
negotiating the on-disk format change and (b) identifying all the code
paths in the system that need adjustment; these cost pretty much the
same no matter what the change is.
--
David A. Holland
dholland@netbsd.org
From: matthew green <mrg@eterna.com.au>
To: gnats-bugs@netbsd.org
Cc: dholland@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: re: kern/48449: y2038 shortcomings of ffsv1
Date: Sat, 12 Jun 2021 12:33:48 +1000
one thing is that not all platforms can cope without ffsv1.
the canonical example we have is shark, which reads netbsd
kernels out of ffsv1 and they have to appear to be a.out.
this is a firmware limitation, and the one person who has
attempted to create an ofwboot for shark failed.
From: mlelstv@serpens.de (Michael van Elst)
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/48449: y2038 shortcomings of ffsv1
Date: Sat, 12 Jun 2021 06:27:05 -0000 (UTC)
mrg@eterna.com.au (matthew green) writes:
>one thing is that not all platforms can cope without ffsv1.
>the canonical example we have is shark, which reads netbsd
>kernels out of ffsv1 and they have to appear to be a.out.
>this is a firmware limitation, and the one person who has
>attempted to create an ofwboot for shark failed.
On the other hand, having correct timestamps on such a boot
filesystem aren't that important.
N.B. timestamps that cannot be represented shouldn't be truncated
but saturated. Time then ends with 2^32-1.
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/48449: y2038 shortcomings of ffsv1
Date: Sat, 12 Jun 2021 18:42:01 +0700
Date: Fri, 11 Jun 2021 22:35:01 +0000 (UTC)
From: David Holland <dholland@netbsd.org>
Message-ID: <20210611223501.5DC9C1A923B@mollari.NetBSD.org>
| Your argument seems to be that ffsv1 should be deprecated entirely;
Almost entirely. Removed from the kernel (at about the end of this
century -- none of us will ever see it happen... (probably))
| that's fine if we can pull it off, but given that among other things
| there are machines with ffs-reading firmware that don't understand
| ffsv2, probably isn't realistic.
As I said, we can keep support for FFSv1 for this kind of use via FUSE,
or otherwise (makefs would be sufficient too, for this only the firmware
needs to be able to read the thing) with user level conversion that can
be tailored to the needs, and do whatever is best to do, given the limitations.
But trying to keep FFSv1 alive in general is folly -- remember that its
limited time range is one of its lesser problems, it also has limited
uid/gid ranges, and more importantly, limited block number ranges - it
isn't quite the case yet that you cannot buy a disk small enough for FFSv1
to work on it, but we're approaching that point. And while partitioning
can avoid that problem, only if the partitioning scheme works for the reason
that you're needing FFSv1 in the first place - I am fairly sure nothing that
has ffs reading firmware has GPT support in there, and disklabels (and MBR,
but that tends not to be relevant) are also block number size limited, so
they don't help either. FFSv1 is simply doomed.
We cope with keeping this stuff working (assuming that any of it will still
exist to work in the next century - emulators can easily get given "updated
firmware" in this area, so they can boot other ways than the genuine article).
in the same way we keep support for a.out which this kind of firmware also
tends to expect, not by keeping it as a first class citizen, but by making
tools which support it in a somewhat hackish way.
| It's not hard to do this and do it in a way that's adequately
| recoverable/restartable. It's nowhere near as complicated as
| resize_ffs.
["this" is changing the time offset in a FFSv1 filesystem]
It terms of implementation complexity, no, that's easy. The problem is
that this conversion requires touching every (allocated) inode in the
filesystem, and that takes time (human measurable time) and so must be
done in a way that's restartable. The only way that really works for
this is a "converted" bit in each inode (trying to do the inodes in
ascending sequence and recording where we're up to has non-atomic update
problems, so isn't safe). The issue here is that FFSv1 inodes really
have no suitable free bits (there are free bits in the flags word, but
those aren't really suitable for this - though it could be forced in I
guess).
Incidentally there's another unbelievably gross hack that could be done
to extend the time range of FFSv1 (from 136 years or whatever, to > 500
years) ... each of the time fields is really 2 32 bit values. One for the
(old) time_t (seconds) which is the one that is causing all of these
problems, the other is the nanoseconds field (seconds+nanoseconds == the
timestamp). One property of nanoseconds is that their range of values
is 0..999999999 (they're never negative in this use) and 999999999
only takes 30 bits to represent. That means we have two currently
always zero bits associated with each time field in every inode (the
"last write time" fields in the superblock and cylinder groups don't
have this I think, but there are other spare bits in those things
which could be co-opted).
We could use those 2 bits to give us a 34 bits of seconds value instead
of 32 (whether that's 34 bits unsigned, or signed, would be a question for
whoever decided this approach wasn't too absurdly ugly and decided to
try it.)
| Maybe. It isn't clear that large timespans are necessary in practice,
| especially for system volumes, where ordinarily nothing is older than
| the original OS install.
Once again, nothing cares about FFSv1 for this kind of purpose.
Boot filesystems for those architectures that need it can be just
that, they don't need to have dual role as the root, and they don't
need to be (kernel supported) mountable.
The only system volumes that matter are the ones that already exist,
and those have times old enough to limit what we can possibly do with
modifiable time bases. If one were to assume that no NetBSD FFSv1 can
be older than 1990, we could use unsigned times and allow up to 2125
or something, rather than the 2105 we'd get with the 1970 origin. Is
that extra 20 years really worth the effort? The latest we can possibly
move forward would be a base of 2022 (on the assumption that we tell people
to stop using FFSv1 now, so any (real usable, not firmware boot) FFSv1
will have been created 2021 or earlier, and so have dates 2021 (or earlier)
in the filesystem - which means the upper limit on (unsigned) 32 bit
time support is 2158 (and almost certainly quite a lot earlier).
Of course, all of this kind of analysis ignores the fact that when files
are restored to a filesystem, their (older) dates come along with them,
which is how my (I think 3 year old now) laptop has old e-mail messages
(and other files) with dates from the mid 1990's.
Those are the times that matter - and no, you cannot arbitrarily change
the mod times of files that exist. (Not that this matters to me, of course,
I'm not using FFSv1, why would I?)
| (And it doesn't matter if unpacking some
| 200-year-old 3rd-party source tarball to build it gives it all
| timestamps of INT_MIN.)
It does if that source tarball contains pre-built files which will now
not look younger than their sources, and require 200-year-old (no longer
supported) 3rd-party applications to regenerate them. Of course, hacks
can be done, but why would anyone be inflicting FFSv1 upon themselves
when they don't need it?
| If one is going to change the on-disk format at all, it's best to not
| do it halfassedly. In the computer biz doing things halfassedly is
| almost always a mistake, especially in the long term.
Sure, but there's no long term here, all of this is just to keep FFSv1
workable just barely long enough that no-one cares any more. After that
ancient filesystems can be accessed via user level tools (unpacking a
(max sized) 2TB FFSv1 into a tmpfs using a "dump/restore" or similar
procedure will be nothing ... the tmpfs wouldn't even notice that 2TB
used (a fraction of a percent of its available space probably).
| The biggest costs of either of these approaches (or any other) are (a)
| negotiating the on-disk format change and
That's this discussion, right?
| (b) identifying all the code
| paths in the system that need adjustment; these cost pretty much the
| same no matter what the change is.
Yes.
But those are only the significant costs if the result is so good that
there are never any problems. Otherwise, over time, dealing with the
various issues becomes orders of magnitude more than these.
Since I see no way (certainly none that has been suggested, including mine)
which has any potential of being issue free, that is the cost that really
matters, and the only way I see to be rid of that is to be rid of FFSv1
(as a first class object that anyone can simply use).
kre
ps: when considering 2038 issues, while dealing with 32 bit times in
FFSv1 is a problem, a much bigger problem approaching in 2038 is what
happens with old applications linked with libs that use the 32 bit time_t
interfaces everywhere. That one we cannot simply wiggle our noses at and
magically cause things to alter - those applications know the range of
a (32 bit) time_t, and aren't going to alter themselves.
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/48449: y2038 shortcomings of ffsv1
Date: Wed, 16 Jun 2021 00:52:32 +0000
On Sat, Jun 12, 2021 at 11:45:02AM +0000, Robert Elz wrote:
> | Your argument seems to be that ffsv1 should be deprecated entirely;
>
> Almost entirely. Removed from the kernel (at about the end of this
> century -- none of us will ever see it happen... (probably))
> [...]
Maybe. However, to keep it even that long something has to be done
about the timestamps, and there's no compelling reason to do a
halfassed job of that.
> | It's not hard to do this and do it in a way that's adequately
> | recoverable/restartable. It's nowhere near as complicated as
> | resize_ffs.
>
> ["this" is changing the time offset in a FFSv1 filesystem]
>
> It terms of implementation complexity, no, that's easy. The problem is
> that this conversion requires touching every (allocated) inode in the
> filesystem, and that takes time (human measurable time) and so must be
> done in a way that's restartable. The only way that really works for
> this is [...]
Eh, it's a very simple exercise in journaling. Create a file
/.epoch (or something), write the old and new epochs into it, fsync.
Then for each cylinder group, read all the inodes, write the
timestamps into /.epoch, fsync it, then update the inodes one at a
time, then sync and go on to the next cylinder group. At the end,
update the epoch in the superblock and delete /.epoch.
If you crash, on restart you read /.epoch and resume updating the
inodes in the last cylinder group you were working on. You can tell
which ones got updated and written out before the crash because you
wrote out all the original timestamps before you started changing
them.
> Incidentally there's another unbelievably gross hack that could be done
> to extend the time range of FFSv1 (from 136 years or whatever, to > 500
> years) ... each of the time fields is really 2 32 bit values. One for the
> (old) time_t (seconds) which is the one that is causing all of these
> problems, the other is the nanoseconds field (seconds+nanoseconds == the
> timestamp). One property of nanoseconds is that their range of values
> is 0..999999999 (they're never negative in this use) and 999999999
> only takes 30 bits to represent. That means we have two currently
> always zero bits associated with each time field in every inode (the
> "last write time" fields in the superblock and cylinder groups don't
> have this I think, but there are other spare bits in those things
> which could be co-opted).
Or truncate to microseconds, which gives 12 extra bits.
> | The biggest costs of either of these approaches (or any other) are (a)
> | negotiating the on-disk format change and
>
> That's this discussion, right?
No, that's agreeing with FreeBSD on what the exact on-disk format
change should be, so that interoperability remains.
> Since I see no way (certainly none that has been suggested, including mine)
> which has any potential of being issue free, that is the cost that really
> matters, and the only way I see to be rid of that is to be rid of FFSv1
> (as a first class object that anyone can simply use).
By that standard we should be rid of everything else too :-)
> ps: when considering 2038 issues, while dealing with 32 bit times in
> FFSv1 is a problem, a much bigger problem approaching in 2038 is what
> happens with old applications linked with libs that use the 32 bit time_t
> interfaces everywhere. That one we cannot simply wiggle our noses at and
> magically cause things to alter - those applications know the range of
> a (32 bit) time_t, and aren't going to alter themselves.
They will break, or need to be run in a VM that lives in the past.
Nothing can reasonably be done about that.
--
David A. Holland
dholland@netbsd.org
From: Robert Elz <kre@munnari.OZ.AU>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/48449: y2038 shortcomings of ffsv1
Date: Wed, 16 Jun 2021 16:34:38 +0700
Date: Wed, 16 Jun 2021 00:55:01 +0000 (UTC)
From: David Holland <dholland-bugs@netbsd.org>
Message-ID: <20210616005501.3F1121A923B@mollari.NetBSD.org>
| Maybe. However, to keep it even that long something has to be done
| about the timestamps, and there's no compelling reason to do a
| halfassed job of that.
Not half assed, just simple, correct, and all that is actually needed.
There is zero reason to keep supporting building new general purpose
FFSv1 filesystems into the indefinite future. There are plenty of
reasons not to.
| Eh, it's a very simple exercise in journaling.
That's not simple, and seems to require doing the modifications to a
running filesystem (otherwise "create a file" is not nearly as simple,
and in any case doesn't work for full filesystems). On a running
filesystem you also need to deal with inodes that are currently in the
vnode cache.
All of this for a pointless change...
[On my bizarre suggestion to steal the upper 2 bits from the
nanoseconds field]:
| Or truncate to microseconds, which gives 12 extra bits.
That would also require modifying the existing inodes, which really
is the one thing that we don't want to require. And while that would
possibly allow FFSv1 to outlast humanity (> half a million years), I don't
see any point keeping it beyond the end of this century, and certainly
not beyond the 500 year range that 2 extra bits would provide (not that
doing that is rational either).
| No, that's agreeing with FreeBSD on what the exact on-disk format
| change should be, so that interoperability remains.
OK. And of the various suggestions that have been made here, which do
you believe that they're more likely to agree to?
I will talk to Kirk about it.
| They will break, or need to be run in a VM that lives in the past.
| Nothing can reasonably be done about that.
Hmm - I wonder if the "VM that lives in the past" could be the kernel with
a hack to shuffle time in some way - I'm not sure it is really possible
to deal with stat() of a file modified in 2020 and time() returning a
value indicating that the current year is 1985 though. "break" is looking
more and more probable... Sad.
kre
From: =?UTF-8?B?SmFyb23DrXIgRG9sZcSNZWs=?= <jaromir.dolecek@gmail.com>
To: "gnats-bugs@NetBSD.org" <gnats-bugs@netbsd.org>
Cc:
Subject: Re: kern/48449: y2038 shortcomings of ffsv1
Date: Wed, 16 Jun 2021 22:33:27 +0200
Le mer. 16 juin 2021 =C3=A0 11:40, Robert Elz <kre@munnari.oz.au> a =C3=A9c=
rit :
> All of this for a pointless change...
>
> [On my bizarre suggestion to steal the upper 2 bits from the
> nanoseconds field]:
I really like the suggestion for reusing the unused upper 2 bits for
the time for FFSv1 filesystems. It's guaranteed to be unused on all
existing filesystems, and does not require any changes to existing
inodes nor new superblock fields. Perfect for when you fire up some
old forgotten VM in 30 years.
It should be also relatively easy to convince other BSDs to mask out
those bits at least, so it would be somewhat interoperable.
As for active support for old FFSv1 hardware .. Well, I don't know. Is
there actually anything produced in the last 20 years? How likely is
it that any of that will be still operational in another 18 years?
Jaromir
From: Joerg Sonnenberger <joerg@bec.de>
To: gnats-bugs@netbsd.org
Cc: dholland@netbsd.org, gnats-admin@netbsd.org, netbsd-bugs@netbsd.org
Subject: Re: kern/48449: y2038 shortcomings of ffsv1
Date: Thu, 17 Jun 2021 00:44:56 +0200
On Wed, Jun 16, 2021 at 08:35:01PM +0000, Jaromír Doleček wrote:
> The following reply was made to PR kern/48449; it has been noted by GNATS.
>
> From: =?UTF-8?B?SmFyb23DrXIgRG9sZcSNZWs=?= <jaromir.dolecek@gmail.com>
> To: "gnats-bugs@NetBSD.org" <gnats-bugs@netbsd.org>
> Cc:
> Subject: Re: kern/48449: y2038 shortcomings of ffsv1
> Date: Wed, 16 Jun 2021 22:33:27 +0200
>
> Le mer. 16 juin 2021 =C3=A0 11:40, Robert Elz <kre@munnari.oz.au> a =C3=A9c=
> rit :
> > All of this for a pointless change...
> >
> > [On my bizarre suggestion to steal the upper 2 bits from the
> > nanoseconds field]:
>
> I really like the suggestion for reusing the unused upper 2 bits for
> the time for FFSv1 filesystems. It's guaranteed to be unused on all
> existing filesystems, and does not require any changes to existing
> inodes nor new superblock fields. Perfect for when you fire up some
> old forgotten VM in 30 years.
The easiest way would actually be to use the sign bit as flag and switch
to a different time encoding iff it is set.
Joerg
From: David Holland <dholland-bugs@netbsd.org>
To: gnats-bugs@netbsd.org
Cc:
Subject: Re: kern/48449: y2038 shortcomings of ffsv1
Date: Thu, 17 Jun 2021 00:00:02 +0000
On Wed, Jun 16, 2021 at 09:40:01AM +0000, Robert Elz wrote:
> | Maybe. However, to keep it even that long something has to be done
> | about the timestamps, and there's no compelling reason to do a
> | halfassed job of that.
>
> Not half assed, just simple, correct, and all that is actually needed.
>
> There is zero reason to keep supporting building new general purpose
> FFSv1 filesystems into the indefinite future. There are plenty of
> reasons not to.
Timestamp issues are not a compelling reason not to.
> | Eh, it's a very simple exercise in journaling.
>
> That's not simple, and seems to require doing the modifications to a
> running filesystem (otherwise "create a file" is not nearly as simple,
> and in any case doesn't work for full filesystems). On a running
> filesystem you also need to deal with inodes that are currently in the
> vnode cache.
Creating a file is still pretty straightforward. Or you could create
the file first with enough space in it to serve, then unmount.
And yes, it is simple. Now let me explain undo/redo logging with
rolling checkpoints...... :-)
> | Or truncate to microseconds, which gives 12 extra bits.
>
> That would also require modifying the existing inodes, which really
> is the one thing that we don't want to require.
Again, there is absolutely no problem updating existing inodes.
> | No, that's agreeing with FreeBSD on what the exact on-disk format
> | change should be, so that interoperability remains.
>
> OK. And of the various suggestions that have been made here, which do
> you believe that they're more likely to agree to?
>
> I will talk to Kirk about it.
If you do it starting from the assumption that it's pointless to do
anything, you aren't likely to get him to disagree :-)
Anyway, truncating to microseconds is probably the best approach of
the ones discussed so far, and it only requires a flag in the
superblock, not a whole field.
> | They will break, or need to be run in a VM that lives in the past.
> | Nothing can reasonably be done about that.
>
> Hmm - I wonder if the "VM that lives in the past" could be the kernel with
> a hack to shuffle time in some way - I'm not sure it is really possible
> to deal with stat() of a file modified in 2020 and time() returning a
> value indicating that the current year is 1985 though. "break" is looking
> more and more probable... Sad.
You'd need to shuffle all times so they stay in order, at least in
general. No reason that can't be a kernel patch, but setting the clock
in a VM is simpler.
It might be possible to _scale_ all the times by a factor of 2 (so
1980 reads as 1975, 1990 as 1980, etc.) up to some limit like say 24
hours prior to current (so realtime computations don't go off the
rails); that would need to be a kernel patch but probably wouldn't be
all that horrible to do, and could be a compat mode coexisting with
normal processes as long as the processes in the asylum don't try to
exchange timestamps with them.
--
David A. Holland
dholland@netbsd.org
>Unformatted:
(Contact us)
$NetBSD: query-full-pr,v 1.46 2020/01/03 16:35:01 leot Exp $
$NetBSD: gnats_config.sh,v 1.9 2014/08/02 14:16:04 spz Exp $
Copyright © 1994-2020
The NetBSD Foundation, Inc. ALL RIGHTS RESERVED.
Home