Subject: [nfsv4] Question about NFSv4.2 SEEK HOLE at the end of the file

From: Olga Kornievskaia <[email-masked]>
Date: Fri, 6 Aug 2021 14:01:42 -0400

Hi folks,

I would like to get some clarification with regards to SEEK behaviour
and validity of offset - sa_offset - value.

The spec says that each file must have a hole at the end of the file.
The spec also talks about if the offset is beyond the end of the file
the server returns an error (ERR_NXIO).

First I would like to confirm/establish that
1. offset=file_length-1 is the last valid offset value.
2. offset=file_length and probably uncondendent offset > file_length
are "beyond the end of the file"?

There was a thread before (by Rick Macklem) which considered
offset=file_length for SEEK which Dave was arguing that because there
is an implicit hole then looking for hole at offset=file_length (or it
stems to think any other offset beyond the end of the file) would
"find" the sa_what and thus should return OK, eof=1,
offset=file_length.

Then what check comes first: the check for validity of offset or
condition that there must be an implicit hole at the end of the file?

If the validity of offset comes first, then what happens when
file_size is 0 and we are SEEKing for HOLE offset=0. Then offset 0 is
"beyond" the end of the file and the server must return ERR_NXIO but
then we can't satisfy "each file has a hole at the end".

If the "hole" condition comes first, then for SEEK (HOLE) are there
any invalid offsets? It seems that SEEK HOLE will never return
ERR_NXIO.

Or is offset=file_length (either not technically invalid) should be
considered a valid value but offset > file_length is from where SEEK
returns ERR_NXIO?

---

From: David Noveck <[email-masked]>
Date: Fri, 6 Aug 2021 20:46:41 -0400

> and validity of offset - sa_offset - value.

I'll try to provide some clarifying remarks but no guarantees.

> The spec says that each file must have a hole at the end of the file.

Actually it says "MUST".   I have some doubts about that so you might not
wind up with the clarification you are looking for :-(

> It does.  It also uses a "MUST".

> 1. offset=file_length-1 is the last valid offset value.

that implies that, for a zero length file.minus one is the last valid
offset, i.e. that there
really is none.   I would disagree and feel that offset zero (only) is a
valid offset value.

> are "beyond the end of the file"?

I believe that offset=file-length is *at* the end of file, rather than
beyond it.

> is an implicit hole then looking for hole at offset=file_length

I probably said that.

> (or it
> stems to think any other offset beyond the end of the file)

never said that and disagree now.

would
> "find" the sa_what and thus should return OK, eof=1,

agree with that but only for the offset=file_length.

> condition that there must be an implicit hole at the end of the file?

It isn't a question of priority.  if you have two "MUST"''s that are
contradictory, the
spec is broken.   Maybe some 4/1, I'll submit an i-d proposing definitions
for "BROKEN" and "BOGUS" :-)

> file_size is 0 and.

we are SEEKing for HOLE offset=0. Then offset 0 is
> "beyond" the end of the file

it isn't.

> and the server must return ERR_NXIO but
> You can satisfy that but note that  the spec does not say that any
particular request has to report that fact.

The reason that "MUST" is wrong is the case of a file of length 2**64.
Such a file cannot
have a hole at the end so that "MUST" needs to be revised.   I'll think
about an
errata report.

> any invalid offsets?

Any one beyond the end of file.

> It seems that SEEK HOLE will never return
> ERR_NXIO.
>

it doesn't seem so to me.

> considered a valid value but offset > file_length is from where SEEK
.
I think so.

> _______________________________________________
> nfsv4 mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/nfsv4
>

---

From: Rick Macklem <[email-masked]>
Date: Sat, 7 Aug 2021 02:44:36 +0000

David Noveck wrote:
[good stuff snipped]
> The reason that "MUST" is wrong is the case of a file of length 2**64.  Such a file cannot
> have a hole at the end so that "MUST" needs to be revised.   I'll think about an
> errata report.
Since the Size attribute is uint64_t, doesn't that imply that files >= 2**64 are not supported
by NFSv4?

rick

[email-masked]<mailto:[email-masked]>

---

From: David Noveck <[email-masked]>
Date: Sat, 7 Aug 2021 06:02:34 -0400

> Since the Size attribute is uint64_t, doesn't that imply that files >=

Yes, it does.  However, there will be limit,. A file of length 2**64-1 can
exist and would not be extendible.  Such a file could not have a hole
following the data and the spec says it "MUST" have one.

---

From: Olga Kornievskaia <[email-masked]>
Date: Sat, 7 Aug 2021 10:41:27 -0400

On Fri, Aug 6, 2021 at 8:47 PM David Noveck <[email-masked]> wrote:
> that implies that, for a zero length file.minus one is the last valid offset, i.e. that there
> really is none.   I would disagree and feel that offset zero (only) is a valid offset value.

Thanks. I think this clarifies things. If offset=file_length is a
valid offset then we don't have an issue with 0 file size and SEEK.

> I believe that offset=file-length is at the end of file, rather than beyond it.
>>
> It isn't a question of priority.  if you have two "MUST"''s that are contradictory, the
> spec is broken.   Maybe some 4/1, I'll submit an i-d proposing definitions for "BROKEN" and "BOGUS" :-)
>>
>>
> You can satisfy that but note that  the spec does not say that any particular request has to report that fact.
>

---

From: Olga Kornievskaia <[email-masked]>
Date: Sat, 7 Aug 2021 11:33:16 -0400

> valid offset then we don't have an issue with 0 file size and SEEK.

I would like to say that Linux (local file system implementations, eg
xfs) consider offset=eof to be invalid and return ERR_NXIO. Also since
an NFS client for any version less than 4.2, would call into the
generic VFS implementation would be subjected to the same semantics.
Seems like we'd get different answers if we do lseek thru v3 or 4.2.

Is there a generic definition of a valid offset range or does a file
system can choose what its validity range is?

---

From: Rick Macklem <[email-masked]>
Date: Sat, 7 Aug 2021 15:54:11 +0000

David Noveck wrote:
>On Fri, Aug 6, 2021, 10:44 PM Rick Macklem <[email-masked]<mailto:[email-masked]>> wrote:
>David Noveck wrote:
>[good stuff snipped]
>Yes, it does.  However, there will be limit,. A file of length 2**64-1 can exist and would not be extendible.  Such a file >could not have a hole following the data and the spec says it "MUST" have one.
My interpretation of saying there is a hole at EOF simply means that a:
SEEK for SEEK_HOLE with an offset past last hole in file (before EOF) will return sr_offset == file_length,
which would be 2**64-1. I don't think saying there is a hole at EOF implies that the hole must be replacible
by data. I haven't looked lately, but I thought the POSIX draft for SEEK_DATA/SEEK_HOLE described it as a
"virtual hole at EOF".

rick

rick

[email-masked]<mailto:[email-masked]><mailto:[email-masked]<mailto:[email-masked]>>

---

From: Rick Macklem <[email-masked]>
Date: Sat, 7 Aug 2021 16:10:30 +0000

Rick Macklem wrote:
>>Yes, it does.  However, there will be limit,. A file of length 2**64-1 can exist and would not be extendible.  Such a file 
>>could not have a hole following the data and the spec says it "MUST" have one.
Just fyi, here's the POSIX draft's definition of a hole, plus virtual hole at EOF:
3.192 Hole
A hole is a contiguous region of bytes within a file, all having the
value of zero. Not all bytes with the value zero need belong to a hole;
however, all seekable files shall have a virtual hole starting at the
current size of the file. A hole is typically created via truncate( ),
or if an lseek( ) call has been made to position beyond the end of a
file and data subsequently written at that point, although it is up to
the implementation to define when sparse files can be created and with
what granularity for the size of holes.
--> Although I don't see a separate definition for "virtual hole", it obviously
       cannot be read like a regular hole.

Maybe the word "virtual" in front of hole would be a useful edit for RFC 7862?

rick

rick