Skip to content

Add new cfg gnu_file_offset_bits64 corresponding to _FILE_OFFSET_BITS=64 #4345

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 21 commits into from
Apr 9, 2025

Conversation

snogge
Copy link
Contributor

@snogge snogge commented Mar 20, 2025

Description

Add a new cfg gnu_file_offset_bits64 corresponding to
-D_FILE_OFFSET_BITS=64.
The cfg can also be enabled by setting the environment variable
RUST_LIBC_UNSTABLE_GNU_FILE_OFFSET_BITS64=1.

This is the second part of what was #3175 .

Sources

  1. off_t
    1. https://sourceware.org/git/?p=glibc.git;a=blob;f=posix/sys/types.h;h=ab3037a9dab25fdbbe4eccb3875b2a3305c7f342;hb=HEAD#l83
  2. ino_t
    1. https://sourceware.org/git/?p=glibc.git;a=blob;f=posix/sys/types.h;h=ab3037a9dab25fdbbe4eccb3875b2a3305c7f342;hb=HEAD#l45
  3. blkcnt_t
    1. https://sourceware.org/git/?p=glibc.git;a=blob;f=posix/sys/types.h;h=ab3037a9dab25fdbbe4eccb3875b2a3305c7f342;hb=HEAD#l191
  4. fsblkcnt_t
    1. https://sourceware.org/git?p=glibc.git;a=blob;f=posix/sys/types.h;h=ab3037a9dab25fdbbe4eccb3875b2a3305c7f342;hb=HEAD#l208
  5. fsfilcnt_t
    1. https://sourceware.org/git?p=glibc.git;a=blob;f=posix/sys/types.h;h=ab3037a9dab25fdbbe4eccb3875b2a3305c7f342;hb=HEAD#l213
  6. rlim_t
    1. https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/sysv/linux/bits/resource.h;h=51c07da01933a696853e17b4c32e2c5f8e10acf5;hb=HEAD#l130
    2. https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/sysv/linux/mips/bits/resource.h;h=05f8e6ad353a471b347af2b750c30d8c160da08d;hb=HEAD#l138
    3. https://sourceware.org/git?p=glibc.git;a=blob;f=sysdeps/unix/sysv/linux/sparc/bits/resource.h;h=05f8e6ad353a471b347af2b750c30d8c160da08d;hb=HEAD#l138
  7. F_GETLK
    1. https://github.com/torvalds/linux/blob/4701f33a10702d5fc577c32434eb62adde0a1ae1/include/uapi/asm-generic/fcntl.h#L104
    2. https://github.com/torvalds/linux/blob/4701f33a10702d5fc577c32434eb62adde0a1ae1/include/uapi/asm-generic/fcntl.h#L119
    3. https://github.com/torvalds/linux/blob/4701f33a10702d5fc577c32434eb62adde0a1ae1/arch/mips/include/uapi/asm/fcntl.h#L40
    4. https://github.com/torvalds/linux/blob/4701f33a10702d5fc577c32434eb62adde0a1ae1/arch/mips/include/uapi/asm/fcntl.h#L48
  8. F_SETLK and F_SETLKW
    1. https://github.com/torvalds/linux/blob/4701f33a10702d5fc577c32434eb62adde0a1ae1/include/uapi/asm-generic/fcntl.h#L105
    2. https://github.com/torvalds/linux/blob/4701f33a10702d5fc577c32434eb62adde0a1ae1/include/uapi/asm-generic/fcntl.h#L106
    3. https://github.com/torvalds/linux/blob/4701f33a10702d5fc577c32434eb62adde0a1ae1/include/uapi/asm-generic/fcntl.h#L120
    4. https://github.com/torvalds/linux/blob/4701f33a10702d5fc577c32434eb62adde0a1ae1/include/uapi/asm-generic/fcntl.h#L121
    5. https://github.com/torvalds/linux/blob/4701f33a10702d5fc577c32434eb62adde0a1ae1/arch/mips/include/uapi/asm/fcntl.h#L41
    6. https://github.com/torvalds/linux/blob/4701f33a10702d5fc577c32434eb62adde0a1ae1/arch/mips/include/uapi/asm/fcntl.h#L42
    7. https://github.com/torvalds/linux/blob/4701f33a10702d5fc577c32434eb62adde0a1ae1/arch/mips/include/uapi/asm/fcntl.h#L49
    8. https://github.com/torvalds/linux/blob/4701f33a10702d5fc577c32434eb62adde0a1ae1/arch/mips/include/uapi/asm/fcntl.h#L50
    9. https://github.com/torvalds/linux/blob/4701f33a10702d5fc577c32434eb62adde0a1ae1/arch/sparc/include/uapi/asm/fcntl.h#L44
    10. https://github.com/torvalds/linux/blob/4701f33a10702d5fc577c32434eb62adde0a1ae1/arch/sparc/include/uapi/asm/fcntl.h#L45
  9. RLIM_INFINITY
    1. https://github.com/torvalds/linux/blob/master/include/uapi/asm-generic/resource.h#L58
  10. getrlimit, setrlimit, prlimit
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/resource/sys/resource.h#L49
    2. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/resource/sys/resource.h#L61
    3. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/sysdeps/unix/sysv/linux/bits/resource.h#L202
  11. pread, pwrite, preadv2, pwritev2
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/misc/sys/uio.h#L57
    2. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/misc/sys/uio.h#L131
  12. aio_read, aio_write, aio_error, aio_return, aio_cancel, lio_listio
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/rt/aio.h#L141
  13. fallocate, posix_fallocate, posix_fadvise, open, creat, fcntl,
    lockf, openat
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/sysdeps/unix/sysv/linux/bits/fcntl-linux.h#L427
    2. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/io/fcntl.h#L322
    3. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/io/fcntl.h#L297
    4. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/io/fcntl.h#L208
    5. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/io/fcntl.h#L254
    6. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/io/fcntl.h#L176
    7. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/io/fcntl.h#L283
    8. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/io/fcntl.h#L232
  14. glob, globfree
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/posix/glob.h#L145
  15. mkstemps, mkostemp, mkostemps, mkstemp
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/stdlib/stdlib.h#L826
    2. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/stdlib/stdlib.h#L880
    3. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/stdlib/stdlib.h#L900
    4. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/stdlib/stdlib.h#L826
  16. sendfile
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/io/sys/sendfile.h#L32
  17. statfs, fstatfs
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/io/sys/statfs.h#L30
    2. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/io/sys/statfs.h#L49
  18. fopen, freopen, tmpfile, fgetpos, fsetpos, fseeko, ftello
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/libio/stdio.h#L259
    2. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/libio/stdio.h#L193
    3. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/libio/stdio.h#L811
    4. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/libio/stdio.h#L785
  19. stat, fstat, fstatat, lstat
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/io/sys/stat.h#L203
    2. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/io/sys/stat.h#L310
  20. statfs, fstatfs
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/io/sys/statfs.h
  21. readdir, readdir_r
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/dirent/dirent.h#L163
    2. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/dirent/dirent.h#L184
  22. lseek, pread, pwrite, truncate, ftruncate
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/posix/unistd.h#L338
    2. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/posix/unistd.h#L382
    3. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/posix/unistd.h#L1025
    4. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/posix/unistd.h#L1048
  23. mmap
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/misc/sys/mman.h#L56
  24. struct statvfs, struct fstatvfs
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/io/sys/statvfs.h
  25. struct stat, struct stat64
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/sysdeps/unix/sysv/linux/bits/struct_stat.h
    2. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/sysdeps/unix/sysv/linux/mips/bits/struct_stat.h
    3. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/sysdeps/unix/sysv/linux/powerpc/bits/struct_stat.h
    4. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/sysdeps/unix/sysv/linux/sparc/bits/struct_stat.h
  26. struct statfs, struct statfs64
    1. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/sysdeps/unix/sysv/linux/bits/statfs.h
    2. https://github.com/bminor/glibc/blob/bfb2f2f481147da54237ade3266f2586a51d43c9/sysdeps/unix/sysv/linux/mips/bits/statfs.h
  27. struct fpos_t, struct fpos64_t
    1. https://github.com/bminor/glibc/blob/c5113a838b28a8894da19794ca7a69c5ace959a3/libio/bits/types/__fpos_t.h#L10
    2. https://github.com/bminor/glibc/blob/c5113a838b28a8894da19794ca7a69c5ace959a3/libio/bits/types/__fpos64_t.h#L10
  28. struct aiocb
    1. https://github.com/bminor/glibc/blob/c5113a838b28a8894da19794ca7a69c5ace959a3/rt/aio.h#L50
  29. struct flock
    1. https://github.com/bminor/glibc/blob/c5113a838b28a8894da19794ca7a69c5ace959a3/sysdeps/unix/sysv/linux/mips/bits/fcntl.h#L67

Checklist

  • Relevant tests in libc-test/semver have been updated
  • No placeholder or unstable values like *LAST or *MAX are
    included (see #3131)
  • Tested locally (cd libc-test && cargo test --target mytarget);
    especially relevant for platforms that may not be checked in CI

I've tested this as widely as possible, but I have not been able to
run the sparc tests. mips and powerpc tests have been run on rust
1.85 using the yocto project, but it's not a fully compatible
environment for some reason.

@rustbot
Copy link
Collaborator

rustbot commented Mar 20, 2025

r? @tgross35

rustbot has assigned @tgross35.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

Copy link
Contributor

@tgross35 tgross35 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't checked the type changes in detail but this looks pretty good to me. Left a handful of comments mostly related to configuration.

Thank you so much for working on this!

@tgross35 tgross35 added the stable-nominated This PR should be considered for cherry-pick to libc's stable release branch label Mar 22, 2025
@snogge snogge force-pushed the gnu-file-offset-64 branch from aba1e20 to cf9c316 Compare March 24, 2025 13:24
@snogge
Copy link
Contributor Author

snogge commented Mar 24, 2025

@rustbot ready

Copy link
Contributor

@tgross35 tgross35 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left one nit and I think it needs a rebase, but overall this looks great to me.

set_cfg("gnu_file_offset_bits64");
}
}
_ => {}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you add something like Ok(val) if val != "32" => panic!("RUST_LIBC_UNSTABLE_GNU_FILE_OFFSET_BITS may only be set to '32' or '64'),?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added that check for both build.rs and libc-test/build.rs.

@snogge snogge force-pushed the gnu-file-offset-64 branch from cf9c316 to 125f4bf Compare April 7, 2025 12:23
@snogge
Copy link
Contributor Author

snogge commented Apr 7, 2025

@rustbot ready

Copy link
Contributor

@tgross35 tgross35 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for moving this along!

@tgross35 tgross35 enabled auto-merge April 7, 2025 20:49
@tgross35
Copy link
Contributor

tgross35 commented Apr 8, 2025

...I have no idea why ctest is suddenly failing on musl. Probably not related to this PR.

snogge added 4 commits April 9, 2025 12:32
Less commands makes for a cleaner `set -x` log.  And it is more
efficient.
Variables set with `env` in the matrix never propagated into the
environment.  Add a step in test_tier1 and test_tier2 that reads the
env context from the matrix and adds the variables to the environment
used by later steps.
The `Create I artifacts` step is always run, whether earlier steps
succeeds or not.  But the upload step would only run if all preceeding
steps wer successfull.  Add a conditional to always run except if
artifact creation failed.
@tgross35 tgross35 force-pushed the gnu-file-offset-64 branch from 125f4bf to 4a7c9a9 Compare April 9, 2025 17:32
@tgross35 tgross35 added this pull request to the merge queue Apr 9, 2025
Merged via the queue into rust-lang:main with commit 1a54496 Apr 9, 2025
46 of 47 checks passed
@safinaskar
Copy link

Discussion about off_t and time_t in 32-bit archs happens here right now: https://rust-lang.zulipchat.com/#narrow/channel/219381-t-libs/topic/Questions.20and.20polls.20about.20.60libc.60.201.2E0

tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
Less commands makes for a cleaner `set -x` log.  And it is more
efficient.

(backport <rust-lang#4345>)
(cherry picked from commit afa5c65)
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
Variables set with `env` in the matrix never propagated into the
environment.  Add a step in test_tier1 and test_tier2 that reads the
env context from the matrix and adds the variables to the environment
used by later steps.

(backport <rust-lang#4345>)
(cherry picked from commit 84a04a1)
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
The `Create I artifacts` step is always run, whether earlier steps
succeeds or not.  But the upload step would only run if all preceeding
steps wer successfull.  Add a conditional to always run except if
artifact creation failed.

(backport <rust-lang#4345>)
(cherry picked from commit f10e8e4)
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
Add new jobs for i686 in test_tier1 and arm and powerpc in test_tier2
where RUST_LIBC_UNSTABLE_GNU_FILE_OFFSET_BITS=64.

Use artifact-tag to avoid artifact name collisions.

(backport <rust-lang#4345>)
(cherry picked from commit 874e399)
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
Set the basic types correctly for
gnu_file_offset_bits64 (_FILE_OFFSET_BITS=64).

(backport <rust-lang#4345>)
(cherry picked from commit f44fdc1)
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
gnu_file_offset_bits64 means _FILE_OFFSET_BITS=64.

(backport <rust-lang#4345>)
(cherry picked from commit 862ba8a)
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
When _FILE_OFFSET_BITS=64, glibc redirects some function calls to 64
bit versions.  These symbols are sometimes the public LFS variants,
sometimes hidden variants.

(backport <rust-lang#4345>)
(cherry picked from commit 2b4fafb)
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
Like mips, the stat struct will become different once support for
gnu_file_offset_bits64 is added.

(backport <rust-lang#4345>)
(cherry picked from commit 0c6d56c)
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
Like mips and powerpc, the stat struct will become different once
support for gnu_file_offset_bits64 is added.

(backport <rust-lang#4345>)
(cherry picked from commit e134959)
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
Change the __padX members in b32/mod.rs from short to uint even though
they are actually unsigned short in C.  Using unsigned int will give
the same alignment, and make the struct equivalent to stat64 when
gnu_file_offset_bits64 is set.

(backport <rust-lang#4345>)
(cherry picked from commit 5a5abc2)
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
Struct stat and stat64 needs to match when gnu_file_offset_bits64 is
set.

(backport <rust-lang#4345>)
(cherry picked from commit 22ac02c)
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
The __f_unused field should be the same in statvfs and
statvfs64 (where it was already included) as can be seen in
https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/sysv/linux/bits/statvfs.h;h=1aed2f54aa86e43ac1c1d3a33197b3232be76580;hb=HEAD

(backport <rust-lang#4345>)
(cherry picked from commit 131efe9)
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
tgross35 pushed a commit to tgross35/rust-libc that referenced this pull request Apr 11, 2025
@tgross35 tgross35 mentioned this pull request Apr 11, 2025
@tgross35 tgross35 added stable-applied This PR has been cherry-picked to libc's stable release branch and removed stable-nominated This PR should be considered for cherry-pick to libc's stable release branch labels Apr 11, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-CI Area: CI-related items O-arm O-gnu O-linux O-linux-like O-mips O-sparc O-unix O-x86 S-waiting-on-review stable-applied This PR has been cherry-picked to libc's stable release branch
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants