Skip to content

io::copy does not use copy_file_range consistently #114341

Closed
@xstaticxgpx

Description

@xstaticxgpx

I tried this code:

test.rs:

use std::fs::File;
use std::io;
use std::os::fd::FromRawFd;

fn main() -> io::Result<()> {
    let file = "./test";
    let mut stdout = unsafe { File::from_raw_fd(1) };
    for _ in 0..3 {
        let mut _file = File::open(&file)?;
        let _ = io::copy(&mut _file, &mut stdout);
    }
    Ok(())
}

Testing:

rustc test.rs
./test >test.out # Reads itself and redirects stdout to regular file (now truncated)
strace ./test >test.out # See sendfile is used first before `copy_file_range_candidate` is hit
...
sendfile(1, 3, NULL, 2147479552)        = 10065520
sendfile(1, 3, NULL, 2147479552)        = 0       
...
copy_file_range(-1, NULL, -1, NULL, 1, 0) = -1 EBADF (Bad file descriptor)                                                                                                                                                                                                                                                   
copy_file_range(3, NULL, 1, NULL, 1073741824, 0) = 10065520                                                                                                                                                                                                                                                                  
copy_file_range(3, NULL, 1, NULL, 1073741824, 0) = 0                                                                                                                                                                                                                                                                         
...
copy_file_range(3, NULL, 1, NULL, 1073741824, 0) = 10065520
copy_file_range(3, NULL, 1, NULL, 1073741824, 0) = 0
...

I expected to see this happen:

io::copy initially detects that copy_file_range is available for the copy from regular file to regular file (truncated by the shell redirection)

Instead, this happened:

io::copy initially uses sendfile on the first iteration and then determines copy_file_range is a candidate
ie. when stdout redirected to a regular file which is now meta.len() > 0

The meta.len() > 0 logic does not make sense for outputs AFAICT - only inputs (ie. /proc filesystems, as documented)

FdMeta::Metadata(meta) if meta.is_file() && meta.len() > 0 => true,

copy_file_range can populate the initially empty (regular) output file without issue, that's how coreutils cat works today.

Meta

rustc --version --verbose:

rustc 1.71.0 (8ede3aae2 2023-07-12) (Arch Linux rust 1:1.71.0-1)
binary: rustc
commit-hash: 8ede3aae28fe6e4d52b38157d7bfe0d3bceef225
commit-date: 2023-07-12
host: x86_64-unknown-linux-gnu
release: 1.71.0
LLVM version: 15.0.7

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-ioArea: `std::io`, `std::fs`, `std::net` and `std::path`C-enhancementCategory: An issue proposing an enhancement or a PR with one.O-linuxOperating system: LinuxT-libsRelevant to the library team, which will review and decide on the PR/issue.

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions