Description
While updating my code base from 1.49 to 1.50 we noticed a failures in a program that rebuilds files from small chunks.
We are opening a file in append mode, one in read mode, and we stream one into the other (as for the snippet below).
Testing this out confirmed that the error happens on the reader file when the writer is in append mode, whenever we use a BufWriter/BufReader or not. If we load the whole file into a string and then use std::io::copy on the same data it works as intended, as does setting the file in write mode (while it becomes necessary to seek to the end).
Checking what got shipped into 1.50 we suspect the probable culprit to be 028754a#diff-4280ab12d5278289ca8b2e83cad374850eaeac0c18f49a474f5a9b5bf55d3991
What we are guessing is that this change is using the sendfile syscall that doesn't support files in append mode even when the file is in append mode, triggering the Bad File Descriptor error
Code
I tried this code (playground):
use std::fs::{File, OpenOptions};
use std::io::prelude::*;
fn main() -> std::io::Result<()> {
write_file("first.txt")?;
write_file("second.txt")?;
println!("{}", read_file("first.txt")?);
copy_file("first.txt", "second.txt")?;
Ok(())
}
fn write_file(name: &str) -> std::io::Result<()> {
let mut file = File::create(name)?;
file.write_all(b"Hello, world!")?;
Ok(())
}
fn read_file(name: &str) -> std::io::Result<String> {
let mut file = File::open(name)?;
let mut contents = String::new();
file.read_to_string(&mut contents)?;
Ok(contents)
}
fn copy_file(from: &str, to: &str) -> std::io::Result<()> {
let mut source = OpenOptions::new().read(true).open(from)?;
let mut dest = OpenOptions::new().append(true).open(to)?;
let bytes_written = std::io::copy(&mut source, &mut dest)?;
println!("Copied {} bytes", bytes_written);
std::fs::remove_file(from)?;
Ok(())
}I expected to see this happen: the files to be concatenated
Instead, this happened: std::io::copy returns Error: Os { code: 9, kind: Other, message: "Bad file descriptor" }
Strace for the code snipped:
open("first.txt", O_RDONLY|O_CLOEXEC) = 3
open("second.txt", O_WRONLY|O_APPEND|O_CLOEXEC) = 4
futex(0x7f376caad0ec, FUTEX_WAKE_PRIVATE, 2147483647) = 0
syscall_332(0, 0, 0, 0xfff, 0, 0) = -1 (errno 38)
fstat(3, {st_mode=S_IFREG|0644, st_size=13, ...}) = 0
syscall_326(0xffffffff, 0, 0xffffffff, 0, 0x1, 0) = -1 (errno 9)
syscall_326(0x3, 0, 0x4, 0, 0x40000000, 0) = -1 (errno 9)
close(4) = 0
close(3) = 0
Example that works with loading the file into a string before copy (playground):
use std::fs::{File, OpenOptions};
use std::io::BufReader;
use std::io::BufWriter;
use std::io::Read;
use std::io::prelude::*;
fn main() -> std::io::Result<()> {
write_file("first.txt")?;
write_file("second.txt")?;
println!("{}", read_file("first.txt")?);
copy_file("first.txt", "second.txt")?;
println!("destination data {}", read_file("second.txt")?);
Ok(())
}
fn write_file(name: &str) -> std::io::Result<()> {
let mut file = File::create(name)?;
file.write_all(b"Hello, world!")?;
Ok(())
}
fn read_file(name: &str) -> std::io::Result<String> {
let mut file = File::open(name)?;
let mut contents = String::new();
file.read_to_string(&mut contents)?;
Ok(contents)
}
fn copy_file(from: &str, to: &str) -> std::io::Result<()> {
let mut source = BufReader::new(OpenOptions::new().read(true).open(from)?);
let mut source_str = String::new();
source.read_to_string(&mut source_str)?;
let mut reader = BufReader::new(source_str.as_bytes());
let mut dest = BufWriter::new(OpenOptions::new().append(true).open(to)?);
let bytes_written = std::io::copy(&mut reader, &mut dest)?;
println!("Copied {} bytes", bytes_written);
std::fs::remove_file(from)?;
Ok(())
}Version it worked on
It most recently worked on: 1.49
Version with regression
1.50
rustc --version --verbose:
rustc 1.50.0 (cb75ad5db 2021-02-10)
binary: rustc
commit-hash: cb75ad5db02783e8b0222fee363c5f63f7e2cf5b
commit-date: 2021-02-10
host: x86_64-unknown-linux-gnu
release: 1.50.0
Backtrace
no backtrace