Skip to content

Vec::recycle (experimental API implementation) #66069

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 7 commits into from
Closed
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions src/liballoc/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -123,6 +123,7 @@
#![feature(alloc_layout_extra)]
#![feature(try_trait)]
#![feature(associated_type_bounds)]
#![feature(recycle_vec)]

// Allow testing this library

Expand Down
1 change: 1 addition & 0 deletions src/liballoc/tests/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
#![feature(associated_type_bounds)]
#![feature(binary_heap_into_iter_sorted)]
#![feature(binary_heap_drain_sorted)]
#![feature(recycle_vec)]

use std::hash::{Hash, Hasher};
use std::collections::hash_map::DefaultHasher;
Expand Down
70 changes: 70 additions & 0 deletions src/liballoc/tests/vec.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1264,6 +1264,76 @@ fn test_try_reserve_exact() {

}

/// Tests that `recycle` successfully re-interprets the type to have different lifetime from the original
#[test]
fn test_recycle_lifetime() {
let s_1 = "foo".to_string();
let mut buf = Vec::with_capacity(100);
{
let mut buf2 = buf;
let s_2 = "foo".to_string();
buf2.push(s_2.as_str());

assert_eq!(buf2.len(), 1);
assert_eq!(buf2.capacity(), 100);

buf = buf2.recycle();
}
buf.push(s_1.as_str());
}

/// Tests that `recycle` successfully re-interprets the type itself
#[test]
fn test_recycle_type() {
let s = "foo".to_string();
let mut buf = Vec::with_capacity(100);
{
let mut buf2 = buf.recycle();

let mut i = Vec::new();
i.push(1);
i.push(2);
i.push(3);

buf2.push(i.as_slice());

assert_eq!(buf2.len(), 1);
assert_eq!(buf2.capacity(), 100);

buf = buf2.recycle();
}
buf.push(s.as_str());
}

/// Tests that `recycle` successfully panics with incompatible sizes
#[test]
#[should_panic]
fn test_recycle_incompatible_size() {
let mut buf = Vec::with_capacity(100);
buf.push(1_u16);
{
let mut buf2 = buf.recycle();
buf2.push(1_u32);
buf = buf2.recycle();
}
buf.push(1_u16);
}


/// Tests that `recycle` successfully panics with incompatible alignments
#[test]
#[should_panic]
fn test_recycle_incompatible_alignment() {
let mut buf = Vec::with_capacity(100);
buf.push([0_u16, 1_u16]);
{
let mut buf2 = buf.recycle();
buf2.push(1_u32);
buf = buf2.recycle();
}
buf.push([0_u16, 1_u16]);
}

#[test]
fn test_stable_push_pop() {
// Test that, if we reserved enough space, adding and removing elements does not
Expand Down
86 changes: 86 additions & 0 deletions src/liballoc/vec.rs
Original file line number Diff line number Diff line change
Expand Up @@ -603,6 +603,92 @@ impl<T> Vec<T> {
self.buf.try_reserve_exact(self.len, additional)
}

/// Allows re-interpreting the type of a Vec to reuse the allocation.
/// The vector is emptied and any values contained in it will be dropped.
/// The target type must have the same size and alignment as the source type.
/// This API doesn't transmute any values of T to U, because it makes sure
/// to empty the vector before any unsafe operations.
///
/// # Example
///
/// This API is useful especially when using `Vec` as a
/// temporary storage for data with short lifetimes.
/// By recycling the allocation, the `Vec` is able to safely
/// outlive the lifetime of the type that was stored in it.
/// ```
/// # #![feature(recycle_vec)]
/// # use std::error::Error;
/// #
/// # struct Stream(bool);
/// #
/// # impl Stream {
/// # fn new() -> Self {
/// # Stream(false)
/// # }
/// #
/// # fn next(&mut self) -> Option<&[u8]> {
/// # if self.0 {
/// # None
/// # } else {
/// # self.0 = true;
/// # Some(&b"foo"[..])
/// # }
/// # }
/// # }
/// #
/// # fn process(input: &[Object<'_>]) -> Result<(), Box<dyn Error>> {
/// # for obj in input {
/// # let _ = obj.reference;
/// # }
/// # Ok(())
/// # }
/// #
/// # struct Object<'a> {
/// # reference: &'a [u8],
/// # }
/// #
/// # fn deserialize<'a>(input: &'a [u8], output: &mut Vec<Object<'a>>) -> Result<(), Box<dyn Error>> {
/// # output.push(Object { reference: input });
/// # Ok(())
/// # }
/// #
/// # fn main() -> Result<(), Box<dyn Error>> {
/// # let mut stream = Stream::new();
/// let mut objects: Vec<Object<'static>> = Vec::new(); // Any lifetime goes here
///
/// while let Some(byte_chunk) = stream.next() { // `byte_chunk` lifetime starts
/// let mut temp: Vec<Object<'_>> = objects.recycle(); // `temp` lifetime starts
///
/// // Zero-copy parsing; deserialized `Object`s have references to `byte_chunk`
/// deserialize(byte_chunk, &mut temp)?;
/// process(&temp)?;
///
/// objects = temp.recycle(); // `temp` lifetime ends
/// } // `byte_chunk` lifetime ends
/// # Ok(())
/// # }
/// ```
///
/// # Panics
/// Panics if the size or alignment of the source and target types don't match.
///
/// # Note about stabilization
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should be removed now?

/// The size and alignment contract is enforceable at compile-time,
/// so we will want to wait until compile-time asserts become stable and
/// modify this API to cause a compile error instead of panicking
/// before stabilizing it.
#[unstable(feature = "recycle_vec", reason = "new API", issue = "0")]
pub fn recycle<U>(mut self) -> Vec<U> {
self.truncate(0);
// TODO make these const asserts once it becomes possible
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there an actual plan for making this possible? Even with compile time assertions, you wouldn't actually be able to reference T and U inside e.g. a const item defined in this body.

Copy link
Contributor Author

@golddranks golddranks Nov 4, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is a question that needs to be resolved. It is mentioned as an unresolved question in the RFC. The problem is that changing to compile-time error is a breaking change, so whatever gets decided, we must commit to before stabilizing.

I assumed that once the const functions support conditionals, creating compile time assertion for this case would be possible, but I had a hidden assumption that if there is a generic const function, I could pass it generic types that are in scope. Is the latter not the case? I wonder if there are explicit plans about supporting or not supporting that, or is it just more about how it currently is?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's not the case and there are no current plans to change that so don't expect it to change within the next 1.5 years at least.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All right, I removed the FIXME. I guess it seems good enough just to panic, although I'd love to have a compile-time guarantee, and I think I'm not alone. I wonder if there is any other avenue of attack that would accomplish this.

assert_eq!(core::mem::size_of::<T>(), core::mem::size_of::<U>());
assert_eq!(core::mem::align_of::<T>(), core::mem::align_of::<U>());
let capacity = self.capacity();
let ptr = self.as_mut_ptr() as *mut U;
core::mem::forget(self);
unsafe { Vec::from_raw_parts(ptr, 0, capacity) }
}

/// Shrinks the capacity of the vector as much as possible.
///
/// It will drop down as close as possible to the length but the allocator
Expand Down