rust - Initializing Rc<[u8]> directly without double allocation - Stack Overflow

admin管理员组

文章数量:1277380

I am trying to implement deserialization into Rc<[u8]> without double allocating. The solutions I saw always creates an intermediate Vec<u8> and filling it in, and then converting it to an Rc. But this causes double allocation, first when filling in the Vec<u8>, and second converting it into Rc<[u8]>.

Is there a way to directly allocate N bytes (N is read from the deserialization reader) into Rc<[u8]>, and using the buffer as input to the reader? I am OK to unsafe usage as long as there are no memory leaks.

I am trying to implement deserialization into Rc<[u8]> without double allocating. The solutions I saw always creates an intermediate Vec<u8> and filling it in, and then converting it to an Rc. But this causes double allocation, first when filling in the Vec<u8>, and second converting it into Rc<[u8]>.

Is there a way to directly allocate N bytes (N is read from the deserialization reader) into Rc<[u8]>, and using the buffer as input to the reader? I am OK to unsafe usage as long as there are no memory leaks.

• rust

Share Improve this question Follow edited Feb 24 at 8:24 cafce25 27.6k55 gold badges4444 silver badges5656 bronze badges asked Feb 24 at 7:54 Ahmet YazıcıAhmet Yazıcı 72466 silver badges1818 bronze badges

Add a comment  | 

2 Answers 2

Sorted by: Reset to default 9

This can be accomplished in safe code, by using Rc::from_iter with a TrustedLen iterator:

Iterators of known length

When your Iterator implements TrustedLen and is of an exact size, a single allocation will be made for the Rc<[T]>. For example:

use std::rc::Rc;

pub fn rc_slice<T: Default>(n: usize) -> Rc<[T]> {
    Rc::from_iter((0..n).map(|_| T::default()))
}

fn main() {
    let mut data = rc_slice::<u8>(16);

    // TODO: ovewrite contents of `data` with deserialized data
    *Rc::get_mut(&mut data).unwrap()[0] = ...;
}

Note: This solution relies on a zero/Default initialization of the Rc contents prior to deserialization, which works for u8. If this isn't available for your type, consider using unsafe to allocate an uninitialized Rc-slice.

Finn Bear's answer is good and provides a way to allocate exactly once, however it has a problem. TrustedLen is currently a nightly-only trait. So if you have a custom deserialisation logic, that cannot be expressed with iterators from standard library already implementing TrustedLen, then this specialisation will not work, and compiler will still use the default "collect into a Vec, and then reallocate" strategy.

However, there is another way. Since Rust 1.82.0 you can use Rc::new_uninit_slice to allocate Rc<[MaybeUnint<T>]>, then get a mutable reference to this slice with Rc::get_mut, with which you can initialise elements one by one, and finally after all elements has been initialised, you can call Rc::assume_init (which is obviously unsafe), to convert it into Rc<[T]>.

You have to be very careful with handling partially initialised data and do a proper cleanup if you encounter a panic mid-initialisation, but a simple "guard" struct with proper Drop implementation will do that for you.

I don't know how you perform deserialisation, so as an example I'll just paste code snipped from the documentation of Rc::assume_init:

let mut values = Rc::<[u32]>::new_uninit_slice(3);

// Deferred initialization:
let data = Rc::get_mut(&mut values).unwrap();
data[0].write(1);
data[1].write(2);
data[2].write(3);

// SAFETY: all values have been initialised
let values = unsafe { values.assume_init() };

assert_eq!(*values, [1, 2, 3])

本文标签： rustInitializing Rcltu8gt directly without double allocationStack Overflow