[rust-dev] [ANN] Iobuf

Discussion:

Clark Gaebel

2014-09-04 05:17:47 UTC

Hey everyone!

Have you ever needed to communicate with the outside world from a rust application? Do you need to send data through a network interface, or touch a disk? Then you need Iobufs!

An Iobuf is a nifty abstraction over an array of bytes, which makes writing things like highly efficient zero-copy speculative network protocol parsers easy! Any time I need to do I/O, I reach for an Iobuf to do the heavy lifting.

? ? ? ? ? ? https://github.com/cgaebel/iobuf

Enjoy,
? - Clark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/rust-dev/attachments/20140903/7ea756b8/attachment.html>

Vladimir Matveev

2014-09-04 07:46:06 UTC

Permalink

Hi!

I?ve noticed this piece of code in your library:

#[inline]
fn as_mut_slice(&self) -> &mut [u8] {
unsafe {
match self {
&OwnedBuffer(ref v) => {
let mut_v: &mut Vec<u8> = mem::transmute(v);
mut_v.as_mut_slice()
},
&BorrowedBuffer(ref s) => {
let mut_s: &mut &mut [u8] = mem::transmute(s);
mut_s.as_mut_slice()
},
}
}
}

I was under impression that transmuting & to &mut is undefined behavior in Rust, and you need to use RefCell (or UnsafeCell) for this. Am I wrong?

Post by Clark Gaebel
Hey everyone!
Have you ever needed to communicate with the outside world from a rust application? Do you need to send data through a network interface, or touch a disk? Then you need Iobufs!
An Iobuf is a nifty abstraction over an array of bytes, which makes writing things like highly efficient zero-copy speculative network protocol parsers easy! Any time I need to do I/O, I reach for an Iobuf to do the heavy lifting.
https://github.com/cgaebel/iobuf
Enjoy,
- Clark
_______________________________________________
Rust-dev mailing list
Rust-dev at mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

Clark Gaebel

2014-09-04 08:15:24 UTC

Permalink

I think you?re right! Thanks for pointing at UnsafeCell. That seems like exactly what I want. It has been fixed.

Thanks a ton for the catch!

? - Clark

On Thu, Sep 4, 2014 at 12:46 AM, Vladimir Matveev <dpx.infinity at gmail.com>

Post by Vladimir Matveev
Hi!
#[inline]
fn as_mut_slice(&self) -> &mut [u8] {
unsafe {
match self {
&OwnedBuffer(ref v) => {
let mut_v: &mut Vec<u8> = mem::transmute(v);
mut_v.as_mut_slice()
},
&BorrowedBuffer(ref s) => {
let mut_s: &mut &mut [u8] = mem::transmute(s);
mut_s.as_mut_slice()
},
}
}
}
I was under impression that transmuting & to &mut is undefined behavior in Rust, and you need to use RefCell (or UnsafeCell) for this. Am I wrong?

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/rust-dev/attachments/20140904/74c4fc91/attachment.html>

Kevin Ballard

2014-09-05 03:36:38 UTC

Permalink

I?m still seeing bad transmutes.

fn from_str<'a>(s: &'a str) -> RawIobuf<'a> {
unsafe {
let bytes: &mut [u8] = mem::transmute(s.as_bytes());
RawIobuf::of_buf(BorrowedBuffer(bytes))
}
}

This is taking a `&str`, converting to `&[u8]`, and then transmuting to `&mut [u8]`. Besides being undefined, I have to assume it's also possible for other code later on to end up attempting to actually mutate this data, which will either a) be really bad, or b) not even be possible if it's a string constant in read-only member.

-Kevin

Post by Clark Gaebel
I think you?re right! Thanks for pointing at UnsafeCell. That seems like exactly what I want. It has been fixed.
Thanks a ton for the catch!
- Clark
Hi!
#[inline]
fn as_mut_slice(&self) -> &mut [u8] {
unsafe {
match self {
&OwnedBuffer(ref v) => {
let mut_v: &mut Vec<u8> = mem::transmute(v);
mut_v.as_mut_slice()
},
&BorrowedBuffer(ref s) => {
let mut_s: &mut &mut [u8] = mem::transmute(s);
mut_s.as_mut_slice()
},
}
}
}
I was under impression that transmuting & to &mut is undefined behavior in Rust, and you need to use RefCell (or UnsafeCell) for this. Am I wrong?

_______________________________________________
Rust-dev mailing list
Rust-dev at mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4118 bytes
Desc: not available
URL: <http://mail.mozilla.org/pipermail/rust-dev/attachments/20140904/197df465/attachment.p7s>

Clark Gaebel

2014-09-05 06:55:28 UTC

Permalink

from_str is only used to construct read-only Iobufs, which do not support writing into the buffer. Also, if you modify the string, it should invalidate the Iobuf since the lifetime is bound to the borrowed reference of the string.

? - Clark

Post by Kevin Ballard
I?m still seeing bad transmutes.
fn from_str<'a>(s: &'a str) -> RawIobuf<'a> {
unsafe {
let bytes: &mut [u8] = mem::transmute(s.as_bytes());
RawIobuf::of_buf(BorrowedBuffer(bytes))
}
}
This is taking a `&str`, converting to `&[u8]`, and then transmuting to `&mut [u8]`. Besides being undefined, I have to assume it's also possible for other code later on to end up attempting to actually mutate this data, which will either a) be really bad, or b) not even be possible if it's a string constant in read-only member.
-Kevin

_______________________________________________
Rust-dev mailing list
Rust-dev at mozilla.org
https://mail.mozilla.org/listinfo/rust-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/rust-dev/attachments/20140904/05348711/attachment.html>

Kevin Ballard

2014-09-05 09:55:27 UTC

Permalink

It's still undefined behavior to construct the &mut reference like that, even if it's never actually mutated. I think you need to ditch &mut and start using *mut instead.

-Kevin

Post by Clark Gaebel
from_str is only used to construct read-only Iobufs, which do not support writing into the buffer. Also, if you modify the string, it should invalidate the Iobuf since the lifetime is bound to the borrowed reference of the string.
- Clark

<smime.p7s>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/rust-dev/attachments/20140905/ea445726/attachment.html>

Clark Gaebel

2014-09-05 15:49:39 UTC

Permalink

It just seems weird to me that transmuting to a raw::Slice<u8>, and then casting the data ptr to a *mut u8 when I need it is safer than just using the &?a mut [u8] directly.

But alas, I?ll fix it.

? - Clark

Post by Kevin Ballard
It's still undefined behavior to construct the &mut reference like that, even if it's never actually mutated. I think you need to ditch &mut and start using *mut instead.
-Kevin

<smime.p7s>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.mozilla.org/pipermail/rust-dev/attachments/20140905/6d63f550/attachment.html>