buffered io and deep copy

NTDEV
1

Hello,

I have a question, which arised entirely from theoretical interest.
Something which I didn’t know (and I have not done either) was the fact
that the buffer type of IO when utilizng IOCTL cannot actually perform a
deep copy of a structure. Does that mean that even if you use buffered
i/o between um<=>km you still have to utilize the
ProbeForRead/ProbeForWrite etc. apis when you are likely to manipulate
values which are stored by reference in the passed structure, because,
in essence, you just get a pointer to this value (in the copied
structure) but the actual backing memory might go in any moment if UM
was to fiddle with it? What are some rules of thumb when you are present
with such a situation?

Regards

2

It means that you shouldn’t pass embedded pointers like this with buffered
i/o - they aren’t treated specially in any way.

What are trying to do here? Is there any way you could ‘flatten’ the i/o
packet to remove the pointers?

mm

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of lorddoskias
Sent: Sunday, May 06, 2012 4:51 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] buffered io and deep copy

Hello,

I have a question, which arised entirely from theoretical interest.
Something which I didn’t know (and I have not done either) was the fact that
the buffer type of IO when utilizng IOCTL cannot actually perform a deep
copy of a structure. Does that mean that even if you use buffered i/o
between um<=>km you still have to utilize the ProbeForRead/ProbeForWrite
etc. apis when you are likely to manipulate values which are stored by
reference in the passed structure, because, in essence, you just get a
pointer to this value (in the copied
structure) but the actual backing memory might go in any moment if UM was to
fiddle with it? What are some rules of thumb when you are present with such
a situation?

Regards

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

3

What made you think a deep copy occurred? How do you think that would work given the API call takes a pvoid?

d

debt from my phone

From: Martin O’Brien
Sent: 5/6/2012 4:55 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] buffered io and deep copy

It means that you shouldn’t pass embedded pointers like this with buffered
i/o - they aren’t treated specially in any way.

What are trying to do here? Is there any way you could ‘flatten’ the i/o
packet to remove the pointers?

mm

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of lorddoskias
Sent: Sunday, May 06, 2012 4:51 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] buffered io and deep copy

Hello,

I have a question, which arised entirely from theoretical interest.
Something which I didn’t know (and I have not done either) was the fact that
the buffer type of IO when utilizng IOCTL cannot actually perform a deep
copy of a structure. Does that mean that even if you use buffered i/o
between um<=>km you still have to utilize the ProbeForRead/ProbeForWrite
etc. apis when you are likely to manipulate values which are stored by
reference in the passed structure, because, in essence, you just get a
pointer to this value (in the copied
structure) but the actual backing memory might go in any moment if UM was to
fiddle with it? What are some rules of thumb when you are present with such
a situation?

Regards

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

4

lorddoskias wrote:

I have a question, which arised entirely from theoretical interest.
Something which I didn’t know (and I have not done either) was the fact
that the buffer type of IO when utilizng IOCTL cannot actually perform a
deep copy of a structure.

Of course. There is no possible way I/O manager could do a deep copy.
It copies the bytes – however many bytes were specified in the I/O
call. I/O manager doesn’t know what those bytes represent. It doesn’t
know which bytes are addresses, and even if it could determine that
something was an address, it couldn’t know how many bytes to copy.

Does that mean that even if you use buffered
i/o between um<=>km you still have to utilize the
ProbeForRead/ProbeForWrite etc. apis when you are likely to manipulate
values which are stored by reference in the passed structure, because,
in essence, you just get a pointer to this value (in the copied
structure) but the actual backing memory might go in any moment if UM
was to fiddle with it?

Absolutely, yes. This is why it is bad practice to pass pointers inside
a buffer like that. (The Kernel Streaming ioctls do this, but they are
already marked as METHOD_NEITHER, which is an alias for “here be dragons”.)

What are some rules of thumb when you are present with such a situation?

  1. Don’t do it.

Many such abominations can be cleaned up by using METHOD_IN_DIRECT and
METHOD_OUT_DIRECT instead. Pass the meta information in the first
buffer, and pass the pointer as the second buffer. Then, I/O manager
does the locking for you.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

5

To add one thing to what Tim said, if you have an interface you need
with embedded pointers, then create a wrapper function to package the
call. Typically the approach is to copy everything into one buffer, or
use both buffers with METHOD_IN_DIRECT if there is no output. You can
do this as simply as copying the original structure and making the
pointers offsets into the buffer, and then copy the data the pointers
referred to, to the specified offset.

Don Burn
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

“Tim Roberts” wrote in message news:xxxxx@ntdev:

> lorddoskias wrote:
> > I have a question, which arised entirely from theoretical interest.
> > Something which I didn’t know (and I have not done either) was the fact
> > that the buffer type of IO when utilizng IOCTL cannot actually perform a
> > deep copy of a structure.
>
> Of course. There is no possible way I/O manager could do a deep copy.
> It copies the bytes – however many bytes were specified in the I/O
> call. I/O manager doesn’t know what those bytes represent. It doesn’t
> know which bytes are addresses, and even if it could determine that
> something was an address, it couldn’t know how many bytes to copy.
>
>
> > Does that mean that even if you use buffered
> > i/o between um<=>km you still have to utilize the
> > ProbeForRead/ProbeForWrite etc. apis when you are likely to manipulate
> > values which are stored by reference in the passed structure, because,
> > in essence, you just get a pointer to this value (in the copied
> > structure) but the actual backing memory might go in any moment if UM
> > was to fiddle with it?
>
> Absolutely, yes. This is why it is bad practice to pass pointers inside
> a buffer like that. (The Kernel Streaming ioctls do this, but they are
> already marked as METHOD_NEITHER, which is an alias for “here be dragons”.)
>
>
> > What are some rules of thumb when you are present with such a situation?
>
> 1. Don’t do it.
>
> Many such abominations can be cleaned up by using METHOD_IN_DIRECT and
> METHOD_OUT_DIRECT instead. Pass the meta information in the first
> buffer, and pass the pointer as the second buffer. Then, I/O manager
> does the locking for you.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.

6

Once again, I want a “like” button for Tim’s response.

Jake Oshins
Windows Kernel Team

This message offers no warranties and confers no rights.

“Tim Roberts” wrote in message news:xxxxx@ntdev…

lorddoskias wrote:

I have a question, which arised entirely from theoretical interest.
Something which I didn’t know (and I have not done either) was the fact
that the buffer type of IO when utilizng IOCTL cannot actually perform a
deep copy of a structure.

Of course. There is no possible way I/O manager could do a deep copy.
It copies the bytes – however many bytes were specified in the I/O
call. I/O manager doesn’t know what those bytes represent. It doesn’t
know which bytes are addresses, and even if it could determine that
something was an address, it couldn’t know how many bytes to copy.

Does that mean that even if you use buffered
i/o between um<=>km you still have to utilize the
ProbeForRead/ProbeForWrite etc. apis when you are likely to manipulate
values which are stored by reference in the passed structure, because,
in essence, you just get a pointer to this value (in the copied
structure) but the actual backing memory might go in any moment if UM
was to fiddle with it?

Absolutely, yes. This is why it is bad practice to pass pointers inside
a buffer like that. (The Kernel Streaming ioctls do this, but they are
already marked as METHOD_NEITHER, which is an alias for “here be dragons”.)

What are some rules of thumb when you are present with such a situation?

  1. Don’t do it.

Many such abominations can be cleaned up by using METHOD_IN_DIRECT and
METHOD_OUT_DIRECT instead. Pass the meta information in the first
buffer, and pass the pointer as the second buffer. Then, I/O manager
does the locking for you.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

7

> deep copy of a structure. Does that mean that even if you use buffered

i/o between um<=>km you still have to utilize the
ProbeForRead/ProbeForWrite etc. apis when you are likely to manipulate
values which are stored by reference in the passed structure, because,
in essence, you just get a pointer to this value

Yes.

And the rule of thumb is to avoid such things with IOCTLs.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com