Passing variable-sized array from Kernel to User

Hi all,

I know this is a very simple question ( I apologize in advanced ) but
I am just curious what the *accepted* approach is for passing a
variable sized array from the kernel to user-space.

I am familiar with sending data from user to kernel, but the other way
seems a bit more complicated.

Should I create two ioctls i.e. one for getting the size, and then
another for sending it down to the driver? Or, perhaps create a MAX
static sized buffer and have the driver populate that?

Thanks,
J

The problem with the two ioctl approach (get size, then get buffer) is the window between the two. The size can change in which case you have to indicate “there is more” somehow. With the get buffer alone, you have the same issue of how to indicate there is more. So, since they both devolve to the same problem, just have a single get buffer ioctl. The driver can choose to fill in as many slots in the array as it has data

d

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of JonathonS
Sent: Monday, September 17, 2012 5:30 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Passing variable-sized array from Kernel to User

Hi all,

I know this is a very simple question ( I apologize in advanced ) but I am just curious what the *accepted* approach is for passing a variable sized array from the kernel to user-space.

I am familiar with sending data from user to kernel, but the other way seems a bit more complicated.

Should I create two ioctls i.e. one for getting the size, and then another for sending it down to the driver? Or, perhaps create a MAX static sized buffer and have the driver populate that?

Thanks,
J


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

There are several possible approaches, but there’s a bit of a glitch from
drivers. The problem I found was that if I returned an error code, like
buffer-too-small, but had filled in as much of the buffer as I had, in
buffered mode the buffer was not copied back, and in direct mode it
returned 0 for bytes-transferred

So the one way I know that will work is the following:

BOOL ok;
SIZE_T size;

size = some_initial_value_of_your_choice;
WHATEVER * buffer = new WHATEVER[size];

while(TRUE)
{ /* look */
ok = DeviceIoControl( …size… &bytesTrannsferred …);
if(!ok)
{ /* failed */
DWORD err = ::GetLastError();
if(err == ERROR_WHATEVER) // buffer too small
{ /* buffer too small */
size * = 2;
delete buffer;
buffer = new WHATEVER[size];
continue;
} /* buffer too small */
break; // exit loop
} /* failed */
else
{ /* no error */
if(bytesTransferred == size)
{ /* exact match */
size *= 2;
delete buffer;
buffer = new WHATEVER[size];
continue;
} /* exact match */
} /* no error */
break;
} /* loop */
if(!ok)
{ /* solid error */
…deal with it
} /* solid error */

That is, if you fill the buffer exactly, you might have lost data, so the
right thing to do is to try again with a larger buffer, and continue until
you get an only-partially-filled buffer. There are several user-level
APIs for which this is the only possible solution.

Note the code above can be refactored to avoid duplication, but I’m typing
this in without a decent program editor, so I didn’t bother to reduce it,
as I would have done had I been coding it in an actual deliverable
product. Also, in writing C++, I would more likely use a collection like
CArray or std::vector, which is why I’m a bit sketchy
about details like how to delete and reallocate the buffer I would not
use delete and new as shown, but something more complex but more robust.
joe

> The problem with the two ioctl approach (get size, then get buffer) is the
> window between the two. The size can change in which case you have to
> indicate “there is more” somehow. With the get buffer alone, you have the
> same issue of how to indicate there is more. So, since they both devolve
> to the same problem, just have a single get buffer ioctl. The driver can
> choose to fill in as many slots in the array as it has data
>
> d
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of JonathonS
> Sent: Monday, September 17, 2012 5:30 PM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] Passing variable-sized array from Kernel to User
>
> Hi all,
>
> I know this is a very simple question ( I apologize in advanced ) but I am
> just curious what the accepted approach is for passing a variable sized
> array from the kernel to user-space.
>
> I am familiar with sending data from user to kernel, but the other way
> seems a bit more complicated.
>
> Should I create two ioctls i.e. one for getting the size, and then another
> for sending it down to the driver? Or, perhaps create a MAX static sized
> buffer and have the driver populate that?
>
> Thanks,
> J
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

You have to return a value that is a warning, not a failure, to copy back data into the output buffer

C:\Program Files (x86)\Windows Kits\8.0\Include\shared>findstr STATUS_BUFFER_TOO_SMALL ntstatus.h
// MessageId: STATUS_BUFFER_TOO_SMALL
#define STATUS_BUFFER_TOO_SMALL ((NTSTATUS)0xC0000023L) <– error

C:\Program Files (x86)\Windows Kits\8.0\Include\shared>findstr STATUS_BUFFER_OVERFLOW ntstatus.h
// MessageId: STATUS_BUFFER_OVERFLOW
#define STATUS_BUFFER_OVERFLOW ((NTSTATUS)0x80000005L) <– warning

From ntstatus.h, explaining the top two bits

// Values are 32 bit values laid out as follows:
//
// 3 3 2 2 2 2 2 2 2 2 2 2 1 1 1 1 1 1 1 1 1 1
// 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0 9 8 7 6 5 4 3 2 1 0
// ±–±±±----------------------±------------------------------+
// |Sev|C|R| Facility | Code |
// ±–±±±----------------------±------------------------------+
//
// where
//
// Sev - is the severity code
//
// 00 - Success
// 01 - Informational
// 10 - Warning
// 11 - Error

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@flounder.com
Sent: Monday, September 17, 2012 7:56 PM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] Passing variable-sized array from Kernel to User

There are several possible approaches, but there’s a bit of a glitch from drivers. The problem I found was that if I returned an error code, like buffer-too-small, but had filled in as much of the buffer as I had, in buffered mode the buffer was not copied back, and in direct mode it returned 0 for bytes-transferred

So the one way I know that will work is the following:

BOOL ok;
SIZE_T size;

size = some_initial_value_of_your_choice;
WHATEVER * buffer = new WHATEVER[size];

while(TRUE)
{ /* look */
ok = DeviceIoControl( …size… &bytesTrannsferred …);
if(!ok)
{ /* failed */
DWORD err = ::GetLastError();
if(err == ERROR_WHATEVER) // buffer too small
{ /* buffer too small */
size * = 2;
delete buffer;
buffer = new WHATEVER[size];
continue;
} /* buffer too small */
break; // exit loop
} /* failed */
else
{ /* no error */
if(bytesTransferred == size)
{ /* exact match */
size *= 2;
delete buffer;
buffer = new WHATEVER[size];
continue;
} /* exact match */
} /* no error */
break;
} /* loop */
if(!ok)
{ /* solid error */
…deal with it
} /* solid error */

That is, if you fill the buffer exactly, you might have lost data, so the right thing to do is to try again with a larger buffer, and continue until you get an only-partially-filled buffer. There are several user-level APIs for which this is the only possible solution.

Note the code above can be refactored to avoid duplication, but I’m typing this in without a decent program editor, so I didn’t bother to reduce it, as I would have done had I been coding it in an actual deliverable product. Also, in writing C++, I would more likely use a collection like CArray or std::vector, which is why I’m a bit sketchy about details like how to delete and reallocate the buffer I would not use delete and new as shown, but something more complex but more robust.
joe

> The problem with the two ioctl approach (get size, then get buffer) is
> the window between the two. The size can change in which case you have
> to indicate “there is more” somehow. With the get buffer alone, you
> have the same issue of how to indicate there is more. So, since they
> both devolve to the same problem, just have a single get buffer ioctl.
> The driver can choose to fill in as many slots in the array as it has
> data
>
> d
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of JonathonS
> Sent: Monday, September 17, 2012 5:30 PM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] Passing variable-sized array from Kernel to User
>
> Hi all,
>
> I know this is a very simple question ( I apologize in advanced ) but
> I am just curious what the accepted approach is for passing a
> variable sized array from the kernel to user-space.
>
> I am familiar with sending data from user to kernel, but the other way
> seems a bit more complicated.
>
> Should I create two ioctls i.e. one for getting the size, and then
> another for sending it down to the driver? Or, perhaps create a MAX
> static sized buffer and have the driver populate that?
>
> Thanks,
> J
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

will the application be reading a stream of data or a series of packets of
variable size? If the first, then it doesn’t really matter as the driver
can just complete the call with whatever data it has and some calls will
read more and some less. If the second, and there is a maximum size, then a
trivial approach is to have the app always allocate enough for the maximum
size. If the maximum size is much larger than the typical size, or if it
cannot be predetermined, then Doron & Joe’s advise about returning a warning
status is probably the best.

“JonathonS” wrote in message news:xxxxx@ntdev…

Hi all,

I know this is a very simple question ( I apologize in advanced ) but
I am just curious what the *accepted* approach is for passing a
variable sized array from the kernel to user-space.

I am familiar with sending data from user to kernel, but the other way
seems a bit more complicated.

Should I create two ioctls i.e. one for getting the size, and then
another for sending it down to the driver? Or, perhaps create a MAX
static sized buffer and have the driver populate that?

Thanks,
J

Thanks guys for the help. It is much appreciated :slight_smile:

J

On Tue, Sep 18, 2012 at 2:18 PM, m wrote:
> will the application be reading a stream of data or a series of packets of
> variable size? If the first, then it doesn’t really matter as the driver
> can just complete the call with whatever data it has and some calls will
> read more and some less. If the second, and there is a maximum size, then a
> trivial approach is to have the app always allocate enough for the maximum
> size. If the maximum size is much larger than the typical size, or if it
> cannot be predetermined, then Doron & Joe’s advise about returning a warning
> status is probably the best.
>
> “JonathonS” wrote in message news:xxxxx@ntdev…
>
>
> Hi all,
>
> I know this is a very simple question ( I apologize in advanced ) but
> I am just curious what the accepted approach is for passing a
> variable sized array from the kernel to user-space.
>
> I am familiar with sending data from user to kernel, but the other way
> seems a bit more complicated.
>
> Should I create two ioctls i.e. one for getting the size, and then
> another for sending it down to the driver? Or, perhaps create a MAX
> static sized buffer and have the driver populate that?
>
> Thanks,
> J
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer