Inverted Callback sending strings

John_Reed · July 15, 2011, 11:45am

I have a question as the best way to allocate memory to send a string to the user mode. I am using inverted callback and currently use a shared structure that I fill in the kernel mode and copy to the IRP that has the same structure and complete it. When I do this I just declare my variables with the size of the string in the structure i.e.

wchar_t path[512];

However this is inefficient because what if the path is smaller or larger. So I want to declare the variable in the structure like this:

wchar_t* path;

and allocate the size of the string and send this back with allocating only the correct size. Now the problem I am having is that when I send the structure with the IRP its not going to know the size of the string until it wants to complete the IRP so I cant allocate memory beforehand for the string. Am I going to have to send an IRP to get the size of the string go back to user mode and allocate enough memory for the string then send another IRP with enough space for the string, copy it and send it back? Is that the correct way of doing this? Or is there a more efficient way of doing this?
Thanks for the help.

Alex_Grig · July 15, 2011, 12:25pm

Make the array the end of your structure, and return only as much bytes as actually used.

You can’t reliably allocate memory in kernel and give it to the user process. There are ways, but when you consider all the corner cases (especially cleanup after the application termination), you’d rather don’t go there.

Doron_Holan · July 15, 2011, 12:27pm

Either you live with the fixed larger size or you do as you suggest ,one request to get the size , another request with the size needed (buffet allocated in user mode )

d

debt from my phone

-----Original Message-----
From: xxxxx@yahoo.com
Sent: Friday, July 15, 2011 8:46 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] Inverted Callback sending strings

I have a question as the best way to allocate memory to send a string to the user mode. I am using inverted callback and currently use a shared structure that I fill in the kernel mode and copy to the IRP that has the same structure and complete it. When I do this I just declare my variables with the size of the string in the structure i.e.

wchar_t path[512];

However this is inefficient because what if the path is smaller or larger. So I want to declare the variable in the structure like this:

wchar_t* path;

and allocate the size of the string and send this back with allocating only the correct size. Now the problem I am having is that when I send the structure with the IRP its not going to know the size of the string until it wants to complete the IRP so I cant allocate memory beforehand for the string. Am I going to have to send an IRP to get the size of the string go back to user mode and allocate enough memory for the string then send another IRP with enough space for the string, copy it and send it back? Is that the correct way of doing this? Or is there a more efficient way of doing this?
Thanks for the help.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Tim_Roberts · July 15, 2011, 1:15pm

xxxxx@yahoo.com wrote:

I have a question as the best way to allocate memory to send a string to the user mode. I am using inverted callback and currently use a shared structure that I fill in the kernel mode and copy to the IRP that has the same structure and complete it. … Am I going to have to send an IRP to get the size of the string go back to user mode and allocate enough memory for the string then send another IRP with enough space for the string, copy it and send it back? Is that the correct way of doing this?

That is a perfectly fine way to handle it.

Or is there a more efficient way of doing this?

Unless you are making this call 1,000 times a second, efficiency isn’t
really a concern. The two-call method is safe, and that’s more important.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

John_Reed · July 15, 2011, 3:18pm

Thanks for the help. Ill go with the 2 call method then

OSR_Community_User · July 15, 2011, 9:35pm

See below…

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@yahoo.com
Sent: Friday, July 15, 2011 11:48 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] Inverted Callback sending strings

I have a question as the best way to allocate memory to send a string to the
user mode. I am using inverted callback and currently use a shared structure
that I fill in the kernel mode and copy to the IRP that has the same
structure and complete it.

****
What I think you are saying is that in the inverted call, you are using
buffered I/O, and there is a struct which defines the buffer. The phrase
“copy to the IRP” does not make sense; what you probably meant to say was
“copying the internal structure in the kernel to the buffer passed in via
the IRP”
****

When I do this I just declare my variables with the size of the string in
the structure i.e.

wchar_t path[512];

****
If this is actually a path, then it would MAX_PATH in length (which I think
is 260 characters). If you want to accommodate the maximum possible path,
then you have to allow for 32,768 characters (which is unusual to the point
where you can simply declare that you don’t support it).
****

However this is inefficient because what if the path is smaller or larger.
So I want to declare the variable in the structure like this:

wchar_t* path;

****
While this can work in the kernel, you can’t do this in user space in any
meaningful way, because the kernel cannot create allocations in user space.
Note that you have to define “inefficient” in terms of a modern x86 system
(that is, 512 characters, 1024 bytes, represents 0.00005% of the 2GB address
space you have available, and thus is so tiny as to not be worth worrying
about (allocated on the stack, this would not be true). So it is not clear
what you are “optimizing” here; we are no longer programming PDP-11s. Too
many books are written by people who programmed computers that had 64K
memory and seem to emphasize the wrong aspects of programming.
*****

and allocate the size of the string and send this back with allocating only
the correct size. Now the problem I am having is that when I send the
structure with the IRP its not going to know the size of the string until it
wants to complete the IRP so I cant allocate memory beforehand for the
string. Am I going to have to send an IRP to get the size of the string go
back to user mode and allocate enough memory for the string then send
another IRP with enough space for the string, copy it and send it back? Is
that the correct way of doing this? Or is there a more efficient way of
doing this?
****
One thing you can do is that instead of completing the IRP with the data,
you complete an IRP that has the data length, then send down another IRP to
retrieve the string, but this involves possible race conditions and how to
properly associate the string with the previously-completed IRP.

Another conjectured solution might be to include something like

typedef struct {
…stuff
wchar_t * path;
UINT pathlength;
…other stuff
} MyStruct;

Now, you can allocate any size you want in user space, and copy into it as
long as your path is < pathlength characters. But this introduces a whole
new level of nightmare. You have to read that user level address in ‘path’,
make it a kernel address, lock it down, and deal with problems of how you
migrate to a 64-bit world, particularly one in which a 32-bit app might be
running and calling I/O. Overall, your idea of returning the length and
then asking for the string is better. But I’d simplify it by simply stating
that only paths of MAX_PATH length are supported. The exceptions are going
to be fairly small, and I wouldn’t worry about infinitesimally tiny sizes
like 520 bytes (MAX_PATH * sizeof(WCHAR)). After all, you aren’t going to
have more than one of these per inverted call, and this is just silly to
optimize! You don’t start worrying about optimizing storage until you
create structures whose multiple instances involve tens of megabytes and can
buy more than 10% with an optimization.

A sense of proportion is important in deciding what to optimize.
****

Thanks for the help.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Maxim_S_Shatskih · July 18, 2011, 12:47pm

> Thanks for the help. Ill go with the 2 call method then

If you will provide the large enough buffer for the 1st call, then there are good chances it will do the operation immediately and thus the 2nd call will not be needed.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

John_Reed · July 20, 2011, 11:50am

Ok now I have a question about deallocation. Lets say I have the structure below:
{
wchar_t* appPath;
int appPathSize;
}STRUCT, *PSTRUCT;

I allocate this structure. Then I allocate space for the appPath variable. When I want to deallocate do I have to first deallocate the appPath variable and then the structure? Or will deallocating the structure also deallocate the appPath variable?

Ive been trying to deallocate the appPath and then deallocate the structure, but I am getting the Bug Check 0x19: BAD_POOL_HEADER with the pool block header size is corrupt.

Don_Burn_1 · July 20, 2011, 11:57am

You need to free the memory pointed to by appPath as a separate
operation. I hope you are not passing this structure to a driver, it
would mean you are doing all the memory verification yourself which is a
bad idea.

Don Burn
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

“xxxxx@yahoo.com” wrote in message
news:xxxxx@ntdev:

> Ok now I have a question about deallocation. Lets say I have the structure below:
> {
> wchar_t* appPath;
> int appPathSize;
> }STRUCT, *PSTRUCT;
>
> I allocate this structure. Then I allocate space for the appPath variable. When I want to deallocate do I have to first deallocate the appPath variable and then the structure? Or will deallocating the structure also deallocate the appPath variable?
>
> Ive been trying to deallocate the appPath and then deallocate the structure, but I am getting the Bug Check 0x19: BAD_POOL_HEADER with the pool block header size is corrupt.

Tim_Roberts · July 20, 2011, 1:11pm

xxxxx@yahoo.com wrote:

Ok now I have a question about deallocation. Lets say I have the structure below:
{
wchar_t* appPath;
int appPathSize;
}STRUCT, *PSTRUCT;

I allocate this structure. Then I allocate space for the appPath variable. When I want to deallocate do I have to first deallocate the appPath variable and then the structure? Or will deallocating the structure also deallocate the appPath variable?

This is C Programming 101. The order doesn’t matter, but it should be
obvious that you have to free appPath separately. The structure
consists of exactly 8 bytes (or 12…). It doesn’t include any strings.

Ive been trying to deallocate the appPath and then deallocate the structure, but I am getting the Bug Check 0x19: BAD_POOL_HEADER with the pool block header size is corrupt.

What’s the sequence here? How does this relate to inverted callback?
Is the structure being allocated in user mode? If so, then you can’t
free it in kernel mode.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

MBond · July 20, 2011, 7:55pm

Try this kind of logic in UM. (lol this takes me back

NOTE in KM you will still need to check of BufferSize is valid versus the
size of the buffer that was sent (and that it is big enough to hold a
SOME_STRUCT structure too)

typedef struct tagSOME_STRUCT
{
DWORD BufferSize;
DWORD RequiredBufferSize;
WCHAR Buffer;
}SOME_STRUCT;

SomeFunction()
{
SOME_STRUCT* pSomething;
DWORD BufferSize;

BufferSize = CONST_INITIAL_BUFFER_SIZE;

TryAgain:
pSomething = (SOME_STRUCT*)HeapAlloc(GetProcessHeap(), 0,
sizeof(SOME_STRUCT) + BufferSize * sizeof(WCHAR))
if(pSomthing == NULL)
{
’ do something
}

pSomething->BufferSize = BufferSize;

// pass to your driver via IOCTL / ReadFile / WriteFile
// your driver will fill this structure with chars up to BufferSize
chars and put the full number of chars available into RequiredBufferSize

if(pSomthing->BufferSize < pSomthing->RequiredBufferSize)
{
// the buffer size we tried initially is too small. Free it and
allocate a new one that will be large enough for the data
BufferSize = pSomthing->RequiredBufferSize
HeapFree(GetProcessHeap(), 0, pSomething);
goto TryAgain;
}

// do something with the returned chars

// free the memory used for pSomething now that whatever it is needed
for is over
HeapFree(GetProcessHeap(), 0, pSomething);

}

wrote in message news:xxxxx@ntdev…

Ok now I have a question about deallocation. Lets say I have the structure
below:
{
wchar_t* appPath;
int appPathSize;
}STRUCT, *PSTRUCT;

I allocate this structure. Then I allocate space for the appPath variable.
When I want to deallocate do I have to first deallocate the appPath variable
and then the structure? Or will deallocating the structure also deallocate
the appPath variable?

Ive been trying to deallocate the appPath and then deallocate the structure,
but I am getting the Bug Check 0x19: BAD_POOL_HEADER with the pool block
header size is corrupt.

OSR_Community_User · July 20, 2011, 9:36pm

Note: you clearly need to read a book on introductory C programming! The
answer is obvious if you understand C. If you delete an instance of STRUCT,
NOTHING AT ALL WILL HAPPEN to any storage referenced by appPath. How could
it? Do you see anything that could possibly do this? Answer: no, because
there isn’t.

Now in C++, I could create a ~STRUCT() destructor that would call delete on
appPath, assuming that appPath was not shared with any other object, or I
could use smart pointers to manage the storage referenced by appPath so the
storage was freed when the last object referencing it was deleted. But
that’s C++, not C. In C, if you allocate it, you must free it; there is no
magic. It wouldn’t even make *sense* to have a free operation recursively
free everything pointed to by pointers in the structure! What you are
missing here is the concept of “ownership” of an object, that is, the
(abstract) owner is the person responsible for cleaning things up. In C,
the notion of ownership is based on what code you write, because it is a
very abstract concept that you are responsible for implementing correctly
(this is why so many C programs crash with bad memory references, because
the programmer has mismanaged what is going on). In C++, the concept of
ownership is also the responsibility of the programmer, but you can manage
it more coherently. Of course, no sane C++ programmer would ever use a
wchar_t * for anything, because std::wstring would be the mechanism of
choice.

But I think you are still missing the whole point of what is going on here.
If this is a structure you need to pass into the kernel, you cannot put
user-mode pointers in it, so forget it! If you need to fill in the path
from the kernel, the correct structure would be
typedef struct {
WCHAR appPath[MAX_PATH];
ULONG appPathSize;
} STRUCT, *PSTRUCT;

The structure you are showing makes no sense whatsoever for
application-to-kernel communications, unless you plan to do all pointer
validation yourself, and that is not something you want to consider! For
example, you have to validate the buffer address in the kernel, and in a
64-bit driver, that struct may be either a 64-bit WCHAR * or a 32-bit WCHAR
* (unless you do not allow 32-bit apps running under Win64 to call your
driver, and are prepared to detect and reject such connections).

Overall, your problems stem from a completely misplaced concept of
“efficiency” of storage, which is that you must manage variable-length
strings in a context where such strings provide no advantage, and in fact
provide significant disadvantage! If I were supervising this project, the
first thing I would have told you is that you have no idea what trouble this
will get you into, and just forget it. It would then be your responsibility
to convince me that saving a couple hundred bytes of 2GB (or 8TB) user space
justified the time, expense, and compromises to robustness that the pointer
solution would require. I think the whole design path you are following is
erroneous, and the sooner you abandon it, the better off you will be.
joe

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@yahoo.com
Sent: Wednesday, July 20, 2011 11:50 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Inverted Callback sending strings

Ok now I have a question about deallocation. Lets say I have the structure
below:
{
wchar_t* appPath;
int appPathSize;
}STRUCT, *PSTRUCT;

I allocate this structure. Then I allocate space for the appPath variable.
When I want to deallocate do I have to first deallocate the appPath variable
and then the structure? Or will deallocating the structure also deallocate
the appPath variable?

Ive been trying to deallocate the appPath and then deallocate the structure,
but I am getting the Bug Check 0x19: BAD_POOL_HEADER with the pool block
header size is corrupt.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer