Guarantee OVERLAPPED structure from user's DeviceIoControl

Why stop there?

The point is that there is already an extensive layer of software written by
Microsoft that deals with these sorts of issues in a standard way. This sort
of programmer error is outside the domain that a device driver needs to
concern itself with. An example of what a driver ought to concern itself
with is IOCTL content validation.

As somebody else mentioned, if this minor issue of application programmer
idiocy, idiocy that can only cause said application and not your driver
problems, really bothers you, write a user mode dll that provides a wrapper
around your device IOCTLs that controls user mode access they way you want.

As you have redefined your problem to determining in your driver if a caller
of the Win32 DeviceIoControl API has specified an OVERLAPPED object in that
specific call to DeviceIoControl, the answer is no, your driver cannot
determine that from the IRP contents as that is a win32 construct that has
no manifestation in the IRP data available to your driver in its
DeviceControl dispatch routine.

On Nov 16, 2007 11:54 AM, wrote:

> Mark: I’m sorry I haven’t been able to explain things clearly. I
> completely understand that it doesn’t matter how the IOCTL was called, as
> far as my driver is concerned. My intention was for the driver to help
> prevent user errors. The particular error I was trying to prevent was the
> case of a user calling DeviceIoControl with lpOverlapped set to NULL on a
> file that was opened with FILE_FLAG_OVERLAPPED. The documentation for
> DeviceIoControl says that this fails unpredictably. Others have said this is
> ok. I’m trying to find out which one is true.
>
> Thanks for your patience,
> Josh
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>


Mark Roddy

Tim: Thank you for your response. I was hoping there would be a consensus of some sort on this point, but it seems some people are confident in disregarding the documentation’s warning. I agree with you however and I don’t want to take the chance either.

Doron: I realized earlier that what I had originally asked about was not feasible, and even if it was it should still not be done. I continued asking questions to try and clear up my confusion on DeviceIoControl’s documentation, which Tim cleared up for me.

I’m sorry I frustrated everyone. My intention was to try and keep the user from doing the ‘wrong’ thing. As people have pointed out, what I was intending to protect against just isn’t my problem and I should ignore it. Thanks for the alternatives and explanations that were posted, I appreciate it. Sorry again everyone for being obtuse. (I hope I didn’t raise anyone’s blood pressure! :slight_smile: )

Thanks,
Josh

Well that is just the semantics of the win32 API. It is, once again, an
application level programming issue, not a driver programming issue.

On Nov 16, 2007 12:07 PM, Maxim S. Shatskih wrote:

> > your driver to do this checking at all. A thread that opens your device
> in
> > synchronous mode is not going to force your driver to behave
> synchronously,
> > nor is it going to prevent some other thread from opening your device in
> > asynchronous mode.
>
> Yes, but the file object created without _OVERLAPPED serializes all
> threads
> trying to do IO on it using a mutex-like lock. So, you cannot have more
> then 1
> outstanding IRP a time on such a sync file object.
>
> Once I had a task to write a multithreaded service which will read some
> file
> from many threads. Regardless of the fact I do not need any async IO at
> all, I
> was forced by the above thing to use an overlapped file object, with the
> nuisance of GetOverlappedResult. The only other solution was to create a
> file
> object per thread.
>
> –
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>


Mark Roddy

> I’m sorry I frustrated everyone. My intention was to try and keep the user
from

doing the ‘wrong’ thing.

Just document it, no need to impose limitations.


Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

>> I’m sorry I frustrated everyone. My intention was to try and

> keep the user from doing the ‘wrong’ thing.

Just document it, no need to impose limitations.

So did we beat the “C++ in the kernel” thread?

I was hoping my hooking idea would push us over the threshold…

xxxxx@gmail.com wrote:

>> I’m sorry I frustrated everyone. My intention was to try and
>> keep the user from doing the ‘wrong’ thing.
>>
> Just document it, no need to impose limitations.
>

So did we beat the “C++ in the kernel” thread?

I was hoping my hooking idea would push us over the threshold…

Not even close. Going through my archive, it looks like the last big
“C++ in the kernel thread” (April 2006) ended up with 288 posts.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

No we did not beat C++ … yet. Out of many instances of C++ discussion, I
don’t think we even beat the least credible one ( least credible meaning the
thd has least number of response ).

I’m sure it would have gone beyond any of those, if you put out some hooking
codes ( application hooks, dll hooks, registry hooks, kernel hooks specially
w2ksys, including 32bit and 64bit ways, including patchguard bypass,
replacing kernel codes on the fly etc.etc :slight_smile:

Thankfully it did not take that route :slight_smile:

-pro
----- Original Message -----
From:
To: “Windows System Software Devs Interest List”
Sent: Friday, November 16, 2007 2:21 PM
Subject: RE:[ntdev] Guarantee OVERLAPPED structure from user’s
DeviceIoControl

>>> I’m sorry I frustrated everyone. My intention was to try and
>>> keep the user from doing the ‘wrong’ thing.
>>
>> Just document it, no need to impose limitations.
>
> So did we beat the “C++ in the kernel” thread?
>
> I was hoping my hooking idea would push us over the threshold…
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

True, it is only about a quarter of the way there, and mercifully
appears to be coming to an end, but this thread is most remarkable in
that achieved a consensus, which is rare in and of itself on this list,
did so more or less immediately, and then this process was repeated over
and over and over, pretty much without any variation; even bring up
hooking couldn’t derail this one.

For a while I was thinking that there had to be something in the water.

mm

Tim Roberts wrote:

xxxxx@gmail.com wrote:
>>> I’m sorry I frustrated everyone. My intention was to try and
>>> keep the user from doing the ‘wrong’ thing.
>>>
>> Just document it, no need to impose limitations.
>>
> So did we beat the “C++ in the kernel” thread?
>
> I was hoping my hooking idea would push us over the threshold…
>

Not even close. Going through my archive, it looks like the last big
“C++ in the kernel thread” (April 2006) ended up with 288 posts.

I’ve heard a rumor that one can actually delete old email. I’ve never tried
it myself, but some claim it can be done.

On Nov 16, 2007 5:56 PM, Tim Roberts wrote:

> xxxxx@gmail.com wrote:
> >>> I’m sorry I frustrated everyone. My intention was to try and
> >>> keep the user from doing the ‘wrong’ thing.
> >>>
> >> Just document it, no need to impose limitations.
> >>
> >
> > So did we beat the “C++ in the kernel” thread?
> >
> > I was hoping my hooking idea would push us over the threshold…
> >
>
> Not even close. Going through my archive, it looks like the last big
> “C++ in the kernel thread” (April 2006) ended up with 288 posts.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>


Mark Roddy

I am a computer science dude, and I use those words in the standard fashion that academia uses them. Synchronous is this: the app only continues after the OS returns a status. Asynchronous, on the other hand, returns control immediately to the application, as soon as the request is caught by the OS: notification of completion reaches the app eventually, for example, in a queue or through a message; but the app doesn’t wait for it, except in the sense that the OS positively acknowledges the receipt of the request. Either way, I believe that this kind of thing should be no concern of the driver, it should be an issue between the app and the OS. The API between OS and driver should be pretty much transaction-like, one request demands one response; synchronicity games should be played by the OS alone, drivers are there to handle hardware!

Alberto.

----- Original Message -----
From: Mark Roddy
To: Windows System Software Devs Interest List
Sent: Thursday, November 15, 2007 11:00 PM
Subject: Re: [ntdev] Guarantee OVERLAPPED structure from user’s DeviceIoControl

Yes but you are simply redefining synchronous and asynchronous here from their common meaning within the Windows OS. Well behaved drivers either complete the request ‘immediately’ (for some value of immediately) in the dispatch routine, or mark the request pending and return pending status to the calling thread, completing the request at a later time. That latter behavior is commonly referred to as ‘asynchronous io’ and as the OP is confused enough, I don’t see the point of further confusing him by torturing the commonly understood meaning of these terms in this OS.

On Nov 15, 2007 9:24 PM, Alberto Moreira wrote:

Even if I issue an asynchronous request to a driver, I expect some kind of
positive acknowledgement to be returned to me in a synchronous way. If I
have to issue the request and then somehow wait till the response is
available, that’s not what I’d call asynchronous either. Point being, I
believe that a driver should consider all user requests as synchronous, in
the sense that the user deserves an immediate response - even if that
response is, hang on because it’s going to take time before I give your
answer back.

The good way of thinking about a user-driver api is that of a virtual
machine - pretend that the app is the only thread running, that all calls to
the driver are synchronous - and if the OS wants to asynchronize it, their
problem, but please do not involve the application in it.

A user level app should be able to pretend it’s the only thread running on
the machine. After all, the whole point of having an OS there is to make
sure that applications can be blissfully ignorant of other applications
running concurrently, or of what gyrations the OS has to go through to make
it happen!

In other words, I believe that all app-driver interactions should be
considered as essentialy synchronous unless one cannot possibly avoid doing
it otherwise.

Alberto.

----- Original Message -----
From: “Tim Roberts”
To: “Windows System Software Devs Interest List” < xxxxx@lists.osr.com>
Sent: Thursday, November 15, 2007 5:01 PM
Subject: Re: [ntdev] Guarantee OVERLAPPED structure from user’s
DeviceIoControl

> xxxxx@rtd.com wrote:
>> Well, I am under the impression that it is a no-no to use synchronous
>> IOCTL calls on a device file with FILE_FLAG_OVERLAPPED.
>
> What do you mean by “synchronous IOCTL calls”? Do you mean an ioctl
> that a driver blocks instead of pending? If so, then there’s nothing
> architecturally wrong with that. It might be unexpected, but it’s
> perfectly legal.
>
> An application can’t make a non-overlapped DeviceIoControl call on a
> file handle opened with FILE_FLAG_OVERLAPPED. That’s an application
> error. The documentation says that it “fails in unpredictable ways”.
>
>> If that is the case, I would like to be able to refuse/fail any IOCTL not
>> called with an OVERLAPPED structure in order to prevent misuse of the
>> driver.
>>
>
> A driver should never worry about how it was called. It should assume
> that every request is asynchronous. If a request can be completed
> quickly, then complete it. Otherwise, mark it pending.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


Mark Roddy — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Sorry to jump in :slight_smile:

The final part "synchronicity games should be played by the OS alone ", ( and of course it is not really related to the thread as such ) is not what the current trend(s). Drivers are really extension to kernel ( not just to drive HW).

Frankly I’m scared to the idea, because we are all aware of the fact that “programmers should have options …” this was one of the marked difference between old school and modern school. This is also a very difficult choices for framework designers - we often call it capability to override.

This is to say “BTW, you don’t have to do anything if you agree with default behaviour, and of course you can override the behaviour if you are so willing …”. If you mean this, then I agree, otherwise I/O framework would be to strict.

Still I need to learn a bit about KMDF, but this is what I like about it.

-pro
----- Original Message -----
From: Alberto Moreira
To: Windows System Software Devs Interest List
Sent: Friday, November 16, 2007 7:44 PM
Subject: Re: [ntdev] Guarantee OVERLAPPED structure from user’s DeviceIoControl

I am a computer science dude, and I use those words in the standard fashion that academia uses them. Synchronous is this: the app only continues after the OS returns a status. Asynchronous, on the other hand, returns control immediately to the application, as soon as the request is caught by the OS: notification of completion reaches the app eventually, for example, in a queue or through a message; but the app doesn’t wait for it, except in the sense that the OS positively acknowledges the receipt of the request. Either way, I believe that this kind of thing should be no concern of the driver, it should be an issue between the app and the OS. The API between OS and driver should be pretty much transaction-like, one request demands one response; synchronicity games should be played by the OS alone, drivers are there to handle hardware!

Alberto.

----- Original Message -----
From: Mark Roddy
To: Windows System Software Devs Interest List
Sent: Thursday, November 15, 2007 11:00 PM
Subject: Re: [ntdev] Guarantee OVERLAPPED structure from user’s DeviceIoControl

Yes but you are simply redefining synchronous and asynchronous here from their common meaning within the Windows OS. Well behaved drivers either complete the request ‘immediately’ (for some value of immediately) in the dispatch routine, or mark the request pending and return pending status to the calling thread, completing the request at a later time. That latter behavior is commonly referred to as ‘asynchronous io’ and as the OP is confused enough, I don’t see the point of further confusing him by torturing the commonly understood meaning of these terms in this OS.

On Nov 15, 2007 9:24 PM, Alberto Moreira wrote:

Even if I issue an asynchronous request to a driver, I expect some kind of
positive acknowledgement to be returned to me in a synchronous way. If I
have to issue the request and then somehow wait till the response is
available, that’s not what I’d call asynchronous either. Point being, I
believe that a driver should consider all user requests as synchronous, in
the sense that the user deserves an immediate response - even if that
response is, hang on because it’s going to take time before I give your
answer back.

The good way of thinking about a user-driver api is that of a virtual
machine - pretend that the app is the only thread running, that all calls to
the driver are synchronous - and if the OS wants to asynchronize it, their
problem, but please do not involve the application in it.

A user level app should be able to pretend it’s the only thread running on
the machine. After all, the whole point of having an OS there is to make
sure that applications can be blissfully ignorant of other applications
running concurrently, or of what gyrations the OS has to go through to make
it happen!

In other words, I believe that all app-driver interactions should be
considered as essentialy synchronous unless one cannot possibly avoid doing
it otherwise.

Alberto.

----- Original Message -----
From: “Tim Roberts”
To: “Windows System Software Devs Interest List” < xxxxx@lists.osr.com>
Sent: Thursday, November 15, 2007 5:01 PM
Subject: Re: [ntdev] Guarantee OVERLAPPED structure from user’s
DeviceIoControl

> xxxxx@rtd.com wrote:
>> Well, I am under the impression that it is a no-no to use synchronous
>> IOCTL calls on a device file with FILE_FLAG_OVERLAPPED.
>
> What do you mean by “synchronous IOCTL calls”? Do you mean an ioctl
> that a driver blocks instead of pending? If so, then there’s nothing
> architecturally wrong with that. It might be unexpected, but it’s
> perfectly legal.
>
> An application can’t make a non-overlapped DeviceIoControl call on a
> file handle opened with FILE_FLAG_OVERLAPPED. That’s an application
> error. The documentation says that it “fails in unpredictable ways”.
>
>> If that is the case, I would like to be able to refuse/fail any IOCTL not
>> called with an OVERLAPPED structure in order to prevent misuse of the
>> driver.
>>
>
> A driver should never worry about how it was called. It should assume
> that every request is asynchronous. If a request can be completed
> quickly, then complete it. Otherwise, mark it pending.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


Mark Roddy — NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer