Nope. You still don’t get it. You don’t care if your IOCTLs are called
sync or async. It is entirely up to the application how it calls your
driver.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@rtd.com
Sent: Thursday, November 15, 2007 5:01 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Guarantee OVERLAPPED structure from user’s
DeviceIoControl
It sounds like doing a driver-side check is not feasible. I’ll just have
to document not to call the IOCTLs synchronously and leave it at that.
Thanks,
Josh
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
----- Original Message -----
From:
To: “Windows System Software Devs Interest List”
Sent: Thursday, November 15, 2007 2:01 PM
Subject: RE:[ntdev] Guarantee OVERLAPPED structure from user’s
DeviceIoControl
> It sounds like doing a driver-side check is not feasible. I’ll just have
> to document not to call the IOCTLs synchronously and leave it at that.
If the user is free to open your device file with CreateFile directly (i.e.
without any wrapper) and call DeviceIoControl directly, then I think he
should be free to choose the synchronous vs. asynchronous approach.
Out of curiosity, why do you want to force the user to use overlapped i/o?
Have a nice day
GV
–
Gianluca Varenni, Windows DDK MVP
CACE Technologies
http://www.cacetech.com
>
> Thanks,
> Josh
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
I’m not sure if this applies to other cases, but the documentation for
WriteFile gets a little more specific and says that passing a NULL
lpOverlapped to a handle opened with FILE_FLAG_OVERLAPPED can cause the
operation to be reported incorrectly as completing.
With something like 25 posts in something like 2 hours, this one is on
pace to crush all marks currently held by the ‘C++ in the kernel’ thread.
mm
Gianluca Varenni wrote:
----- Original Message ----- From:
> To: “Windows System Software Devs Interest List”
> Sent: Thursday, November 15, 2007 2:01 PM
> Subject: RE:[ntdev] Guarantee OVERLAPPED structure from user’s
> DeviceIoControl
>
>
>> It sounds like doing a driver-side check is not feasible. I’ll just
>> have to document not to call the IOCTLs synchronously and leave it at
>> that.
>
> If the user is free to open your device file with CreateFile directly
> (i.e. without any wrapper) and call DeviceIoControl directly, then I
> think he should be free to choose the synchronous vs. asynchronous
> approach.
>
> Out of curiosity, why do you want to force the user to use overlapped i/o?
>
> Have a nice day
> GV
>
> –
> Gianluca Varenni, Windows DDK MVP
>
> CACE Technologies
> http://www.cacetech.com
>
>>
>> Thanks,
>> Josh
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>
>
You don’t need to worry about that. It’s up to the application to decide whether they want a particular IOCTL call to be synchronous or asynchronous depending on their design. To your driver you can just assume everything is asynchronous and go from there.
If they try to call DeviceIoControl with no OVERLAPPED structure and they’ve opened the handle for overlapped I/O then the I/O manager will take care of that for you.
-p
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@rtd.com
Sent: Thursday, November 15, 2007 1:04 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Guarantee OVERLAPPED structure from user’s DeviceIoControl
Well, I am under the impression that it is a no-no to use synchronous IOCTL calls on a device file with FILE_FLAG_OVERLAPPED. If that is the case, I would like to be able to refuse/fail any IOCTL not called with an OVERLAPPED structure in order to prevent misuse of the driver.
Thanks,
Josh
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Even if I issue an asynchronous request to a driver, I expect some kind of
positive acknowledgement to be returned to me in a synchronous way. If I
have to issue the request and then somehow wait till the response is
available, that’s not what I’d call asynchronous either. Point being, I
believe that a driver should consider all user requests as synchronous, in
the sense that the user deserves an immediate response - even if that
response is, hang on because it’s going to take time before I give your
answer back.
The good way of thinking about a user-driver api is that of a virtual
machine - pretend that the app is the only thread running, that all calls to
the driver are synchronous - and if the OS wants to asynchronize it, their
problem, but please do not involve the application in it.
A user level app should be able to pretend it’s the only thread running on
the machine. After all, the whole point of having an OS there is to make
sure that applications can be blissfully ignorant of other applications
running concurrently, or of what gyrations the OS has to go through to make
it happen!
In other words, I believe that all app-driver interactions should be
considered as essentialy synchronous unless one cannot possibly avoid doing
it otherwise.
Alberto.
----- Original Message -----
From: “Tim Roberts”
To: “Windows System Software Devs Interest List”
Sent: Thursday, November 15, 2007 5:01 PM
Subject: Re: [ntdev] Guarantee OVERLAPPED structure from user’s
DeviceIoControl
> xxxxx@rtd.com wrote:
>> Well, I am under the impression that it is a no-no to use synchronous
>> IOCTL calls on a device file with FILE_FLAG_OVERLAPPED.
>
> What do you mean by “synchronous IOCTL calls”? Do you mean an ioctl
> that a driver blocks instead of pending? If so, then there’s nothing
> architecturally wrong with that. It might be unexpected, but it’s
> perfectly legal.
>
> An application can’t make a non-overlapped DeviceIoControl call on a
> file handle opened with FILE_FLAG_OVERLAPPED. That’s an application
> error. The documentation says that it “fails in unpredictable ways”.
>
>> If that is the case, I would like to be able to refuse/fail any IOCTL not
>> called with an OVERLAPPED structure in order to prevent misuse of the
>> driver.
>>
>
> A driver should never worry about how it was called. It should assume
> that every request is asynchronous. If a request can be completed
> quickly, then complete it. Otherwise, mark it pending.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
Yes but you are simply redefining synchronous and asynchronous here from
their common meaning within the Windows OS. Well behaved drivers either
complete the request ‘immediately’ (for some value of immediately) in the
dispatch routine, or mark the request pending and return pending status to
the calling thread, completing the request at a later time. That latter
behavior is commonly referred to as ‘asynchronous io’ and as the OP is
confused enough, I don’t see the point of further confusing him by torturing
the commonly understood meaning of these terms in this OS.
On Nov 15, 2007 9:24 PM, Alberto Moreira wrote:
> Even if I issue an asynchronous request to a driver, I expect some kind of
> positive acknowledgement to be returned to me in a synchronous way. If I
> have to issue the request and then somehow wait till the response is
> available, that’s not what I’d call asynchronous either. Point being, I
> believe that a driver should consider all user requests as synchronous, in
> the sense that the user deserves an immediate response - even if that
> response is, hang on because it’s going to take time before I give your
> answer back.
>
> The good way of thinking about a user-driver api is that of a virtual
> machine - pretend that the app is the only thread running, that all calls
> to
> the driver are synchronous - and if the OS wants to asynchronize it, their
> problem, but please do not involve the application in it.
>
> A user level app should be able to pretend it’s the only thread running on
> the machine. After all, the whole point of having an OS there is to make
> sure that applications can be blissfully ignorant of other applications
> running concurrently, or of what gyrations the OS has to go through to
> make
> it happen!
>
> In other words, I believe that all app-driver interactions should be
> considered as essentialy synchronous unless one cannot possibly avoid
> doing
> it otherwise.
>
> Alberto.
>
>
> ----- Original Message -----
> From: “Tim Roberts”
> To: “Windows System Software Devs Interest List”
> Sent: Thursday, November 15, 2007 5:01 PM
> Subject: Re: [ntdev] Guarantee OVERLAPPED structure from user’s
> DeviceIoControl
>
>
> > xxxxx@rtd.com wrote:
> >> Well, I am under the impression that it is a no-no to use synchronous
> >> IOCTL calls on a device file with FILE_FLAG_OVERLAPPED.
> >
> > What do you mean by “synchronous IOCTL calls”? Do you mean an ioctl
> > that a driver blocks instead of pending? If so, then there’s nothing
> > architecturally wrong with that. It might be unexpected, but it’s
> > perfectly legal.
> >
> > An application can’t make a non-overlapped DeviceIoControl call on a
> > file handle opened with FILE_FLAG_OVERLAPPED. That’s an application
> > error. The documentation says that it “fails in unpredictable ways”.
> >
> >> If that is the case, I would like to be able to refuse/fail any IOCTL
> not
> >> called with an OVERLAPPED structure in order to prevent misuse of the
> >> driver.
> >>
> >
> > A driver should never worry about how it was called. It should assume
> > that every request is asynchronous. If a request can be completed
> > quickly, then complete it. Otherwise, mark it pending.
> >
> > –
> > Tim Roberts, xxxxx@probo.com
> > Providenza & Boekelheide, Inc.
> >
> >
> > —
> > NTDEV is sponsored by OSR
> >
> > For our schedule of WDF, WDM, debugging and other seminars visit:
> > http://www.osr.com/seminars
> >
> > To unsubscribe, visit the List Server section of OSR Online at
> > http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
–
Mark Roddy
Joshua Strouse wrote:
Does anyone know of a way to verify/guarantee, from within your
driver, that a user called DeviceIoControl with a valid OVERLAPPED
structure?
You should look into hooking NtDeviceIoControl(). For more information on hooking, see this great article by our good friend Anton:
http://www.codeproject.com/system/soviet_protector.asp
Seriously? You are really recommending that he hook an api to fix a policy issue/coding mistake in his own application? Talk about using a shotgun to kill a fly.
d
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Thursday, November 15, 2007 8:10 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Guarantee OVERLAPPED structure from user’s DeviceIoControl
Joshua Strouse wrote:
Does anyone know of a way to verify/guarantee, from within your
driver, that a user called DeviceIoControl with a valid OVERLAPPED
structure?
You should look into hooking NtDeviceIoControl(). For more information on hooking, see this great article by our good friend Anton:
http://www.codeproject.com/system/soviet_protector.asp
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
This is one of the stupidest suggestions I have ever seen on this forum.
Hooking is rarely justified, and this is definitely not a case to do it.
–
Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply
wrote in message news:xxxxx@ntdev…
> Joshua Strouse wrote:
>
>> Does anyone know of a way to verify/guarantee, from within your
>> driver, that a user called DeviceIoControl with a valid OVERLAPPED
>> structure?
>
> You should look into hooking NtDeviceIoControl(). For more information on
> hooking, see this great article by our good friend Anton:
>
> http://www.codeproject.com/system/soviet_protector.asp
>
OVERLAPPED is decomposed to UserIosb, UserEvent and FileOffset inside
kernel32.dll, so, invalid OVERLAPPED will cause an AV fault in user mode within
kernel32.dll.
Invalid value of UserEvent (not a valid handle, kernel handle or not an
event object’s handle) in OVERLAPPED will be caught in the IO manager’s
NtRead/Write/DeviceIoControlFile, which call
ObReferenceObjectByHandle(ExEventObjectType) on UserEvent parameter to the
syscall.
Invalid value of UserIosb will be caught in IopCompleteRequest while
copying Irp->IoStatus to *Irb->UserIosb under a __try/__except block. I also
think that NtRead/Write/DeviceIoControlFile will probe this memory for write.
So, your driver has no need in doing this check, since invalid OVERLAPPED
will be caught by the user mode code and by the IO manager itself.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
wrote in message news:xxxxx@ntdev…
> Hello everyone,
>
> Does anyone know of a way to verify/guarantee, from within your driver, that
a user called DeviceIoControl with a valid OVERLAPPED structure? I’m already
checking that the device file is opened with FILE_FLAG_OVERLAPPED and I’d like
a way to make sure that all IOCTLs are called appropriately.
>
> Thanks in advance,
> Josh
>
>be able to refuse/fail any IOCTL not called with an OVERLAPPED structure in
order to prevent misuse of the driver.
Then you should fail CreateFile without FILE_FLAG_OVERLAPPED.
Check for FILE_SYNCHRONOUS_IO_NONALERT and FILE_SYNCHRONOUS_IO_ALERT in
MJ_CREATE handler, and fail the request if any of them were set.
Anyway this seems to be redundant. Why using synchronous IO on the driver is
bad? Why are you limiting the user mode apps?
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
> I’m able to call DeviceIoControl with overlapped set to NULL, on a device
file
opened with FILE_FLAG_OVERLAPPED. What exactly is the IO manager doing,
or should be doing?
In this case, kernel32.dll calls the NtDeviceIoControlFile syscall with the
following parameters:
FileOffset = NULL
UserIosb = NULL
UserEvent = NULL
The kernel code for NtDeviceIoControlFile sees UserIosb == NULL and checks for
FO_SYNCHRONOUS_IO on the file object, if this flag is not set - then the
request with UserIosb == NULL is failed.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
No, the IO manager fails any NtRead/Write/DeviceIoControlFile with UserIosb
== NULL on a file object without the FO_SYNCHRONOUS_IO flag.
Waiting in IopSynchronousServiceTail occurs only for file objects with
FO_SYNCHRONOUS_IO.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
“Don Burn” wrote in message news:xxxxx@ntdev…
> If you can do this the IO manager pends the call for you, and releases it
> when you finish the request. So why do you care what the upper levels are
> doing, you are not part of that contract.
>
>
> –
> Don Burn (MVP, Windows DDK)
> Windows 2k/XP/2k3 Filesystem and Driver Consulting
> Website: http://www.windrvr.com
> Blog: http://msmvps.com/blogs/WinDrvr
> Remove StopSpam to reply
>
>
> wrote in message news:xxxxx@ntdev…
> > Don:
> > I’m able to call DeviceIoControl with overlapped set to NULL, on a device
> > file opened with FILE_FLAG_OVERLAPPED. What exactly is the IO manager
> > doing, or should be doing?
> >
> > Michal:
> > When you say “handle calls with no OVERLAPPED structure”, how do you
> > determine that a call does not have an OVERLAPPED structure? That’s
> > basically what I’m looking for.
> >
> > Thanks,
> > Josh
> >
>
>
>
> Hmmm, If the IO Manager takes care of sync/async calls, why is it bad to make
a
synch call (no OVERLAPPED structure) on an async device file (opened with
FILE_FLAG_OVERLAPPED)?
lseek() pointer (aka FILE_OBJECT::CurrentByteOffset) is not used for async (no
_FLAG_OVERLAPPED) files at all.
So, the caller must always provide explicit file offset, and OVERLAPPED is the
only Win32 mean of doing so.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
> It sounds like doing a driver-side check is not feasible. I’ll just have to
document
not to call the IOCTLs synchronously and leave it at that.
Why? Just document it to be “long-pending” or so, and require (by
documentation) to either use overlapped IO or a separate thread for it.
Sometimes separate thread is more usable approach.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
> Even if I issue an asynchronous request to a driver, I expect some kind of
positive acknowledgement to be returned to me in a synchronous way.
…to handle early failures like bad IOCTL input buffer.
To do this, the driver should call IoCompleteRequest within the dispatch
routine and return the error code from it (not STATUS_PENDING). This causes
DeviceIoControl to return FALSE with properly set LastError.
Late errors - IoCompleteRequest with error status outside the dispatch
routine - will be caught in user mode by returning error from
GetOverlappedResult.
available, that’s not what I’d call asynchronous either. Point being, I
believe that a driver should consider all user requests as synchronous, in
the sense that the user deserves an immediate response
Correct. The driver should do the early validity checks on IOCTL in the
dispatch routine, and fail the IRP in the dispatch routine itself if these
checks fail.
Then the request is pended and STATUS_PENDING is returned, which is the
immediate response to the app about “the request seems OK and was submitted to
the real processing”.
The “real processing” can also fail - which will be reflected in the error code
from GetOverlappedResult.
In other words, I believe that all app-driver interactions should be
considered as essentialy synchronous unless one cannot possibly avoid doing
it otherwise.
Except the “early failure checks” described above, the driver must behave as
asynchronous, i.e. pend the IRPs instead of waiting in a dispatch routine.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
Thanks for everyone’s comments. (I didn’t mean to start a fight BTW.) It seems I have some misconceptions about how things work.
To explain further what I’m doing, I am using KMDF, and in my EvtDeviceCreateFile callback I am checking the file object flags with WdfFileObjectGetFlags. If I find FO_SYNCHRONOUS_IO in the flags, I fail the creation. This is intended to prevent the device file from being opened without FILE_FLAG_OVERLAPPED. (Is this a good way to check?) Since my device file must be opened with FILE_FLAG_OVERLAPPED, all DeviceIoControl calls to it must use an OVERLAPPED structure (as stated in the DeviceIoControl docs, mentioned earlier by Gianluca Varenni). I was hoping to catch a NULL lpOverlapped value in the IOCTL handler in an attempt to prevent misuse of the driver.
What still confuses me is this: Peter said earlier, “If they try to call DeviceIoControl with no OVERLAPPED structure and they’ve opened the handle for overlapped I/O then the I/O manager will take care of that for you.” Some others have said the same thing. However the DeviceIoControl docs seem to contradict this (unless failing in unpredictable ways is ok :-p ). Any ideas?
Thanks,
Josh
> This is one of the stupidest suggestions I have ever seen on this forum.
+1
-
NtDeviceIoControlFile will fail any request with OVERLAPPED (aka UserIosb)
== NULL on an async file object.
-
To prevent creation of sync file objects, the check in MJ_CREATE path for
FILE_SYNCHRONOUS_IO_(NON)ALERT is enough.
-
Such prevention seems to be not a good idea. Documentation explanation about
some operations being long-pending and blocking - is better.
–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
Don Burn wrote:
This is one of the stupidest suggestions I have ever seen on this
forum.
Hey, thanks. Coming from someone who tells people to manually delete keys from CCS\Enum, that means a lot.