Hello all,
can a device support both Buffered and direct IO? I want to have a device
which supports both types and depending on the type of the IOCTL either
method can be used?
m_pDeviceObject->Flags |= DO_DIRECT_IO;
m_pDeviceObject->Flags |= DO_BUFFERED_IO;
thanks
al.
The IO values you set on DeviceObject->Flags control read and write irp buffer encoding. They do not control IOCTL buffer encoding. Each IOCTL value can specify how the input and output buffers are encoded. It is not defined behavior to set both direct and buffered io flags on DO->Flags.
d
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of A P
Sent: Friday, April 15, 2016 8:17 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] DO_BUFFERED_IO | DO_DIRECT_IO
Hello all,
can a device support both Buffered and direct IO? I want to have a device which supports both types and depending on the type of the IOCTL either method can be used?
m_pDeviceObject->Flags |= DO_DIRECT_IO;
m_pDeviceObject->Flags |= DO_BUFFERED_IO;
thanks
al.
— NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at To unsubscribe, visit the List Server section of OSR Online at
thanks doron.
So If I want the device to have both kind of IOCTLs what should the IRP
flag be?
should it be set to buffered or direct?
I tried setting it to direct and then using an ioctl with buffered io, and
the buffer came back as all junk, nothing was set by the io manager.
also in case of buffered io the irp (device ioctl in my case) will usually
have the buffer in the Irp’s sysmtebuffer pointer where as in case of
direct io it is the mdl.
if i set the irp flag to be direct io, and if the device io control is
buffered_io should i be looking at the mdl or the systembufferpointer for
the buffer?
On Fri, Apr 15, 2016 at 9:28 PM, Doron Holan
wrote:
> The IO values you set on DeviceObject->Flags control read and write irp
> buffer encoding. They do not control IOCTL buffer encoding. Each IOCTL
> value can specify how the input and output buffers are encoded. It is not
> defined behavior to set both direct and buffered io flags on DO->Flags.
>
>
>
> d
>
>
>
> From: xxxxx@lists.osr.com [mailto:
> xxxxx@lists.osr.com] *On Behalf Of *A P
> Sent: Friday, April 15, 2016 8:17 PM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] DO_BUFFERED_IO | DO_DIRECT_IO
>
>
>
> Hello all,
>
>
>
> can a device support both Buffered and direct IO? I want to have a device
> which supports both types and depending on the type of the IOCTL either
> method can be used?
>
>
>
>
>
> m_pDeviceObject->Flags |= DO_DIRECT_IO;
>
> m_pDeviceObject->Flags |= DO_BUFFERED_IO;
>
>
>
>
>
> thanks
>
>
>
> al.
>
>
>
>
>
> — NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars
> on crash dump analysis, WDF, Windows internals and software drivers!
> Details at To unsubscribe, visit the List Server section of OSR Online at
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer>
></http:>
For ioctl, it doesn’t matter what the device is set for, each ioctl controls things. When you did buffered, did you set the completion info size field, which tells the OS how many bytes to copy back to user space.
Jan
From: > on behalf of A P >
Reply-To: Windows List >
Date: Friday, April 15, 2016 at 10:41 PM
To: Windows List >
Subject: Re: [ntdev] DO_BUFFERED_IO | DO_DIRECT_IO
thanks doron.
So If I want the device to have both kind of IOCTLs what should the IRP flag be?
should it be set to buffered or direct?
I tried setting it to direct and then using an ioctl with buffered io, and the buffer came back as all junk, nothing was set by the io manager.
also in case of buffered io the irp (device ioctl in my case) will usually have the buffer in the Irp’s sysmtebuffer pointer where as in case of direct io it is the mdl.
if i set the irp flag to be direct io, and if the device io control is buffered_io should i be looking at the mdl or the systembufferpointer for the buffer?
On Fri, Apr 15, 2016 at 9:28 PM, Doron Holan > wrote:
The IO values you set on DeviceObject->Flags control read and write irp buffer encoding. They do not control IOCTL buffer encoding. Each IOCTL value can specify how the input and output buffers are encoded. It is not defined behavior to set both direct and buffered io flags on DO->Flags.
d
From:xxxxx@lists.osr.commailto:xxxxx [mailto:xxxxx@lists.osr.commailto:xxxxx] On Behalf Of A P
Sent: Friday, April 15, 2016 8:17 PM
To: Windows System Software Devs Interest List >
Subject: [ntdev] DO_BUFFERED_IO | DO_DIRECT_IO
Hello all,
can a device support both Buffered and direct IO? I want to have a device which supports both types and depending on the type of the IOCTL either method can be used?
m_pDeviceObject->Flags |= DO_DIRECT_IO;
m_pDeviceObject->Flags |= DO_BUFFERED_IO;
thanks
al.
— NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at To unsubscribe, visit the List Server section of OSR Online at
—
NTDEV is sponsored by OSR
Visit the list online at: http:
MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
Details at http:
To unsubscribe, visit the List Server section of OSR Online at http:
— NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at To unsubscribe, visit the List Server section of OSR Online at</http:></http:></http:></mailto:xxxxx></mailto:xxxxx>
It is all discussed here:
https://msdn.microsoft.com/en-us/library/windows/hardware/ff540663(v=vs.85).aspx
On Sat, Apr 16, 2016 at 1:47 AM Jan Bottorff wrote:
> For ioctl, it doesn’t matter what the device is set for, each ioctl
> controls things. When you did buffered, did you set the completion info
> size field, which tells the OS how many bytes to copy back to user space.
>
> Jan
>
> From: on behalf of A P <
> xxxxx@gmail.com>
> Reply-To: Windows List
> Date: Friday, April 15, 2016 at 10:41 PM
> To: Windows List
> Subject: Re: [ntdev] DO_BUFFERED_IO | DO_DIRECT_IO
>
> thanks doron.
>
> So If I want the device to have both kind of IOCTLs what should the IRP
> flag be?
> should it be set to buffered or direct?
>
> I tried setting it to direct and then using an ioctl with buffered io, and
> the buffer came back as all junk, nothing was set by the io manager.
>
> also in case of buffered io the irp (device ioctl in my case) will usually
> have the buffer in the Irp’s sysmtebuffer pointer where as in case of
> direct io it is the mdl.
> if i set the irp flag to be direct io, and if the device io control is
> buffered_io should i be looking at the mdl or the systembufferpointer for
> the buffer?
>
> On Fri, Apr 15, 2016 at 9:28 PM, Doron Holan
> wrote:
>
>> The IO values you set on DeviceObject->Flags control read and write irp
>> buffer encoding. They do not control IOCTL buffer encoding. Each IOCTL
>> value can specify how the input and output buffers are encoded. It is not
>> defined behavior to set both direct and buffered io flags on DO->Flags.
>>
>>
>>
>> d
>>
>>
>>
>> *From:*xxxxx@lists.osr.com [mailto:
>> xxxxx@lists.osr.com] *On Behalf Of *A P
>> Sent: Friday, April 15, 2016 8:17 PM
>> To: Windows System Software Devs Interest List
>> Subject: [ntdev] DO_BUFFERED_IO | DO_DIRECT_IO
>>
>>
>>
>> Hello all,
>>
>>
>>
>> can a device support both Buffered and direct IO? I want to have a device
>> which supports both types and depending on the type of the IOCTL either
>> method can be used?
>>
>>
>>
>>
>>
>> m_pDeviceObject->Flags |= DO_DIRECT_IO;
>>
>> m_pDeviceObject->Flags |= DO_BUFFERED_IO;
>>
>>
>>
>>
>>
>> thanks
>>
>>
>>
>> al.
>>
>>
>>
>>
>>
>> — NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars
>> on crash dump analysis, WDF, Windows internals and software drivers!
>> Details at To unsubscribe, visit the List Server section of OSR Online at
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> Visit the list online at: <
>> http://www.osronline.com/showlists.cfm?list=ntdev>
>>
>> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
>> software drivers!
>> Details at http:
>>
>> To unsubscribe, visit the List Server section of OSR Online at <
>> http://www.osronline.com/page.cfm?name=ListServer>
>>
>
> — NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars
> on crash dump analysis, WDF, Windows internals and software drivers!
> Details at To unsubscribe, visit the List Server section of OSR Online at
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer>
></http:></http:>
How do you think IO Manager is supposed to deal with the whole thing when setting up IRPs for your device if it encounters a “masterpiece” like that??? How is it supposed to know which type of IO is needed in any particular situation??? This would certainly lead to undefined and unpredictable behavior if things worked this way. This is why the IO Manager thinks of IOCTL as of a special case, and defines certain rules for IOCTL codes.
https://msdn.microsoft.com/en-us/library/windows/hardware/ff543023(v=vs.85).aspx
Therefore, IO Manager does not have to look at device flags when setting up IOCTL-related IRPs - intead, it looks at “transfer type” flags of a given IOCTL code. This approach provides a truly flexible mechanism (shit, I refuse to believe that I am typing this - after all, I am speaking about flexibility in context of discussing Windows) that allows you to define your own API for your device, and build it on top of a single entry exactly he same way the OS itself builds its entire system API on top if a single kernel entry…
Anton Bassov
The DeviceObject->Flags field applies only to IRP_MJ_READ and IRP_MJ_WRITE. The buffer method argument on the CTL_CODE macro controls the buffer type,of each individual IOCTL and has no dependence on hats set in DeviceObject-> Flags.
If you’re trying to use buffered I/O for an IOCTL and it’s not working, the usual cause is that you’re not setting IRP->IoStatus.Information to a valid value before calling IoCompleteRequest.
By the way, if you set both DO_BUFFERED_IO and DO_DIRECT_IO in Flags, your read and write operations use Buffered I/O. As Doron said, this is not architecturally defined (and makes no sense, logically) but has been the way the I/O Manager works forever.
Peter
OSR
@OSRDrivers
>and depending on the type of the IOCTL either method can be used?
IOCTLs do not use flags from the device object, they use flags from the IOCTL code itself instead.
–
Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com
thank you all for all your answers. I get it now. I have one further
question in this topic:
this is strictly about IOCTLS…
if it is BUFFERED_IO the IO manager will copy the buffer and pass the
driver a pointer
if it is DIRECT_IO, the nthe MDL can be used by the driver to map the
address in kernel space using MmGetSystemAddressForMdlSafe.
With verifier’s low resources, MmGetSystemAddressForMdlSafe can fail if the
priority is NormalPagePriority or lower. HighPriority has a higher
probability of success but not guaranteed.
What happens with verifier low resources in the case of BUFFERED_IO where
the IO/M copies? Can that fail under low resource simulation as well?
On Sun, Apr 17, 2016 at 4:16 AM, wrote:
>
>
>
> How do you think IO Manager is supposed to deal with the whole thing when
> setting up IRPs for your device if it encounters a “masterpiece” like
> that??? How is it supposed to know which type of IO is needed in any
> particular situation??? This would certainly lead to undefined and
> unpredictable behavior if things worked this way. This is why the IO
> Manager thinks of IOCTL as of a special case, and defines certain rules for
> IOCTL codes.
>
>
>
> https://msdn.microsoft.com/en-us/library/windows/hardware/ff543023(v=vs.85).aspx
>
>
> Therefore, IO Manager does not have to look at device flags when setting
> up IOCTL-related IRPs - intead, it looks at “transfer type” flags of a
> given IOCTL code. This approach provides a truly flexible mechanism (shit,
> I refuse to believe that I am typing this - after all, I am speaking about
> flexibility in context of discussing Windows) that allows you to define
> your own API for your device, and build it on top of a single entry exactly
> he same way the OS itself builds its entire system API on top if a single
> kernel entry…
>
>
> Anton Bassov
>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer>
></http:>
It can fail for buffered io as well. It will fail in the io manager and your driver will never see the request. Note that under extreme conditions the allocation of the PIRP itself can fail no matter the buffering method.
d
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of A P
Sent: Monday, April 18, 2016 12:40 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] DO_BUFFERED_IO | DO_DIRECT_IO
thank you all for all your answers. I get it now. I have one further question in this topic:
this is strictly about IOCTLS…
if it is BUFFERED_IO the IO manager will copy the buffer and pass the driver a pointer
if it is DIRECT_IO, the nthe MDL can be used by the driver to map the address in kernel space using MmGetSystemAddressForMdlSafe.
With verifier’s low resources, MmGetSystemAddressForMdlSafe can fail if the priority is NormalPagePriority or lower. HighPriority has a higher probability of success but not guaranteed.
What happens with verifier low resources in the case of BUFFERED_IO where the IO/M copies? Can that fail under low resource simulation as well?
On Sun, Apr 17, 2016 at 4:16 AM, > wrote:
How do you think IO Manager is supposed to deal with the whole thing when setting up IRPs for your device if it encounters a “masterpiece” like that??? How is it supposed to know which type of IO is needed in any particular situation??? This would certainly lead to undefined and unpredictable behavior if things worked this way. This is why the IO Manager thinks of IOCTL as of a special case, and defines certain rules for IOCTL codes.
https://msdn.microsoft.com/en-us/library/windows/hardware/ff543023(v=vs.85).aspx
Therefore, IO Manager does not have to look at device flags when setting up IOCTL-related IRPs - intead, it looks at “transfer type” flags of a given IOCTL code. This approach provides a truly flexible mechanism (shit, I refuse to believe that I am typing this - after all, I am speaking about flexibility in context of discussing Windows) that allows you to define your own API for your device, and build it on top of a single entry exactly he same way the OS itself builds its entire system API on top if a single kernel entry…
Anton Bassov
—
NTDEV is sponsored by OSR
Visit the list online at: http:
MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers!
Details at http:
To unsubscribe, visit the List Server section of OSR Online at http:
— NTDEV is sponsored by OSR Visit the list online at: MONTHLY seminars on crash dump analysis, WDF, Windows internals and software drivers! Details at To unsubscribe, visit the List Server section of OSR Online at</http:></http:></http:>
A P wrote:
this is strictly about IOCTLS…
if it is BUFFERED_IO the IO manager will copy the buffer and pass the
driver a pointer
if it is DIRECT_IO, the nthe MDL can be used by the driver to map the
address in kernel space using MmGetSystemAddressForMdlSafe.With verifier’s low resources, MmGetSystemAddressForMdlSafe can fail
if the priority is NormalPagePriority or lower. HighPriority has a
higher probability of success but not guaranteed.What happens with verifier low resources in the case of BUFFERED_IO
where the IO/M copies? Can that fail under low resource simulation as
well?
Let me ask you an honest question. Why do you care?
I’ll tell you why I ask. A few people, in order to pass the “low
resource simulation”, got into the habit of retrying everything a couple
of times. That does get you past the simulation, but it completely
misses the point. The simulation is designed to ensure that your driver
fails gracefully when the chips are down, because if there is a REAL low
resource condition, it’s probably not going to clean itself up after 3
retries. The system needs to be able to shut things down without having
a driver muck things up.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
thanks doron, tim.
Tim,
to answer your question why I care…
low resources in real world can happen because of systems going really low
on resources perhaps because of an unrelated component which leaks memory.
verifier’s simulations will also try to see how we behave in such
conditions. but verifier will try to fail allocation which this driver
under test makes. So having buffered_io, in my opinoin…and plese correct
me if i am wrong, will not trigger it at all, as the io manager’s
allocations are not charged to the driver under tst.
so under test conditions, I can actually focus more on weeding out
allocations really required in my driver nad how to gracefully bail out of
those.
In the event IO/M runs out of memory I think all bets are off at that
time…again, in my humble opinion.
so to me it basically drills down to avoid having to add failure paths and
handlers, where ever I can…
On Mon, Apr 18, 2016 at 1:57 PM, Tim Roberts wrote:
> A P wrote:
> >
> > this is strictly about IOCTLS…
> >
> > if it is BUFFERED_IO the IO manager will copy the buffer and pass the
> > driver a pointer
> > if it is DIRECT_IO, the nthe MDL can be used by the driver to map the
> > address in kernel space using MmGetSystemAddressForMdlSafe.
> >
> > With verifier’s low resources, MmGetSystemAddressForMdlSafe can fail
> > if the priority is NormalPagePriority or lower. HighPriority has a
> > higher probability of success but not guaranteed.
> >
> > What happens with verifier low resources in the case of BUFFERED_IO
> > where the IO/M copies? Can that fail under low resource simulation as
> > well?
>
> Let me ask you an honest question. Why do you care?
>
> I’ll tell you why I ask. A few people, in order to pass the “low
> resource simulation”, got into the habit of retrying everything a couple
> of times. That does get you past the simulation, but it completely
> misses the point. The simulation is designed to ensure that your driver
> fails gracefully when the chips are down, because if there is a REAL low
> resource condition, it’s probably not going to clean itself up after 3
> retries. The system needs to be able to shut things down without having
> a driver muck things up.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: <
> http://www.osronline.com/showlists.cfm?list=ntdev>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer>
></http:>