There is no any violation. I/O manager doesn’t check on IRP completion for
this fact, it just uses IRP_BUFFERED_IO and IRP_DEALLOCATE_BUFFER flags to
decide what to do with system buffer.
The matter is the SRV2 doesn’t use IoCompleteRequest to complete IRP and do
all cleanup by itself.
2016-10-28 21:02 GMT+03:00 Tony Mason :
>
> > “Setting up a device object Flags field with DO_DIRECT_IO or > DO_BUFFERED_IO determines how the I/O manager passes access to user buffers > in all data transfer requests subsequently sent to the driver.” > > https://msdn.microsoft.com/en-us/library/windows/hardware/ > ff547807%28v=vs.85%29.aspx > >
> > Interesting. Well that helps support your theory that this is a bug > then. Like many things in this area, sometimes things “work” even if they > were beyond the scope of what was intended. > > Even if you create a bug report with Microsoft, it won’t fix your current > problem. Thus, you may need to find another way to track these buffers. > For example, you could keep a list of such buffers associated with the file > and when you see cleanup free those buffers. It’s not a perfect solution, > but it’s pragmatic - it works around this issue. > > Tony > OSR > > > — > NTFSD is sponsored by OSR > > > MONTHLY seminars on crash dump analysis, WDF, Windows internals and > software drivers! > Details at http: > > To unsubscribe, visit the List Server section of OSR Online at < > http://www.osronline.com/page.cfm?name=ListServer> ></http:>
These flags are used by I/O manager to decide what to do with user buffer,
lock it or copy into separate buffer, on IRP creating time.
2016-10-28 21:07 GMT+03:00 Anatoly Mikhailov :
>
> > There is no any violation. I/O manager doesn’t check on IRP completion for > this fact, it just uses IRP_BUFFERED_IO and IRP_DEALLOCATE_BUFFER flags > to decide what to do with system buffer. > The matter is the SRV2 doesn’t use IoCompleteRequest to complete IRP and > do all cleanup by itself. > > 2016-10-28 21:02 GMT+03:00 Tony Mason : > >>
>> >> “Setting up a device object Flags field with DO_DIRECT_IO or >> DO_BUFFERED_IO determines how the I/O manager passes access to user buffers >> in all data transfer requests subsequently sent to the driver.” >> >> https://msdn.microsoft.com/en-us/library/windows/hardware/ff >> 547807%28v=vs.85%29.aspx >> >>
>> >> Interesting. Well that helps support your theory that this is a bug >> then. Like many things in this area, sometimes things “work” even if they >> were beyond the scope of what was intended. >> >> Even if you create a bug report with Microsoft, it won’t fix your current >> problem. Thus, you may need to find another way to track these buffers. >> For example, you could keep a list of such buffers associated with the file >> and when you see cleanup free those buffers. It’s not a perfect solution, >> but it’s pragmatic - it works around this issue. >> >> Tony >> OSR >> >> >> — >> NTFSD is sponsored by OSR >> >> >> MONTHLY seminars on crash dump analysis, WDF, Windows internals and >> software drivers! >> Details at http: >> >> To unsubscribe, visit the List Server section of OSR Online at < >> http://www.osronline.com/page.cfm?name=ListServer> >> > ></http:>
> > These flags are used by I/O manager to decide what to do with user buffer, > lock it or copy into separate buffer, on IRP creating time. > > 2016-10-28 21:07 GMT+03:00 Anatoly Mikhailov : > >>
>> >> There is no any violation. I/O manager doesn’t check on IRP completion >> for this fact, it just uses IRP_BUFFERED_IO and IRP_DEALLOCATE_BUFFER >> flags to decide what to do with system buffer. >> The matter is the SRV2 doesn’t use IoCompleteRequest to complete IRP and >> do all cleanup by itself. >> >> 2016-10-28 21:02 GMT+03:00 Tony Mason : >> >>>
>>> >>> “Setting up a device object Flags field with DO_DIRECT_IO or >>> DO_BUFFERED_IO determines how the I/O manager passes access to user buffers >>> in all data transfer requests subsequently sent to the driver.” >>> >>> https://msdn.microsoft.com/en-us/library/windows/hardware/ff >>> 547807%28v=vs.85%29.aspx >>> >>>
>>> >>> Interesting. Well that helps support your theory that this is a bug >>> then. Like many things in this area, sometimes things “work” even if they >>> were beyond the scope of what was intended. >>> >>> Even if you create a bug report with Microsoft, it won’t fix your >>> current problem. Thus, you may need to find another way to track these >>> buffers. For example, you could keep a list of such buffers associated >>> with the file and when you see cleanup free those buffers. It’s not a >>> perfect solution, but it’s pragmatic - it works around this issue. >>> >>> Tony >>> OSR >>> >>> >>> — >>> NTFSD is sponsored by OSR >>> >>> >>> MONTHLY seminars on crash dump analysis, WDF, Windows internals and >>> software drivers! >>> Details at http: >>> >>> To unsubscribe, visit the List Server section of OSR Online at < >>> http://www.osronline.com/page.cfm?name=ListServer> >>> >> >> ></http:>
This documentation is what someone implementing this on their own would rely upon. If there were some formal description of how this should behave, it would make life easier. This is the closest thing that I could find to support this view of the world.
The pragmatic answer is: SRV2 is part of Windows and thus gets to define correct behavior. If it does not work on a third party driver, the third party driver writer should fix the problem. It *is* possible to get Microsoft to change behavior, but you must be able to make a clear argument on why it is their bug.
Tony Mason wrote:
<<
“Setting up a device object Flags field with DO_DIRECT_IO or DO_BUFFERED_IO=
determines how the I/O manager passes access to user buffers in all data t=
ransfer requests subsequently sent to the driver.”
>
I see this, but cannot readily translate it into “you cannot allocate and use your own SystemBuffer”.
<<
Interesting. Well that helps support your theory that this is a bug then. =
Like many things in this area, sometimes things “work” even if they were b=
eyond the scope of what was intended.
Even if you create a bug report with Microsoft, it won’t fix your current p=
roblem…
>
I agree. One has to be pragmatic in these cases. I only use my version of FatBufferUserBuffer in IRP_MJ_QUERY_SECURITY and IRP_MJ_DIRECTORY_CONTROL/IRP_MN_QUERY_DIRECTORY. It may be worth eliminating these usages to be on the safe side, despite the short term pain this will cause me.
I am now also thinking that my IRP_MN_QUERY_DIRECTORY processing may be hostile to filters which, following the FastFat example, do not expect to see anything in the SystemBuffer.
Windows kernel reversing and Windows Research Kernel let me understand that
this behaves such way not ambiguous. This is not such kind of things that
could change in future.
For my experience it’s senseless. Even this would be considered as a bug
you’ll have to support existing versions of SRV2 any way.
Besides FatBufferUserBuffer routine, FsRtlNotifyCompleteIrp kernel routine
does the same. It’s not a bug, windows kernel developers designed things
such way.
2016-10-28 21:32 GMT+03:00 :
> Tony Mason wrote: > << > “Setting up a device object Flags field with DO_DIRECT_IO or > DO_BUFFERED_IO= > determines how the I/O manager passes access to user buffers in all data > t= > ransfer requests subsequently sent to the driver.” > >> > > I see this, but cannot readily translate it into “you cannot allocate and > use your own SystemBuffer”. > > << >
> > Interesting. Well that helps support your theory that this is a bug then. > = > Like many things in this area, sometimes things “work” even if they were > b= > eyond the scope of what was intended. > > Even if you create a bug report with Microsoft, it won’t fix your current > p= > roblem… > >> > > I agree. One has to be pragmatic in these cases. I only use my version of > FatBufferUserBuffer in IRP_MJ_QUERY_SECURITY and > IRP_MJ_DIRECTORY_CONTROL/IRP_MN_QUERY_DIRECTORY. It may be worth > eliminating these usages to be on the safe side, despite the short term > pain this will cause me. > > I am now also thinking that my IRP_MN_QUERY_DIRECTORY processing may be > hostile to filters which, following the FastFat example, do not expect to > see anything in the SystemBuffer. > > Bill > > > — > NTFSD is sponsored by OSR > > > MONTHLY seminars on crash dump analysis, WDF, Windows internals and > software drivers! > Details at http: > > To unsubscribe, visit the List Server section of OSR Online at < > http://www.osronline.com/page.cfm?name=ListServer> ></http:>
You can, if you sure that request would be completed by I/O manager with
IoCompleteRequest routine.
2016-10-28 21:47 GMT+03:00 Anatoly Mikhailov :
>
> > Windows kernel reversing and Windows Research Kernel let me understand > that this behaves such way not ambiguous. This is not such kind of things > that could change in future. > > >
> > For my experience it’s senseless. Even this would be considered as a bug > you’ll have to support existing versions of SRV2 any way. > > > >
> > > Besides FatBufferUserBuffer routine, FsRtlNotifyCompleteIrp kernel routine > does the same. It’s not a bug, windows kernel developers designed things > such way. > > 2016-10-28 21:32 GMT+03:00 : > >> Tony Mason wrote: >> << >> “Setting up a device object Flags field with DO_DIRECT_IO or >> DO_BUFFERED_IO= >> determines how the I/O manager passes access to user buffers in all data >> t= >> ransfer requests subsequently sent to the driver.” >> >> >> >> I see this, but cannot readily translate it into “you cannot allocate and >> use your own SystemBuffer”. >> >> << >>
>> >> Interesting. Well that helps support your theory that this is a bug >> then. = >> Like many things in this area, sometimes things “work” even if they were >> b= >> eyond the scope of what was intended. >> >> Even if you create a bug report with Microsoft, it won’t fix your current >> p= >> roblem… >> >> >> >> I agree. One has to be pragmatic in these cases. I only use my version of >> FatBufferUserBuffer in IRP_MJ_QUERY_SECURITY and >> IRP_MJ_DIRECTORY_CONTROL/IRP_MN_QUERY_DIRECTORY. It may be worth >> eliminating these usages to be on the safe side, despite the short term >> pain this will cause me. >> >> I am now also thinking that my IRP_MN_QUERY_DIRECTORY processing may be >> hostile to filters which, following the FastFat example, do not expect to >> see anything in the SystemBuffer. >> >> Bill >> >> >> — >> NTFSD is sponsored by OSR >> >> >> MONTHLY seminars on crash dump analysis, WDF, Windows internals and >> software drivers! >> Details at http: >> >> To unsubscribe, visit the List Server section of OSR Online at < >> http://www.osronline.com/page.cfm?name=ListServer> >> > ></http:>
Bill, why you can’t map Irp->MdlAddress in your IRP handler?
2016-10-28 21:51 GMT+03:00 Anatoly Mikhailov :
>
> > You can, if you sure that request would be completed by I/O manager with > IoCompleteRequest routine. > > 2016-10-28 21:47 GMT+03:00 Anatoly Mikhailov : > >>
>> >> Windows kernel reversing and Windows Research Kernel let me understand >> that this behaves such way not ambiguous. This is not such kind of things >> that could change in future. >> >> >>
>> >> For my experience it’s senseless. Even this would be considered as a bug >> you’ll have to support existing versions of SRV2 any way. >> >> >> >>
>> >> >> Besides FatBufferUserBuffer routine, FsRtlNotifyCompleteIrp kernel >> routine does the same. It’s not a bug, windows kernel developers designed >> things such way. >> >> 2016-10-28 21:32 GMT+03:00 : >> >>> Tony Mason wrote: >>> << >>> “Setting up a device object Flags field with DO_DIRECT_IO or >>> DO_BUFFERED_IO= >>> determines how the I/O manager passes access to user buffers in all >>> data t= >>> ransfer requests subsequently sent to the driver.” >>> >> >>> >>> I see this, but cannot readily translate it into “you cannot allocate >>> and use your own SystemBuffer”. >>> >>> << >>>
>>> >>> Interesting. Well that helps support your theory that this is a bug >>> then. = >>> Like many things in this area, sometimes things “work” even if they >>> were b= >>> eyond the scope of what was intended. >>> >>> Even if you create a bug report with Microsoft, it won’t fix your >>> current p= >>> roblem… >>> >> >>> >>> I agree. One has to be pragmatic in these cases. I only use my version >>> of FatBufferUserBuffer in IRP_MJ_QUERY_SECURITY and >>> IRP_MJ_DIRECTORY_CONTROL/IRP_MN_QUERY_DIRECTORY. It may be worth >>> eliminating these usages to be on the safe side, despite the short term >>> pain this will cause me. >>> >>> I am now also thinking that my IRP_MN_QUERY_DIRECTORY processing may be >>> hostile to filters which, following the FastFat example, do not expect to >>> see anything in the SystemBuffer. >>> >>> Bill >>> >>> >>> — >>> NTFSD is sponsored by OSR >>> >>> >>> MONTHLY seminars on crash dump analysis, WDF, Windows internals and >>> software drivers! >>> Details at http: >>> >>> To unsubscribe, visit the List Server section of OSR Online at < >>> http://www.osronline.com/page.cfm?name=ListServer> >>> >> >> ></http:>
For my experience it’s senseless. Even this would be considered as a bug
you’ll have to support existing versions of SRV2 any way.
>
I agree with both of you. Regardless of whether it is a bug or not, I must make it work with existing versions of SRV2.
<<
Besides FatBufferUserBuffer routine, FsRtlNotifyCompleteIrp kernel routine
does the same. It’s not a bug, windows kernel developers designed things
such way.
>
Thanks for the confirmation.
The original question remains: are you aware of any other such SRV2 gotchas?
Bill, why you can’t map Irp->MdlAddress in your IRP handler?
I wrote this code about 6 months ago and I do not remember all the reasons. I remember that my original implementation mapped Irp->MdlAddress (like FastFat) and then I changed it when I hit some snag specific to my FSD.
I am now looking into this again and attempting to fix it the “proper way” (i.e. map the Irp->MdlAddress). I should have a report soon.
I have a rule for myself, if i’m writing FSD i process requests like the
existing FSDs does. The matter is all windows components
work with this implementation well. So if you would violate this rule you
can’t be sure your FSD would be completely compatible
with rest system.
I think the MdlAddress mapping would fix your issue.
2016-10-28 22:06 GMT+03:00 :
> Anatoly: > > > Bill, why you can’t map Irp->MdlAddress in your IRP handler? > > I wrote this code about 6 months ago and I do not remember all the > reasons. I remember that my original implementation mapped Irp->MdlAddress > (like FastFat) and then I changed it when I hit some snag specific to my > FSD. > > I am now looking into this again and attempting to fix it the “proper way” > (i.e. map the Irp->MdlAddress). I should have a report soon. > > Bill > > > — > NTFSD is sponsored by OSR > > > MONTHLY seminars on crash dump analysis, WDF, Windows internals and > software drivers! > Details at http: > > To unsubscribe, visit the List Server section of OSR Online at < > http://www.osronline.com/page.cfm?name=ListServer> ></http:>
Bill, why you can’t map Irp->MdlAddress in your IRP handler?
Actually looking into this code more closely I seem to recall now.
My FSD acts as a “proxy” for user mode file systems. It translates kernel mode IRP’s to user mode “Requests” and vice-versa (hence my need to pend IRP’s in many cases). To avoid multiple round-trips to user mode it caches the responses received by the user mode file system.
When I originally wrote the IRP_MN_QUERY_DIRECTORY handler, the way I decided to do it was to allocate a SystemBuffer with a large size regardless of the size requested by the process doing the NtQueryDirectoryFile. Then I would make the SystemBuffer available to the user mode file system, cache the response and then return only the requested portion of the SystemBuffer to the system (by properly setting the Irp->IoStatus.Information).
This way if I received a short IRP_MN_QUERY_DIRECTORY buffer (say 4096 bytes) I could still send a larger buffer to the user mode file system (say 64K) and get and cache more results, so that subsequent IRP_MN_QUERY_DIRECTORY requests could be satisfied from the cache.
> so that subsequent IRP_MN_QUERY_DIRECTORY requests could be satisfied
from the cache
I’m not sure i undestood you right.
Did you set Irp->SystemBuffer with address of your allocated cache that was
passed to user mode file system?
2016-10-28 22:32 GMT+03:00 :
> Anatoly: > > > Bill, why you can’t map Irp->MdlAddress in your IRP handler? > > Actually looking into this code more closely I seem to recall now. > > My FSD acts as a “proxy” for user mode file systems. It translates kernel > mode IRP’s to user mode “Requests” and vice-versa (hence my need to pend > IRP’s in many cases). To avoid multiple round-trips to user mode it caches > the responses received by the user mode file system. > > When I originally wrote the IRP_MN_QUERY_DIRECTORY handler, the way I > decided to do it was to allocate a SystemBuffer with a large size > regardless of the size requested by the process doing the > NtQueryDirectoryFile. Then I would make the SystemBuffer available to the > user mode file system, cache the response and then return only the > requested portion of the SystemBuffer to the system (by properly setting > the Irp->IoStatus.Information). > > This way if I received a short IRP_MN_QUERY_DIRECTORY buffer (say 4096 > bytes) I could still send a larger buffer to the user mode file system (say > 64K) and get and cache more results, so that subsequent > IRP_MN_QUERY_DIRECTORY requests could be satisfied from the cache. > > Bill > > > — > NTFSD is sponsored by OSR > > > MONTHLY seminars on crash dump analysis, WDF, Windows internals and > software drivers! > Details at http: > > To unsubscribe, visit the List Server section of OSR Online at < > http://www.osronline.com/page.cfm?name=ListServer> ></http:>
> so that subsequent IRP_MN_QUERY_DIRECTORY requests could be satisfied
from the cache
I’m not sure i undestood you right.
Did you set Irp->SystemBuffer with address of your allocated cache that was
passed to user mode file system?
Good catch. That would of course be wrong.
The SystemBuffer is used only temporarily (and only when necessary – there is no need for a SystemBuffer when the IRP_MN_QUERY_DIRECTORY request can be satisfied from the cache). Here is an outline of the sequence of events.
A SystemBuffer of large size gets allocated and set into the IRP along with the appropriate flags.
The SystemBuffer gets locked for access from the user mode file system.
When the user mode file system responds the SystemBuffer gets *copied* into the cache. Then the portion that was originally requested (IrpSp->Parameters.QueryDirectory.Length) gets set into the Irp->IoStatus.Information.
The SystemBuffer gets unmapped from the user mode file system address space just prior to IRP completion.
I just looked at your code. You have wrote a whole framework
I payed my attention to you reset IRP_INPUT_OPERATION, IRP_BUFFERED_IO and
IRP_DEALLOCATE_BUFFER flags explicitly before IRP completing as well as
Irp->SystemBuffer.
So things are getting more interesting since SRV2 wouldn’t ever see these
flags and Irp->SystemBuffer are set.
2016-10-28 23:06 GMT+03:00 :
> Anatoly wrote: > > >> so that subsequent IRP_MN_QUERY_DIRECTORY requests could be satisfied > from the cache > > I’m not sure i undestood you right. > > Did you set Irp->SystemBuffer with address of your allocated cache that > was > > passed to user mode file system? > > Good catch. That would of course be wrong. > > The SystemBuffer is used only temporarily (and only when necessary – > there is no need for a SystemBuffer when the IRP_MN_QUERY_DIRECTORY request > can be satisfied from the cache). Here is an outline of the sequence of > events. > > 1. A SystemBuffer of large size gets allocated and set into the IRP along > with the appropriate flags. > > 2. The SystemBuffer gets locked for access from the user mode file system. > > 3. When the user mode file system responds the SystemBuffer gets copied > into the cache. Then the portion that was originally requested > (IrpSp->Parameters.QueryDirectory.Length) gets set into the > Irp->IoStatus.Information. > > 4. The SystemBuffer gets unmapped from the user mode file system address > space just prior to IRP completion. > > If you are really curious, the source code is online: > https://github.com/billziss-gh/winfsp/blob/master/src/sys/dirctl.c > > The relevant routines are: 1. FspFsvolQueryDirectoryBufferUserBuffer, 2. > FspFsvolDirectoryControlPrepare, 3. FspFsvolDirectoryControlComplete, 4. > FspFsvolQueryDirectoryRequestFini > > Bill > > > — > NTFSD is sponsored by OSR > > > MONTHLY seminars on crash dump analysis, WDF, Windows internals and > software drivers! > Details at http: > > To unsubscribe, visit the List Server section of OSR Online at < > http://www.osronline.com/page.cfm?name=ListServer> ></http:>
I just looked at your code. You have wrote a whole framework
Yes
<<
I payed my attention to you reset IRP_INPUT_OPERATION, IRP_BUFFERED_IO and
IRP_DEALLOCATE_BUFFER flags explicitly before IRP completing as well as
Irp->SystemBuffer.
So things are getting more interesting since SRV2 wouldn’t ever see these
flags and Irp->SystemBuffer are set.
>
Actually resetting the IRP_INPUT_OPERATION, IRP_BUFFERED_IO and
IRP_DEALLOCATE_BUFFER flags and deallocating the SystemBuffer myself, is my fix for the issue we are discussing. With this fix SRV2 queries work. Without this fix SRV2 queries fail.
> Actually resetting the IRP_INPUT_OPERATION, IRP_BUFFERED_IO and
IRP_DEALLOCATE_BUFFER flags and deallocating the SystemBuffer myself, is
my fix for the issue we are discussing. With this fix SRV2 queries work.
Without this fix SRV2 queries fail.
Seems like i misunderstood you. I thought you haven’t this fix yet.
You use IRP context. I would place our allocated SystemBuffer there. This
simplifies implementation. In this case you don’t have to set and reset the
flags and Irp->SystemBuffer.
2016-10-29 0:11 GMT+03:00 Anatoly Mikhailov :
> > Actually resetting the IRP_INPUT_OPERATION, IRP_BUFFERED_IO and > > IRP_DEALLOCATE_BUFFER flags and deallocating the SystemBuffer myself, is > my fix for the issue we are discussing. With this fix SRV2 queries work. > Without this fix SRV2 queries fail. > > Seems like i misunderstood you. I thought you haven’t this fix yet. >
> Seems like i misunderstood you. I thought you haven’t this fix yet.
I have indeed fixed this issue, but I am not particularly happy with my fix. It has also made me question what else is different under SRV2, which triggered me to create this thread.
You use IRP context. I would place our allocated SystemBuffer there. This
simplifies implementation. In this case you don’t have to set and reset the
flags and Irp->SystemBuffer.
Yes, I am contemplating a “better” fix along those lines.
> I have indeed fixed this issue, but I am not particularly happy with my
fix. It has also made me question what else is different under SRV2, which
triggered me to create this thread.
Just study how FastFat have a deal with IRP structure and do the same. That
will make sure you get your FSD behavior right.
2016-10-29 0:24 GMT+03:00 :
> > Seems like i misunderstood you. I thought you haven’t this fix yet. > > I have indeed fixed this issue, but I am not particularly happy with my > fix. It has also made me question what else is different under SRV2, which > triggered me to create this thread. > > > You use IRP context. I would place our allocated SystemBuffer there. This > simplifies implementation. In this case you don’t have to set and reset the > flags and Irp->SystemBuffer. > > Yes, I am contemplating a “better” fix along those lines. > > Bill > > > — > NTFSD is sponsored by OSR > > > MONTHLY seminars on crash dump analysis, WDF, Windows internals and > software drivers! > Details at http: > > To unsubscribe, visit the List Server section of OSR Online at < > http://www.osronline.com/page.cfm?name=ListServer> ></http:>
