I am having a couple of issues all in the read path of my isolation minifilter. My environment is all Win7 x64.
First, if I open an existing file with notepad all of the transforms works properly and the data is exactly as it should be. The read request I am seeing is PagingIo = TRUE, NonCached = TRUE, SyncIO = TRUE. If I close that file then open it again in a hex editor, I get another read request but this time PagingIO = FALSE. Just like FastFat I check the following [if (!PagingIo && NonCachedIo && (FileObject->SectionObjectPointer->DataSectionObject != NULL))] and attempt to perform the CcFlushCache call. It bugchecks however with DRIVER_PAGE_FAULT_BEYOND_END_OF_ALLOCATION. What I am having trouble understanding is my code to initialize the cache was never called when the data was read the first time by notepad so how is the file now in the cache? If I own the two file objects, how is DataSectionObject being initialized without me calling CcInitializeCacheMap?
Similiarly, if I do not open the file first with notepad and go straight to the hex editor, I get two read requests. The first one has PagingIo = FALSE, NonCache = TRUE, SyncIo = TRUE. It reads the data and does the transforms properly (which I can verify with DbgBreak) however another read comes in right after with the same flags and read length size but because the byte offsets are now at the end of the file I return STATUS_END_OF_FILE in PreRead. When this returns, the hex editor is just filed with zeros but of the proper length. So the first read results seem to be discarded then the empty buffer from the second end of file read is used.
Wordpad is also giving me problems. It doesn’t even get to the read path. It simply says it cannot open the file but all of the underlying requests succeed so I’m not sure how it could be a permissions related if I’m not getting any error codes.
> What I am having trouble understanding is my code to initialize the cache
was never
called when the data was read the first time by notepad so how is the file
now in the
cache? If I own the two file objects, how is DataSectionObject being
initialized without
me calling CcInitializeCacheMap?
Notepad maps a section, so there are no cached reads, just pagefaults.
For the rest I’d suggest checking that you are calling CcSetLegnth
correctly, I’m not sure that that’s it (I’ve never seen these symptoms), but
thats where I’d start
Thanks Rod. Turns out it was all related to how I was modifying some of
the create flags on the lower file object.
What I am now trying to get to work properly is writes that are greater
than a single page. For whatever reason, my writes work if the data size
is less than a page but when it is greater than that the PostWrite routine
returns STATUS_SUCCESS but with with 0 in the Information field. Based an
earlier question that I had about writing that Tony answered, leads me to
think I am still not setting all of the EOF/Alloc/VDL/CBO values correctly.
What I’ve noticed on my debug output when using notepad between a write
that actually writes bytes to a file and one that doesn’t is that the
successful one has a FASTIO call to NetQueryOpen right before
CcAcquireForLazyWrite. I don’t know if that actually makes a difference or
not but it’s the only thing I am seeing.
So just to verify that I am setting the EOF/Alloc/VDL/CBO values in the
correct place, in my non-paging, non-cached write path is where I am
setting these values for the upper file object as well as extending the EOF
of the lower object if necessary. I also have some logic in the
SetInformation path but I am not differentiating between paging and cached
IO. Is that correct or am I missing something?
On Mon, Oct 6, 2014 at 2:57 AM, Rod Widdowson wrote:
> What I am having trouble understanding is my code to initialize the cache >> was never >> called when the data was read the first time by notepad so how is the >> file now in the >> cache? If I own the two file objects, how is DataSectionObject being >> initialized without >> me calling CcInitializeCacheMap? >> > > Notepad maps a section, so there are no cached reads, just pagefaults. > > For the rest I’d suggest checking that you are calling CcSetLegnth > correctly, I’m not sure that that’s it (I’ve never seen these symptoms), > but thats where I’d start > > /Rod > > > > > — > NTFSD is sponsored by OSR > > OSR is hiring!! Info at http://www.osr.com/careers > > For our schedule of debugging and file system seminars visit: > http://www.osr.com/seminars > > To unsubscribe, visit the List Server section of OSR Online at > http://www.osronline.com/page.cfm?name=ListServer >
An IRP_MJ_WRITE/IRP_PAGING_IO that writes beyond EOF will return STATUS_SUCCESS indicating 0 bytes were written. If you haven’t tried this with FAT, I urge you to do so - you can rebuild it from sources and that permits you to trace through the code and see exactly this sequence.
This is tied up into the way in which FAT detects that the caller is allowed to extend the VDL. We’ve found that an isolation filter can interfere with the anticipated behavior here. You can actually set that VDL value, but FAT requires that the file must have been opened with SE_MANAGE_VOLUME privilege (it sets it in the CCB). You can certainly do that, but then you have to verify that your caller has SE_MANAGE_VOLUME privilege for every operation they invoke that goes through your isolation filter.
NTFS has similar behavior but different techniques for implementing some of the logic (e.g., it detects the lazy writer thread differently).
I added a VDL extension call after the EOF extension call and it now
works. What was the difference between a single page and a multiple page
write that made the multiple page write fail?
I am concerned though about the security implications that you and the
documents allude to. So is the best way to verify is to check the original
permissions during CREATE and if the permission is set then set a flag in a
context? Then if a VDL extension comes in without the flag set, deny it?
Or is there another way during each operation to query if it has the
permission set?
On Tue, Oct 7, 2014 at 6:16 PM, Tony Mason wrote:
>
> > You are running into the valid data length issue. > > An IRP_MJ_WRITE/IRP_PAGING_IO that writes beyond EOF will return > STATUS_SUCCESS indicating 0 bytes were written. If you haven’t tried this > with FAT, I urge you to do so - you can rebuild it from sources and that > permits you to trace through the code and see exactly this sequence. > > This is tied up into the way in which FAT detects that the caller is > allowed to extend the VDL. We’ve found that an isolation filter can > interfere with the anticipated behavior here. You can actually set that > VDL value, but FAT requires that the file must have been opened with > SE_MANAGE_VOLUME privilege (it sets it in the CCB). You can certainly do > that, but then you have to verify that your caller has SE_MANAGE_VOLUME > privilege for every operation they invoke that goes through your isolation > filter. > > NTFS has similar behavior but different techniques for implementing some > of the logic (e.g., it detects the lazy writer thread differently). > > Tony > OSR > > > — > NTFSD is sponsored by OSR > > OSR is hiring!! Info at http://www.osr.com/careers > > For our schedule of debugging and file system seminars visit: > http://www.osr.com/seminars > > To unsubscribe, visit the List Server section of OSR Online at > http://www.osronline.com/page.cfm?name=ListServer >
I was serious when I suggested walking through the FAT code. FAT (and NTFS) will discard a paging I/O write that is beyond VDL unless it is the top level request. But they “throw it away” by returning STATUS_SUCCESS with 0 bytes. My suspicion is that you saw success for small I/O operations because the VDL was out “far enough” that your write would succeed, but that’s just a suspicion and it would take debugger work to satisfy that.
The issue with security checks is that you don’t know if the subsequent call will be done with the relevant credentials – think “SMB server impersonating during IRP_MJ_CREATE”. It isn’t going to impersonate during the subsequent IRP_MJ_WRITE so you need to know what the privilege was during create – that’s presumably why the FAT folks store the bit in the CCB and then check it later. You’d need to do something comparable.
It’s not JUST the VDL extension calls you need to worry about: it’s every operation that is protected by this privilege. YOU need to police them all – and VDL is not the only one for manage volume (as I recall, the defrag API is protected by this as well, plus things like FSCTL_ALLOW_EXTENDED_DASD_IO). Since there’s no definitive list of those operations, the risk is that you’ll be exposing some operation to someone that you shouldn’t be permitting. To be honest, I don’t have a solution I find very satisfactory either.
I’ve actually been studying the FastFat write code for the past few hours
so I took your suggestion to heart.
I guess I’ll worry about the security issues once I get all of the various
writing paths working properly. Thanks.
On Wed, Oct 8, 2014 at 11:28 PM, Tony Mason wrote:
> I was serious when I suggested walking through the FAT code. FAT (and > NTFS) will discard a paging I/O write that is beyond VDL unless it is the > top level request. But they “throw it away” by returning STATUS_SUCCESS > with 0 bytes. My suspicion is that you saw success for small I/O > operations because the VDL was out “far enough” that your write would > succeed, but that’s just a suspicion and it would take debugger work to > satisfy that. > > > > The issue with security checks is that you don’t know if the subsequent > call will be done with the relevant credentials – think “SMB server > impersonating during IRP_MJ_CREATE”. It isn’t going to impersonate during > the subsequent IRP_MJ_WRITE so you need to know what the privilege was > during create – that’s presumably why the FAT folks store the bit in the > CCB and then check it later. You’d need to do something comparable. > > > > It’s not JUST the VDL extension calls you need to worry about: it’s every > operation that is protected by this privilege. YOU need to police them all > – and VDL is not the only one for manage volume (as I recall, the defrag > API is protected by this as well, plus things like > FSCTL_ALLOW_EXTENDED_DASD_IO). Since there’s no definitive list of those > operations, the risk is that you’ll be exposing some operation to someone > that you shouldn’t be permitting. To be honest, I don’t have a solution I > find very satisfactory either. > > > > Tony > > OSR > > > > — > NTFSD is sponsored by OSR > > OSR is hiring!! Info at http://www.osr.com/careers > > For our schedule of debugging and file system seminars visit: > http://www.osr.com/seminars > > To unsubscribe, visit the List Server section of OSR Online at > http://www.osronline.com/page.cfm?name=ListServer >