Welcome to the wonderful world of caching (and/or deferral).
Tony
OSR
Welcome to the wonderful world of caching (and/or deferral).
Tony
OSR
The file server on Windows 7/Server 2008 takes advantage of a much improved oplock mechanism that allows it to cache more effectively than previous versions.
Does filter manager also release name information when last reference to the file goes away?
Christian [MSFT]
This posting is provided “AS IS” with no warranties, and confers no rights.
Handle oplocks.
Further, in Windows 8, handle oplocks are now supported on directories; presumably this should allow caching of even more information (e.g., directory contents).
As an aside, it is interesting that while handle oplocks are supported in Windows 7, they’re actually a bit of a challenge to test as the ifs kit tests don’t actually exercise them.
Tony
OSR
Thank you all for the responses.
Unfortunately, we cannot expect all Windows machines in a network to run Vista/Windows 7(+) to benefit from the optimizations.
Hence, revisiting one of the question that I asked earlier: Is “short names not expanded” the only difference between FLT_FILE_NAME_NORMALIZED and FLT_FILE_NAME_OPENED? In that case, can we use some heuristic? e.g. Use FLT_FILE_NAME_OPENED and if the returned path contains any tildes then use FLT_FILE_NAME_NORMALIZED. Agreed, it will cause additional overheads for files accessed using short paths OR if the file genuinely contains tilde character. But at least for files that are accessed using long names, we can optimize?
Thanks.
-Prasad
Normalized names also resolve mount points.
xxxxx@vmware.com wrote:
Thank you all for the responses.
Unfortunately, we cannot expect all Windows machines in a network to run Vista/Windows 7(+) to benefit from the optimizations.
Hence, revisiting one of the question that I asked earlier: Is “short names not expanded” the only difference between FLT_FILE_NAME_NORMALIZED and FLT_FILE_NAME_OPENED? In that case, can we use some heuristic? e.g. Use FLT_FILE_NAME_OPENED and if the returned path contains any tildes then use FLT_FILE_NAME_NORMALIZED. Agreed, it will cause additional overheads for files accessed using short paths OR if the file genuinely contains tilde character. But at least for files that are accessed using long names, we can optimize?
Thanks.
-Prasad
–
Kind regards, Dejan (MSN support: xxxxx@alfasp.com)
http://www.alfasp.com
File system audit, security and encryption kits.
I’m not sure how this would work with open by file ID or object ID. Of course, the simplest thing to do would be to simply try it. That’s what Ladislav’s excellent little testing tool is for - so you can open a file in various ways to see what happens, quickly and without extra programming effort.
Tony
OSR
This won’t work because not all short names have the tilde character. You would need to treat any name shorter than 8 characters as potentially a short name.
What do you use the name for?
Thanks,
Alex
Sent from my iPhone
On Aug 16, 2012, at 6:43 AM, Tony Mason wrote:
> I’m not sure how this would work with open by file ID or object ID. Of course, the simplest thing to do would be to simply try it. That’s what Ladislav’s excellent little testing tool is for - so you can open a file in various ways to see what happens, quickly and without extra programming effort.
>
> Tony
> OSR
>
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Are you back?
Mm
On Aug 16, 2012 7:29 AM, “Alex Carp” wrote:
> This won’t work because not all short names have the tilde character. You
> would need to treat any name shorter than 8 characters as potentially a
> short name.
>
> What do you use the name for?
>
> Thanks,
> Alex
>
> Sent from my iPhone
>
> On Aug 16, 2012, at 6:43 AM, Tony Mason wrote:
>
> > I’m not sure how this would work with open by file ID or object ID. Of
> course, the simplest thing to do would be to simply try it. That’s what
> Ladislav’s excellent little testing tool is for - so you can open a file in
> various ways to see what happens, quickly and without extra programming
> effort.
> >
> > Tony
> > OSR
> >
> >
> > —
> > NTFSD is sponsored by OSR
> >
> > For our schedule of debugging and file system seminars visit:
> > http://www.osr.com/seminars
> >
> > To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
I am , but i’m in redmond at plugfest
Sent from my iPhone
On Aug 16, 2012, at 11:10 AM, “Martin O’Brien” wrote:
> Are you back?
>
> Mm
>
> On Aug 16, 2012 7:29 AM, “Alex Carp” wrote:
> This won’t work because not all short names have the tilde character. You would need to treat any name shorter than 8 characters as potentially a short name.
>
> What do you use the name for?
>
> Thanks,
> Alex
>
> Sent from my iPhone
>
> On Aug 16, 2012, at 6:43 AM, Tony Mason wrote:
>
> > I’m not sure how this would work with open by file ID or object ID. Of course, the simplest thing to do would be to simply try it. That’s what Ladislav’s excellent little testing tool is for - so you can open a file in various ways to see what happens, quickly and without extra programming effort.
> >
> > Tony
> > OSR
> >
> >
> > —
> > NTFSD is sponsored by OSR
> >
> > For our schedule of debugging and file system seminars visit:
> > http://www.osr.com/seminars
> >
> > To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
> — NTFSD is sponsored by OSR For our schedule of debugging and file system seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Outstanding.
mm
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Alex Carp
Sent: Thursday, August 16, 2012 12:41 PM
To: Windows File Systems Devs Interest List
Cc: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Mini filter peformance issues while accessing network files
I am , but i’m in redmond at plugfest
Sent from my iPhone
On Aug 16, 2012, at 11:10 AM, “Martin O’Brien” wrote:
Are you back?
Mm
On Aug 16, 2012 7:29 AM, “Alex Carp” wrote:
This won’t work because not all short names have the tilde character. You would need to treat any name shorter than 8 characters as potentially a short name.
What do you use the name for?
Thanks,
Alex
Sent from my iPhone
On Aug 16, 2012, at 6:43 AM, Tony Mason wrote:
> I’m not sure how this would work with open by file ID or object ID. Of course, the simplest thing to do would be to simply try it. That’s what Ladislav’s excellent little testing tool is for - so you can open a file in various ways to see what happens, quickly and without extra programming effort.
>
> Tony
> OSR
>
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
—
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
— NTFSD is sponsored by OSR For our schedule of debugging and file system seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
—
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
@Dejan, ok, so, we may end up reporting the name without resolving mount points. I am thinking of using the heuristic only for network files.
@Tony, thanks for the pointer. I will give it a try. So you suspect that if the file is opened using file id, opened name may not return us the filename.
@Alex, are you saying that you can have long file names (that don’t fit in 8.3) whose corresponding short filename does not contain tilde character? I looked at http://en.wikipedia.org/wiki/8.3_filename and that doesn’t seem to be the case? We use the name in display and we may re-open the file with that name from our mini-filter.
Thanks.
-Prasad
Yup, that’s what i’m saying. Please don’t forget that it’s possible for a user to set the short name for a file directly as well.
In my opinion if all you do is display the name and use it to re-open then you can just the opened name and not the normalized one. I view the normalized name as necessary only when you’re trying to ‘understand’ the namespace, like when you need to compare paths or possibly do things based on the path ( for path-based policy and such).
Thanks,
Alex.
Sent from my iPhone
On Aug 17, 2012, at 2:35 AM, xxxxx@vmware.com wrote:
@Dejan, ok, so, we may end up reporting the name without resolving mount points. I am thinking of using the heuristic only for network files.
@Tony, thanks for the pointer. I will give it a try. So you suspect that if the file is opened using file id, opened name may not return us the filename.
@Alex, are you saying that you can have long file names (that don’t fit in 8.3) whose corresponding short filename does not contain tilde character? I looked at http://en.wikipedia.org/wiki/8.3_filename and that doesn’t seem to be the case? We use the name in display and we may re-open the file with that name from our mini-filter.
Thanks.
-Prasad
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
@Alex, thanks for pointing this out. However, I guess this is possible only if the end user explicitly sets the short name directly? If the short name is OS generated, then, it will always contain tilde (assuming that the file name doesn’t fit in 8.3) right?
My concern was actually around backward compatibility. Our filter is consumed by third parties and so far we were returning them normalized names. With the change, we will start returning opened names instead and hence I wanted to understand the differences and see if it can affect them.
We are trying to keep the deviation minimal. We are returning opened names only for network files. If the opened name contains tilde, then, we return normalized name.
Based on the discussion so far, there are few cases which we need to worry about.
Thanks.
-Prasad
Thanks,
Alex.
Thanks Alex.
Yes, we are calling it from PostOpCreate. Dejan mentioned earlier that “Normalized names also resolve mount points.”. Hence, I thought that opened name do not resolve it. I will confirm this though.
Thanks.
-Prasad
RtlGenerate8dot3Name happens to always generate names with ~ in them for now. This is an implementation detail that should not be relied upon to remain true. The algorithm for generating short names has changed in the past, and may change in the future.
See the “Remarks” section of http://msdn.microsoft.com/en-us/library/windows/hardware/ff544633(v=vs.85).aspx for the definitions of what constitutes a “normalized”, “opened”, and “short” name. That section notes that mount points are not resolved in a query for FLT_FILE_NAME_OPENED. After all, the “opened name” is the name that the caller used to open the file. If the Filter Manager resolved mount points in a query for the opened name, it would no longer be the name the caller used.
Christian [MSFT]
This posting is provided “AS IS” with no warranties, and confers no rights.
Wait, so my assumption up to this point has been that that opened name is
not exactly “the name the caller used” but rather the “the name the IO
manager used when issuing the IRP_MJ_CREATE”. So for a case where you have
\Device\HarddiskVolume1\foo.txt which reparses to
\Device\HarddiskVolume2\bar.txt, the name the caller used might have been
“c:\foo~1.txt” or “\Device\HarddiskVolume1\foo~1.txt” and the opened name
for the first IRP_MJ_CREATE would be “\Device\HarddiskVolume1\ foo~1.txt”
and the opened name for the second IRP_MJ_CREATE would be
“\Device\HarddiskVolume2\bar.txt”. Is this not the case?
Thanks,
Alex.
I never actually tested it with Opened, since we require Normalized for any successful Post op.
xxxxx@vmware.com wrote:
Thanks Alex.
Yes, we are calling it from PostOpCreate. Dejan mentioned earlier that “Normalized names also resolve mount points.”. Hence, I thought that opened name do not resolve it. I will confirm this though.
Thanks.
-Prasad
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminarsTo unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
–
Kind regards, Dejan (MSN support: xxxxx@alfasp.com)
http://www.alfasp.com
File system audit, security and encryption kits.
Alex,
Christian [MSFT]
This posting is provided “AS IS” with no warranties, and confers no rights.
What about in pre create? I didn’t think that mount points were resolved if
the request was for “opened name”.
Bill Wandel
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@microsoft.com
Sent: Tuesday, August 21, 2012 3:26 PM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] Mini filter peformance issues while accessing network
files
Alex,
Christian [MSFT]
This posting is provided “AS IS” with no warranties, and confers no rights.
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer