Deferring IRP_MJ_CREATE for "CIFS Server" access

I have written a file system filter that will defer processing IRP_MJ_CREATE requests for certain files (and possibly directories). To perform this operation, I set a cancel routine on the IRP (in case the process that issued the IRP exits), call IoMarkIrpPending() in the dispatch routine and return STATUS_PENDING from the dispatch routine. The IRP is queued on a list waiting for service in my filter. On Windows 2000, any local process that is accessing one of these files hangs indefinitely waiting for the IRP to be completed. I simulate this by NOT servicing the list. However, when the file is accessed across a CIFS share, the redir-server connection gives up after about 30-60 seconds, returning STATUS_NETWORK_NAME_DELETED. The IRP is still pending, but the connection between the client and the file server is broken. In all normal cases, the 30-60 seconds is plenty of time to service the request. However, there could be cases where the request will not be serviced for over 30-60 seconds. It seems that this would be a case already handled by Windows 2000’s server when using OFFLINE storage through RSS (where request can take many-many minutes to retrieve). I see a high latency bit in the reparse tag definition, but it seems odd that the server would be reading the reparse data to determine if it should extend the timeout for the connection. Currently, I am not using REPARSE tags and would rather not. Partly for design reasons and partly due to Microsoft’s own disclaimer …

“Reparse Points are a powerful feature of Windows 2000 (not available on Windows NT 4.0), but developers should be aware that there can only be one reparse point per file, and some new Windows 2000 mechanisms use reparse points (HSM, Native Structured Storage). Developers need to have fallback strategies for when the reparse point tag is already in use for a file.”

Thanks for any information, /TomH

The system is not designed to handle high latency create IRPs. RSS does
not recall the file on the open/create operation, instead it recalls the
file on the first read or write which can be delayed for a long time.

If possible you should change your design to not have high latency
creates.

Neal Christiansen
Microsoft File System Filter Group

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: Tom Hansen [mailto:xxxxx@inflectionsystems.com]
Sent: Tuesday, October 22, 2002 6:27 AM
To: File Systems Developers
Subject: [ntfsd] Deferring IRP_MJ_CREATE for “CIFS Server” access

I have written a file system filter that will defer processing
IRP_MJ_CREATE requests for certain files (and possibly directories). To
perform this operation, I set a cancel routine on the IRP (in case the
process that issued the IRP exits), call IoMarkIrpPending() in the
dispatch routine and return STATUS_PENDING from the dispatch routine.
The IRP is queued on a list waiting for service in my filter. On
Windows 2000, any local process that is accessing one of these files
hangs indefinitely waiting for the IRP to be completed. I simulate this
by NOT servicing the list. However, when the file is accessed across a
CIFS share, the redir-server connection gives up after about 30-60
seconds, returning STATUS_NETWORK_NAME_DELETED. The IRP is still
pending, but the connection between the client and the file server is
broken. In all normal cases, the 30-60 seconds is plenty of time to
service the request. However, there could be cases where the request
will not be serviced for over 30-60 seconds. It seems that this would
be a case already handled by Windows 2000’s server when using OFFLINE
storage through RSS (where request can take many-many minutes to
retrieve). I see a high latency bit in the reparse tag definition, but
it seems odd that the server would be reading the reparse data to
determine if it should extend the timeout for the connection.
Currently, I am not using REPARSE tags and would rather not. Partly for
design reasons and partly due to Microsoft’s own disclaimer …

“Reparse Points are a powerful feature of Windows 2000 (not available on
Windows NT 4.0), but developers should be aware that there can only be
one reparse point per file, and some new Windows 2000 mechanisms use
reparse points (HSM, Native Structured Storage). Developers need to have
fallback strategies for when the reparse point tag is already in use for
a file.”

Thanks for any information, /TomH


You are currently subscribed to ntfsd as: xxxxx@Windows.Microsoft.com
To unsubscribe send a blank email to %%email.unsub%%

I do that. I was considering that behavior anyway, to avoid the inadvertant application open function that specifies GENERIC_READ only the purpose of looking at attributes etc. on the file.
Thanks. /TomH

-----Original Message-----
From: Neal Christiansen [mailto:xxxxx@windows.microsoft.com]
Sent: Tuesday, October 22, 2002 2:56 PM
To: File Systems Developers
Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

The system is not designed to handle high latency create IRPs. RSS does
not recall the file on the open/create operation, instead it recalls the
file on the first read or write which can be delayed for a long time.

If possible you should change your design to not have high latency
creates.

Neal Christiansen
Microsoft File System Filter Group

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: Tom Hansen [mailto:xxxxx@inflectionsystems.com]
Sent: Tuesday, October 22, 2002 6:27 AM
To: File Systems Developers
Subject: [ntfsd] Deferring IRP_MJ_CREATE for “CIFS Server” access

I have written a file system filter that will defer processing
IRP_MJ_CREATE requests for certain files (and possibly directories). To
perform this operation, I set a cancel routine on the IRP (in case the
process that issued the IRP exits), call IoMarkIrpPending() in the
dispatch routine and return STATUS_PENDING from the dispatch routine.
The IRP is queued on a list waiting for service in my filter. On
Windows 2000, any local process that is accessing one of these files
hangs indefinitely waiting for the IRP to be completed. I simulate this
by NOT servicing the list. However, when the file is accessed across a
CIFS share, the redir-server connection gives up after about 30-60
seconds, returning STATUS_NETWORK_NAME_DELETED. The IRP is still
pending, but the connection between the client and the file server is
broken. In all normal cases, the 30-60 seconds is plenty of time to
service the request. However, there could be cases where the request
will not be serviced for over 30-60 seconds. It seems that this would
be a case already handled by Windows 2000’s server when using OFFLINE
storage through RSS (where request can take many-many minutes to
retrieve). I see a high latency bit in the reparse tag definition, but
it seems odd that the server would be reading the reparse data to
determine if it should extend the timeout for the connection.
Currently, I am not using REPARSE tags and would rather not. Partly for
design reasons and partly due to Microsoft’s own disclaimer …

“Reparse Points are a powerful feature of Windows 2000 (not available on
Windows NT 4.0), but developers should be aware that there can only be
one reparse point per file, and some new Windows 2000 mechanisms use
reparse points (HSM, Native Structured Storage). Developers need to have
fallback strategies for when the reparse point tag is already in use for
a file.”

Thanks for any information, /TomH


You are currently subscribed to ntfsd as: xxxxx@Windows.Microsoft.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntfsd as: xxxxx@inflectionsystems.com
To unsubscribe send a blank email to %%email.unsub%%

I have also seen some need for a need to delay the create call. How much
latency is acceptable? Can you do a few other requests before allowing the
create to be seen by the file system? Are there some files that should
never have this done to them and how can you detect one of them is involved?

----- Original Message -----
From: “Neal Christiansen”
To: “File Systems Developers”
Sent: Tuesday, October 22, 2002 2:55 PM
Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

The system is not designed to handle high latency create IRPs. RSS does
not recall the file on the open/create operation, instead it recalls the
file on the first read or write which can be delayed for a long time.

If possible you should change your design to not have high latency
creates.

Neal Christiansen
Microsoft File System Filter Group

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: Tom Hansen [mailto:xxxxx@inflectionsystems.com]
Sent: Tuesday, October 22, 2002 6:27 AM
To: File Systems Developers
Subject: [ntfsd] Deferring IRP_MJ_CREATE for “CIFS Server” access

I have written a file system filter that will defer processing
IRP_MJ_CREATE requests for certain files (and possibly directories). To
perform this operation, I set a cancel routine on the IRP (in case the
process that issued the IRP exits), call IoMarkIrpPending() in the
dispatch routine and return STATUS_PENDING from the dispatch routine.
The IRP is queued on a list waiting for service in my filter. On
Windows 2000, any local process that is accessing one of these files
hangs indefinitely waiting for the IRP to be completed. I simulate this
by NOT servicing the list. However, when the file is accessed across a
CIFS share, the redir-server connection gives up after about 30-60
seconds, returning STATUS_NETWORK_NAME_DELETED. The IRP is still
pending, but the connection between the client and the file server is
broken. In all normal cases, the 30-60 seconds is plenty of time to
service the request. However, there could be cases where the request
will not be serviced for over 30-60 seconds. It seems that this would
be a case already handled by Windows 2000’s server when using OFFLINE
storage through RSS (where request can take many-many minutes to
retrieve). I see a high latency bit in the reparse tag definition, but
it seems odd that the server would be reading the reparse data to
determine if it should extend the timeout for the connection.
Currently, I am not using REPARSE tags and would rather not. Partly for
design reasons and partly due to Microsoft’s own disclaimer …

“Reparse Points are a powerful feature of Windows 2000 (not available on
Windows NT 4.0), but developers should be aware that there can only be
one reparse point per file, and some new Windows 2000 mechanisms use
reparse points (HSM, Native Structured Storage). Developers need to have
fallback strategies for when the reparse point tag is already in use for
a file.”

Thanks for any information, /TomH


You are currently subscribed to ntfsd as: xxxxx@Windows.Microsoft.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntfsd as: xxxxx@yoshimuni.com
To unsubscribe send a blank email to %%email.unsub%%

Another reason high latency creates are not a good ideas is because
create IRPs are not cancelable. Even if you define a cancel routine,
the IOManager will not actually cancel them which is why you saw the
applications hang waiting for it’s create to complete. There are plans
to fix this in the future.

Neal Christiansen
Microsoft File System Filter Group

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: Tom Hansen [mailto:xxxxx@inflectionsystems.com]
Sent: Tuesday, October 22, 2002 12:25 PM
To: File Systems Developers
Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

I do that. I was considering that behavior anyway, to avoid the
inadvertant application open function that specifies GENERIC_READ only
the purpose of looking at attributes etc. on the file.
Thanks. /TomH

-----Original Message-----
From: Neal Christiansen [mailto:xxxxx@windows.microsoft.com]
Sent: Tuesday, October 22, 2002 2:56 PM
To: File Systems Developers
Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

The system is not designed to handle high latency create IRPs. RSS does
not recall the file on the open/create operation, instead it recalls the
file on the first read or write which can be delayed for a long time.

If possible you should change your design to not have high latency
creates.

Neal Christiansen
Microsoft File System Filter Group

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: Tom Hansen [mailto:xxxxx@inflectionsystems.com]
Sent: Tuesday, October 22, 2002 6:27 AM
To: File Systems Developers
Subject: [ntfsd] Deferring IRP_MJ_CREATE for “CIFS Server” access

I have written a file system filter that will defer processing
IRP_MJ_CREATE requests for certain files (and possibly directories). To
perform this operation, I set a cancel routine on the IRP (in case the
process that issued the IRP exits), call IoMarkIrpPending() in the
dispatch routine and return STATUS_PENDING from the dispatch routine.
The IRP is queued on a list waiting for service in my filter. On
Windows 2000, any local process that is accessing one of these files
hangs indefinitely waiting for the IRP to be completed. I simulate this
by NOT servicing the list. However, when the file is accessed across a
CIFS share, the redir-server connection gives up after about 30-60
seconds, returning STATUS_NETWORK_NAME_DELETED. The IRP is still
pending, but the connection between the client and the file server is
broken. In all normal cases, the 30-60 seconds is plenty of time to
service the request. However, there could be cases where the request
will not be serviced for over 30-60 seconds. It seems that this would
be a case already handled by Windows 2000’s server when using OFFLINE
storage through RSS (where request can take many-many minutes to
retrieve). I see a high latency bit in the reparse tag definition, but
it seems odd that the server would be reading the reparse data to
determine if it should extend the timeout for the connection.
Currently, I am not using REPARSE tags and would rather not. Partly for
design reasons and partly due to Microsoft’s own disclaimer …

“Reparse Points are a powerful feature of Windows 2000 (not available on
Windows NT 4.0), but developers should be aware that there can only be
one reparse point per file, and some new Windows 2000 mechanisms use
reparse points (HSM, Native Structured Storage). Developers need to have
fallback strategies for when the reparse point tag is already in use for
a file.”

Thanks for any information, /TomH


You are currently subscribed to ntfsd as: xxxxx@Windows.Microsoft.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntfsd as: xxxxx@inflectionsystems.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntfsd as: xxxxx@Windows.Microsoft.com
To unsubscribe send a blank email to %%email.unsub%%

It is ok to do other file Io operations while processing the create, you
simply don’t won’t to hold it off for long periods of time.

Neal Christiansen
Microsoft File System Filter Group

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: David J. Craig [mailto:xxxxx@yoshimuni.com]
Sent: Tuesday, October 22, 2002 12:26 PM
To: File Systems Developers
Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

I have also seen some need for a need to delay the create call. How
much
latency is acceptable? Can you do a few other requests before allowing
the
create to be seen by the file system? Are there some files that should
never have this done to them and how can you detect one of them is
involved?

----- Original Message -----
From: “Neal Christiansen”
To: “File Systems Developers”
Sent: Tuesday, October 22, 2002 2:55 PM
Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

The system is not designed to handle high latency create IRPs. RSS does
not recall the file on the open/create operation, instead it recalls the
file on the first read or write which can be delayed for a long time.

If possible you should change your design to not have high latency
creates.

Neal Christiansen
Microsoft File System Filter Group

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: Tom Hansen [mailto:xxxxx@inflectionsystems.com]
Sent: Tuesday, October 22, 2002 6:27 AM
To: File Systems Developers
Subject: [ntfsd] Deferring IRP_MJ_CREATE for “CIFS Server” access

I have written a file system filter that will defer processing
IRP_MJ_CREATE requests for certain files (and possibly directories). To
perform this operation, I set a cancel routine on the IRP (in case the
process that issued the IRP exits), call IoMarkIrpPending() in the
dispatch routine and return STATUS_PENDING from the dispatch routine.
The IRP is queued on a list waiting for service in my filter. On
Windows 2000, any local process that is accessing one of these files
hangs indefinitely waiting for the IRP to be completed. I simulate this
by NOT servicing the list. However, when the file is accessed across a
CIFS share, the redir-server connection gives up after about 30-60
seconds, returning STATUS_NETWORK_NAME_DELETED. The IRP is still
pending, but the connection between the client and the file server is
broken. In all normal cases, the 30-60 seconds is plenty of time to
service the request. However, there could be cases where the request
will not be serviced for over 30-60 seconds. It seems that this would
be a case already handled by Windows 2000’s server when using OFFLINE
storage through RSS (where request can take many-many minutes to
retrieve). I see a high latency bit in the reparse tag definition, but
it seems odd that the server would be reading the reparse data to
determine if it should extend the timeout for the connection.
Currently, I am not using REPARSE tags and would rather not. Partly for
design reasons and partly due to Microsoft’s own disclaimer …

“Reparse Points are a powerful feature of Windows 2000 (not available on
Windows NT 4.0), but developers should be aware that there can only be
one reparse point per file, and some new Windows 2000 mechanisms use
reparse points (HSM, Native Structured Storage). Developers need to have
fallback strategies for when the reparse point tag is already in use for
a file.”

Thanks for any information, /TomH


You are currently subscribed to ntfsd as: xxxxx@Windows.Microsoft.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntfsd as: xxxxx@yoshimuni.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntfsd as: xxxxx@Windows.Microsoft.com
To unsubscribe send a blank email to %%email.unsub%%

The problem with this model is you can’t access the file’s data stream to
write the re-called data, unless of course the users file object has write
access and you borrow his file object. What model is used by RSS to solve
this problem? Once the IRP_MJ_CREATE is complete, the file’s share mode
restrictions are set in the i/o subsystem. Even when I try to always set
FILE_SHARE_WRITE on the user’s IRP_MJ_CREATE to allow us to write the file
data (which is not a good idea anyway), I get very long delays trying to
open the file while the user’s IRP_MJ_READ is pending. What I see is the
following IRP flow … with a very long delay between REQUEST_BATCH_OPLOCK
and OPLOCK_BREAK_ACKNOWLEDGE, which is probably some sort of deadlock trying
to break the OPLOCK that the redirector acquires on exclusively opened
files.

9004:00000a5c:00000a7c> Request: IRP_MJ_CREATE FO=0xFF234168
File:\SecondCollection\GFiles\Files.9\FileName.95
Flags=0x0 Attribs=0x80 Options=0x1000002 ShareMode=0x1
FileObject->Flags=0x10
9004:00000a5c:00000a7c> Complete: IRP_MJ_FILE_SYSTEM_CONTROL
Minor=FSCTL_REQUEST_BATCH_OPLOCK (USER) IRP=0x8584B488 FO=0x8123C688
(Nested)
File=
Flags=0x00000010 Status=0x00000000
11397:00000008:00000014> Request: IRP_MJ_FILE_SYSTEM_CONTROL
Minor=FSCTL_OPLOCK_BREAK_ACKNOWLEDGE (USER) IRP=0x8584B488 FO=0x8123C688
File=E:\SecondCollection\GFiles\Files.9\FileName.95
Flags=0x00000010 Status=0x00000000
11397:00000a5c:00000a7c> Re-call open for file
\DosDevices\E:\SecondCollection\GFiles\Files.9\FileName.95 returned
0x00000000.

>From: Neal Christiansen [mailto:xxxxx@windows.microsoft.com]
>Sent: Tuesday, October 22, 2002 2:56 PM
>To: File Systems Developers
>Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

>The system is not designed to handle high latency create IRPs. RSS does
>not recall the file on the open/create operation, instead it recalls the
>file on the first read or write which can be delayed for a long time.

>If possible you should change your design to not have high latency creates.

_________________________________________________________________
Broadband? Dial-up? Get reliable MSN Internet Access.
http://resourcecenter.msn.com/access/plans/default.asp

Let me first address the long delay you are seeing on the create. This occurs because you are trying to open the file in the same thread context that the read is occurring in. When this file was originally opened an OPLOCK was granted on the file. Because you are trying to open it again the first oplock needs to be broken. SRV sends a request back asking for an oplock break and waits for a response. Unfortunately your thread can’t respond because it is processing the new open request. SRV times out after 35 seconds and grants the oplock break anyway. This is the long delay you are seeing. Fortunately there is an easy fix for this. Add the flag FILE_COMPLETE_IF_OPLOCKED to the create options and the delay will go away.

How HSM handles the situation you described is this: It opens the desired file requesting only FILE_READ_ATTRIBUTES and SYNCHRONIZE access. After the file is opened it then sets the FileObject->WriteAccess field to TRUE. You can now generate your own write Irps to write to the file.

There is an issue you need to be aware of when you do this. Specifically, you can not do this trick for encrypted or compressed files. NTFS sets up internal state at open time based on the type of access desired. Using this trick those operations will fail because that state is not setup. This however did not affect HSM because at the time of the file recall the original file had already been “sparsed” therefore it never had to operate on an encrypted or compressed file.

Neal Christiansen
Microsoft File System Filter Group

This posting is provided “AS IS” with no warranties, and confers no rights.

-----Original Message-----
From: Tom Hansen [mailto:xxxxx@hotmail.com]
Sent: Tuesday, October 29, 2002 2:52 PM
To: File Systems Developers
Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

The problem with this model is you can’t access the file’s data stream to
write the re-called data, unless of course the users file object has write
access and you borrow his file object. What model is used by RSS to solve
this problem? Once the IRP_MJ_CREATE is complete, the file’s share mode
restrictions are set in the i/o subsystem. Even when I try to always set
FILE_SHARE_WRITE on the user’s IRP_MJ_CREATE to allow us to write the file
data (which is not a good idea anyway), I get very long delays trying to
open the file while the user’s IRP_MJ_READ is pending. What I see is the
following IRP flow … with a very long delay between REQUEST_BATCH_OPLOCK
and OPLOCK_BREAK_ACKNOWLEDGE, which is probably some sort of deadlock trying
to break the OPLOCK that the redirector acquires on exclusively opened
files.

9004:00000a5c:00000a7c> Request: IRP_MJ_CREATE FO=0xFF234168
File:\SecondCollection\GFiles\Files.9\FileName.95
Flags=0x0 Attribs=0x80 Options=0x1000002 ShareMode=0x1
FileObject->Flags=0x10
9004:00000a5c:00000a7c> Complete: IRP_MJ_FILE_SYSTEM_CONTROL
Minor=FSCTL_REQUEST_BATCH_OPLOCK (USER) IRP=0x8584B488 FO=0x8123C688
(Nested)
File=
Flags=0x00000010 Status=0x00000000
11397:00000008:00000014> Request: IRP_MJ_FILE_SYSTEM_CONTROL
Minor=FSCTL_OPLOCK_BREAK_ACKNOWLEDGE (USER) IRP=0x8584B488 FO=0x8123C688
File=E:\SecondCollection\GFiles\Files.9\FileName.95
Flags=0x00000010 Status=0x00000000
11397:00000a5c:00000a7c> Re-call open for file
\DosDevices\E:\SecondCollection\GFiles\Files.9\FileName.95 returned
0x00000000.

>From: Neal Christiansen [mailto:xxxxx@windows.microsoft.com]
>Sent: Tuesday, October 22, 2002 2:56 PM
>To: File Systems Developers
>Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

>The system is not designed to handle high latency create IRPs. RSS does
>not recall the file on the open/create operation, instead it recalls the
>file on the first read or write which can be delayed for a long time.

>If possible you should change your design to not have high latency creates.

_________________________________________________________________
Broadband??Dial-up? Get reliable MSN Internet Access.
http://resourcecenter.msn.com/access/plans/default.asp


You are currently subscribed to ntfsd as: xxxxx@Windows.Microsoft.com
To unsubscribe send a blank email to %%email.unsub%%

I denied the OPLOCKs for files that are OFFLINE which also fixed the time delay. The documentation indicates that when I specify FILE_COMPLETE_IF_OPLOCKED the create will fail, not complete without breaking the locks. I will look at this again. It did not know about the trick of setting the FileObject->WriteAccess field after the open request, but the writes do still work when you open for synchronize. However, you will get assertions on checked builds which check for the appropriate access to the file before allowing the “roll your own” write IRP to be serviced. Thanks.

-----Original Message-----
From: Neal Christiansen [mailto:xxxxx@windows.microsoft.com]
Sent: Thursday, October 31, 2002 11:36 PM
To: File Systems Developers
Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

Let me first address the long delay you are seeing on the create. This occurs because you are trying to open the file in the same thread context that the read is occurring in. When this file was originally opened an OPLOCK was granted on the file. Because you are trying to open it again the first oplock needs to be broken. SRV sends a request back asking for an oplock break and waits for a response. Unfortunately your thread can’t respond because it is processing the new open request. SRV times out after 35 seconds and grants the oplock break anyway. This is the long delay you are seeing. Fortunately there is an easy fix for this. Add the flag FILE_COMPLETE_IF_OPLOCKED to the create options and the delay will go away.

How HSM handles the situation you described is this: It opens the desired file requesting only FILE_READ_ATTRIBUTES and SYNCHRONIZE access. After the file is opened it then sets the FileObject->WriteAccess field to TRUE. You can now generate your own write Irps to write to the file.

There is an issue you need to be aware of when you do this. Specifically, you can not do this trick for encrypted or compressed files. NTFS sets up internal state at open time based on the type of access desired. Using this trick those operations will fail because that state is not setup. This however did not affect HSM because at the time of the file recall the original file had already been “sparsed” therefore it never had to operate on an encrypted or compressed file.

Neal Christiansen
Microsoft File System Filter Group

This posting is provided “AS IS” with no warranties, and confers no rights.

-----Original Message-----
From: Tom Hansen [mailto:xxxxx@hotmail.com]
Sent: Tuesday, October 29, 2002 2:52 PM
To: File Systems Developers
Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

The problem with this model is you can’t access the file’s data stream to
write the re-called data, unless of course the users file object has write
access and you borrow his file object. What model is used by RSS to solve
this problem? Once the IRP_MJ_CREATE is complete, the file’s share mode
restrictions are set in the i/o subsystem. Even when I try to always set
FILE_SHARE_WRITE on the user’s IRP_MJ_CREATE to allow us to write the file
data (which is not a good idea anyway), I get very long delays trying to
open the file while the user’s IRP_MJ_READ is pending. What I see is the
following IRP flow … with a very long delay between REQUEST_BATCH_OPLOCK
and OPLOCK_BREAK_ACKNOWLEDGE, which is probably some sort of deadlock trying
to break the OPLOCK that the redirector acquires on exclusively opened
files.

9004:00000a5c:00000a7c> Request: IRP_MJ_CREATE FO=0xFF234168
File:\SecondCollection\GFiles\Files.9\FileName.95
Flags=0x0 Attribs=0x80 Options=0x1000002 ShareMode=0x1
FileObject->Flags=0x10
9004:00000a5c:00000a7c> Complete: IRP_MJ_FILE_SYSTEM_CONTROL
Minor=FSCTL_REQUEST_BATCH_OPLOCK (USER) IRP=0x8584B488 FO=0x8123C688
(Nested)
File=
Flags=0x00000010 Status=0x00000000
11397:00000008:00000014> Request: IRP_MJ_FILE_SYSTEM_CONTROL
Minor=FSCTL_OPLOCK_BREAK_ACKNOWLEDGE (USER) IRP=0x8584B488 FO=0x8123C688
File=E:\SecondCollection\GFiles\Files.9\FileName.95
Flags=0x00000010 Status=0x00000000
11397:00000a5c:00000a7c> Re-call open for file
\DosDevices\E:\SecondCollection\GFiles\Files.9\FileName.95 returned
0x00000000.

>From: Neal Christiansen [mailto:xxxxx@windows.microsoft.com]
>Sent: Tuesday, October 22, 2002 2:56 PM
>To: File Systems Developers
>Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

>The system is not designed to handle high latency create IRPs. RSS does
>not recall the file on the open/create operation, instead it recalls the
>file on the first read or write which can be delayed for a long time.

>If possible you should change your design to not have high latency creates.

_________________________________________________________________
Broadband??Dial-up? Get reliable MSN Internet Access.
http://resourcecenter.msn.com/access/plans/default.asp


You are currently subscribed to ntfsd as: xxxxx@Windows.Microsoft.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntfsd as: xxxxx@inflectionsystems.com
To unsubscribe send a blank email to %%email.unsub%%

I don’t know what documentation you are reading about FILE_COMPLETE_IF_OPLOCKED but the current docs state:

“Complete this operation immediately with an alternate success code if the target file is oplocked, rather than blocking the caller’s thread. If the file is oplocked, another caller already has access to the file over the network. This flag is irrelevant to device and intermediate drivers.”

This means it does not fail the operation, it simply returns a different SUCCESS code, but the operation does succeed.

Neal Christiansen
Microsoft File System Filter Group

This posting is provided “AS IS” with no warranties, and confers no rights.

-----Original Message-----
From: Tom Hansen [mailto:xxxxx@inflectionsystems.com]
Sent: Friday, November 01, 2002 8:34 AM
To: File Systems Developers
Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

I denied the OPLOCKs for files that are OFFLINE which also fixed the time delay. The documentation indicates that when I specify FILE_COMPLETE_IF_OPLOCKED the create will fail, not complete without breaking the locks. I will look at this again. It did not know about the trick of setting the FileObject->WriteAccess field after the open request, but the writes do still work when you open for synchronize. However, you will get assertions on checked builds which check for the appropriate access to the file before allowing the “roll your own” write IRP to be serviced. Thanks.

-----Original Message-----
From: Neal Christiansen [mailto:xxxxx@windows.microsoft.com]
Sent: Thursday, October 31, 2002 11:36 PM
To: File Systems Developers
Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

Let me first address the long delay you are seeing on the create. This occurs because you are trying to open the file in the same thread context that the read is occurring in. When this file was originally opened an OPLOCK was granted on the file. Because you are trying to open it again the first oplock needs to be broken. SRV sends a request back asking for an oplock break and waits for a response. Unfortunately your thread can’t respond because it is processing the new open request. SRV times out after 35 seconds and grants the oplock break anyway. This is the long delay you are seeing. Fortunately there is an easy fix for this. Add the flag FILE_COMPLETE_IF_OPLOCKED to the create options and the delay will go away.

How HSM handles the situation you described is this: It opens the desired file requesting only FILE_READ_ATTRIBUTES and SYNCHRONIZE access. After the file is opened it then sets the FileObject->WriteAccess field to TRUE. You can now generate your own write Irps to write to the file.

There is an issue you need to be aware of when you do this. Specifically, you can not do this trick for encrypted or compressed files. NTFS sets up internal state at open time based on the type of access desired. Using this trick those operations will fail because that state is not setup. This however did not affect HSM because at the time of the file recall the original file had already been “sparsed” therefore it never had to operate on an encrypted or compressed file.

Neal Christiansen
Microsoft File System Filter Group

This posting is provided “AS IS” with no warranties, and confers no rights.

-----Original Message-----
From: Tom Hansen [mailto:xxxxx@hotmail.com]
Sent: Tuesday, October 29, 2002 2:52 PM
To: File Systems Developers
Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

The problem with this model is you can’t access the file’s data stream to
write the re-called data, unless of course the users file object has write
access and you borrow his file object. What model is used by RSS to solve
this problem? Once the IRP_MJ_CREATE is complete, the file’s share mode
restrictions are set in the i/o subsystem. Even when I try to always set
FILE_SHARE_WRITE on the user’s IRP_MJ_CREATE to allow us to write the file
data (which is not a good idea anyway), I get very long delays trying to
open the file while the user’s IRP_MJ_READ is pending. What I see is the
following IRP flow … with a very long delay between REQUEST_BATCH_OPLOCK
and OPLOCK_BREAK_ACKNOWLEDGE, which is probably some sort of deadlock trying
to break the OPLOCK that the redirector acquires on exclusively opened
files.

9004:00000a5c:00000a7c> Request: IRP_MJ_CREATE FO=0xFF234168
File:\SecondCollection\GFiles\Files.9\FileName.95
Flags=0x0 Attribs=0x80 Options=0x1000002 ShareMode=0x1
FileObject->Flags=0x10
9004:00000a5c:00000a7c> Complete: IRP_MJ_FILE_SYSTEM_CONTROL
Minor=FSCTL_REQUEST_BATCH_OPLOCK (USER) IRP=0x8584B488 FO=0x8123C688
(Nested)
File=
Flags=0x00000010 Status=0x00000000
11397:00000008:00000014> Request: IRP_MJ_FILE_SYSTEM_CONTROL
Minor=FSCTL_OPLOCK_BREAK_ACKNOWLEDGE (USER) IRP=0x8584B488 FO=0x8123C688
File=E:\SecondCollection\GFiles\Files.9\FileName.95
Flags=0x00000010 Status=0x00000000
11397:00000a5c:00000a7c> Re-call open for file
\DosDevices\E:\SecondCollection\GFiles\Files.9\FileName.95 returned
0x00000000.

>From: Neal Christiansen [mailto:xxxxx@windows.microsoft.com]
>Sent: Tuesday, October 22, 2002 2:56 PM
>To: File Systems Developers
>Subject: [ntfsd] RE: Deferring IRP_MJ_CREATE for “CIFS Server” access

>The system is not designed to handle high latency create IRPs. RSS does
>not recall the file on the open/create operation, instead it recalls the
>file on the first read or write which can be delayed for a long time.

>If possible you should change your design to not have high latency creates.

_________________________________________________________________
Broadband??Dial-up? Get reliable MSN Internet Access.
http://resourcecenter.msn.com/access/plans/default.asp


You are currently subscribed to ntfsd as: xxxxx@Windows.Microsoft.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntfsd as: xxxxx@inflectionsystems.com
To unsubscribe send a blank email to %%email.unsub%%


You are currently subscribed to ntfsd as: xxxxx@Windows.Microsoft.com
To unsubscribe send a blank email to %%email.unsub%%