RE: Postscript on closing FileObjects created by IoCr eateStreamFile

One thing I know for sure, FO_CLEANUP_COMPLETE does not intefere with CREATE.

Paul

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Thursday, October 09, 2003 7:08 AM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
eateStreamFile

Matt -

The comment was really a reminder that these flags must be cleared. The code
copies the flags from the original FileObject which does indeed have these
flags absent. Perhaps the totally correct thing to do would be copy the
flags AND clear the unwanted ones.

Ex: cloneFileObject->Flags = fileObject->Flags & ~(FO_STREAM_FILE |
FO_HANDLE_CREATED | FO_CLEANUP_COMPLETE);

Also - I’m not sure, but I think the CREATE will fail if any of these are
set.

Thanks, /ted

-----Original Message-----
From: Matthew N. White [mailto:xxxxx@bitarmor.com]
Sent: Wednesday, October 08, 2003 7:12 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] RE: Postscript on closing FileObjects created by
IoCreateStreamFile

Ted,
Is there any reason that the line that clears the FO_STREAM_FILE,
FO_HANDLE_CREATED, FO_CLEANUP_COMPLETE flags in the code is commented out?
Below you say that you “MUST clear the FO_STREAM_FILE, FO_HANDLE_CREATED &
FO_CLEANUP_COMPLETE flags to successfully open this FO via a CREATE IRP.”
Just wondering about that, since the code just copies the flags over.

Matt

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ted Hess
Sent: Tuesday, October 07, 2003 2:39 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Postscript on closing FileObjects created by
IoCreateStreamFile

I’ve been meaning to write up these observations for a couple of months now
but just haven’t had the time. Many thanks to Tony and Nick for their info
and help on this subject back in June when I was struggling with it.

I basically followed the scenario outlined on the OSR on-line site and the
latest NT Insider article with a couple of differences which I believe to be
significant. In particular, NTFS seems to be very sensitive to the Flags
field in the FileObject.

My first attempt essentially followed the steps outlined on the OSR web
site. However, I noticed I was leaking FileObjects!! Why was this? Well, it
turns out the the CLEANUP IRP wasn’t really doing anything like it should
have been and the ObDereferenceObject never issued the CLOSE. I was left
with a FO and a cache reference. Also, the file sharing state was whacked
'cause the CLEANUP didn’t reset it.

After a bit of hair pulling and experimentation, I discovered that this
problem only happened on NTFS and not FAT. FAT does not care about the same
FileObject states as does NTFS (at least from my observations of NTFS
behavior and code inspection in the FAT IFS kit sources). Here’s what I
think needs to happen:

After creating the stream FO, you MUST clear the FO_STREAM_FILE,
FO_HANDLE_CREATED & FO_CLEANUP_COMPLETE flags to successfully open this FO
via a CREATE IRP.

If the CREATE fails, you need to set FO_STREAM_FILE before calling
ObDereferenctObject.

If the CREATE succeeds, you will need to set FO_HANDLE_CREATED before
issuing the CLEANUP and set FO_STREAM_FILE before calling
ObDereferenceObject. Otherwise, things just don’t work as expected.

Comments, mockery and/or derision welcome…

/ted

Attached is an abbreviated, and hopefully useful code snippet for opening a
file and then successfully closing it in your pre-Create filter path.

NTSTATUS
xxxPreCreateCheck(PDEVICE_OBJECT targetDeviceObject, PIRP Irp) {
NTSTATUS Status, RetStatus =
STATUS_SUCCESS;
UNICODE_STRING fileName;
PUNICODE_STRING pFOName;
PFILE_OBJECT fileObject, cloneFileObject;

PIO_STACK_LOCATION irpSp = IoGetCurrentIrpStackLocation(Irp);
PIO_STACK_LOCATION nxtSp = IoGetNextIrpStackLocation(Irp);

ACCESS_MASK desiredAccess =
irpSp->Parameters.Create.SecurityContext->DesiredAccess;
ACCESS_MASK desiredAccessCreate = desiredAccess;

ULONG createOptions = irpSp->Parameters.Create.Options &
FILE_VALID_OPTION_FLAGS;
ULONG createDisposition = (irpSp->Parameters.Create.Options >>
24) & 0xFF;

// The subject of this excercise (save it)
fileObject = irpSp->FileObject;

// Make a copy of the callers buffer here 'cause we don’t know
who’ll free it
fileName.Length = fileObject->FileName.Length;
fileName.MaximumLength = fileObject->FileName.MaximumLength;
fileName.Buffer = ExAllocatePool(PagedPool, fileName.MaximumLength);

// Bag it if no memory available
if (!fileName.Buffer) {
return STATUS_INSUFFICIENT_RESOURCES;
}
// Copy original contents into temp buffer
RtlCopyMemory(fileName.Buffer, fileObject->FileName.Buffer,
fileName.MaximumLength);

// Allocate a new, temporary FO
#if defined(NT_40)
// Watch out for the gratuitous CLEANUP IRP!
cloneFileObject = IoCreateStreamFileObject(fileObject, NULL); #else
cloneFileObject = IoCreateStreamFileObjectLite(fileObject,
NULL); #endif

// (Re-)init newly created FO
// cloneFileObject->Flags &= ~(FO_STREAM_FILE | FO_HANDLE_CREATED |
FO_CLEANUP_COMPLETE);
cloneFileObject->Flags = fileObject->Flags;
cloneFileObject->RelatedFileObject =
fileObject->RelatedFileObject;

// Give it a name
pFOName = &cloneFileObject->FileName;
*pFOName = fileName;

// Initialize embedded event structures
KeInitializeEvent(&cloneFileObject->Lock, SynchronizationEvent,
FALSE);
KeInitializeEvent(&cloneFileObject->Event, NotificationEvent,
FALSE);

// Make a copy of the create irp (preserve flags)
RtlCopyMemory(nxtSp, irpSp, sizeof(IO_STACK_LOCATION));
nxtSp->Control = 0;
// Use temporary FO and underlying DO
nxtSp->FileObject = cloneFileObject;
nxtSp->DeviceObject = DeviceObject;

// Setup completion routine
IoSetCompletionRoutine (Irp, xxxSimpleEventCompletion,
&cloneFileObject->Event, TRUE, TRUE, TRUE);

//
// Ensure that the create disposition requested
// indicates all of the appropriate “rights” necessary
// for the given operation.
//
if (createDisposition == FILE_SUPERSEDE) {
desiredAccessCreate |= DELETE;
}

if ((createDisposition == FILE_OVERWRITE) ||
(createDisposition == FILE_OVERWRITE_IF)) {
desiredAccessCreate |= (FILE_WRITE_DATA | FILE_WRITE_EA
| FILE_WRITE_ATTRIBUTES);
}

// Setup our desired access
nxtSp->Parameters.Create.SecurityContext->DesiredAccess =
desiredAccessCreate & ~SYNCHRONIZE;
nxtSp->Parameters.Create.ShareAccess = FILE_SHARE_VALID_FLAGS;
// Specify open only (no create/supersede)
nxtSp->Parameters.Create.Options = (FILE_OPEN << 24) |
(createOptions &
(FILE_OPEN_NO_RECALL | FILE_OPEN_FOR_BACKUP_INTENT |

FILE_COMPLETE_IF_OPLOCKED | FILE_NON_DIRECTORY_FILE |

FILE_DIRECTORY_FILE | FILE_OPEN_REPARSE_POINT)) |

FILE_NO_INTERMEDIATE_BUFFERING;

//
// Chuck the request to the underlying driver
//
Status = IoCallDriver(DeviceObject, Irp);

if (Status == STATUS_PENDING) {
// Wait for event if pended
KeWaitForSingleObject(&cloneFileObject->Event,
Executive, KernelMode, FALSE, 0);
Status = Irp->IoStatus.Status;
}

// Now check for file open failures
if(!NT_SUCCESS(Status) || (Irp->IoStatus.Status ==
STATUS_REPARSE)) {
// Restore original IRP parameters
irpSp->Parameters.Create.SecurityContext->DesiredAccess
= desiredAccess;

// Free temp (or FSD) buffer if still allocated
if (pFOName->Buffer) {
ExFreePool(pFOName->Buffer);
pFOName->Buffer = NULL;
pFOName->Length = 0;
pFOName->MaximumLength = 0;
}

// Reset FO stuff
cloneFileObject->Flags |= FO_STREAM_FILE;
cloneFileObject->RelatedFileObject = NULL;

// Toss temp FO
ObDereferenceObject(cloneFileObject);

// Nothing has happened yet
Irp->PendingReturned = FALSE;

// OK to proceed if cannot open file
return STATUS_SUCCESS;
}

//
// *** OK - we now have an opened FileObject with a valid FsContext
pointer
//

// RetStatus = <<< *** Insert your desired actions here *** >>>

//
// *** Done with FileObject
//

// Free temp (or FSD) buffer if still allocated
if (pFOName->Buffer) {
ExFreePool(pFOName->Buffer);
pFOName->Buffer = NULL;
pFOName->Length = 0;
pFOName->MaximumLength = 0;
}

// Need to light “handle created” flag
cloneFileObject->Flags |= FO_HANDLE_CREATED;

// Send a CLEANUP down to the FSD
nxtSp->MajorFunction = IRP_MJ_CLEANUP;
nxtSp->MinorFunction = 0;
nxtSp->Flags = 0;
nxtSp->Control = 0;
nxtSp->DeviceObject = DeviceObject;
nxtSp->FileObject = cloneFileObject;

// Init our completion event
KeInitializeEvent(&cloneFileObject->Event, NotificationEvent,
FALSE);

// Setup completion routine
IoSetCompletionRoutine (Irp, xxxSimpleEventCompletion,
&cloneFileObject->Event, TRUE, TRUE, TRUE);

//
// Call the FSD to release any share access, cache refs, etc.
//
Status = IoCallDriver(DeviceObject, Irp);

//
// Wait for the cleanup to finish
//
if (Status == STATUS_PENDING) {
KeWaitForSingleObject(&cloneFileObject->Event,
Executive, KernelMode, FALSE, 0);
}

// Reset FO stuff
cloneFileObject->Flags |= FO_STREAM_FILE;
cloneFileObject->RelatedFileObject = NULL;

// We allocated the FO, now we must free it
ObDereferenceObject(cloneFileObject);

// Restore original IRP parameters
irpSp->Parameters.Create.SecurityContext->DesiredAccess =
desiredAccess;

// Nothing has happened yet
Irp->PendingReturned = FALSE;
Irp->IoStatus.Status = STATUS_SUCCESS;
Irp->IoStatus.Information = 0;

// return STATUS_SUCCESS if it is OK to go ahead and let the caller
open the file
return RetStatus;
}


You are currently subscribed to ntfsd as: xxxxx@bitarmor.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@livevault.com To unsubscribe
send a blank email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@arkivio.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

There is also a following technique to open some file in filter’s CREATE
path:

  • save the original FileObject->FileName, RelatedFileObject and the whole
    CREATE stack location
  • set up the new file name and the new CREATE stack location
  • pass the IRP down
  • work with the opened file
  • then handicraft CLEANUP and CLOSE (in the same IRP and the same stack
    location) and send them down
  • then reset FO_CLEANUP_COMPLETE on the file object
  • then restore the saved CREATE stack location, FileName and
    RelatedFileObject, and pass the original CREATE down at last.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Paul Mu”
To: “Windows File Systems Devs Interest List”
Sent: Saturday, October 11, 2003 3:35 AM
Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
eateStreamFile

> One thing I know for sure, FO_CLEANUP_COMPLETE does not intefere with CREATE.
>
> Paul
>
> -----Original Message-----
> From: Ted Hess [mailto:xxxxx@livevault.com]
> Sent: Thursday, October 09, 2003 7:08 AM
> To: Windows File Systems Devs Interest List
> Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
> eateStreamFile
>
>
> Matt -
>
> The comment was really a reminder that these flags must be cleared. The code
> copies the flags from the original FileObject which does indeed have these
> flags absent. Perhaps the totally correct thing to do would be copy the
> flags AND clear the unwanted ones.
>
> Ex: cloneFileObject->Flags = fileObject->Flags & ~(FO_STREAM_FILE |
> FO_HANDLE_CREATED | FO_CLEANUP_COMPLETE);
>
> Also - I’m not sure, but I think the CREATE will fail if any of these are
> set.
>
> Thanks, /ted
>
>
>
> -----Original Message-----
> From: Matthew N. White [mailto:xxxxx@bitarmor.com]
> Sent: Wednesday, October 08, 2003 7:12 PM
> To: Windows File Systems Devs Interest List
> Subject: [ntfsd] RE: Postscript on closing FileObjects created by
> IoCreateStreamFile
>
>
> Ted,
> Is there any reason that the line that clears the FO_STREAM_FILE,
> FO_HANDLE_CREATED, FO_CLEANUP_COMPLETE flags in the code is commented out?
> Below you say that you “MUST clear the FO_STREAM_FILE, FO_HANDLE_CREATED &
> FO_CLEANUP_COMPLETE flags to successfully open this FO via a CREATE IRP.”
> Just wondering about that, since the code just copies the flags over.
>
> Matt
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Ted Hess
> Sent: Tuesday, October 07, 2003 2:39 PM
> To: Windows File Systems Devs Interest List
> Subject: [ntfsd] Postscript on closing FileObjects created by
> IoCreateStreamFile
>
>
> I’ve been meaning to write up these observations for a couple of months now
> but just haven’t had the time. Many thanks to Tony and Nick for their info
> and help on this subject back in June when I was struggling with it.
>
> I basically followed the scenario outlined on the OSR on-line site and the
> latest NT Insider article with a couple of differences which I believe to be
> significant. In particular, NTFS seems to be very sensitive to the Flags
> field in the FileObject.
>
> My first attempt essentially followed the steps outlined on the OSR web
> site. However, I noticed I was leaking FileObjects!! Why was this? Well, it
> turns out the the CLEANUP IRP wasn’t really doing anything like it should
> have been and the ObDereferenceObject never issued the CLOSE. I was left
> with a FO and a cache reference. Also, the file sharing state was whacked
> 'cause the CLEANUP didn’t reset it.
>
> After a bit of hair pulling and experimentation, I discovered that this
> problem only happened on NTFS and not FAT. FAT does not care about the same
> FileObject states as does NTFS (at least from my observations of NTFS
> behavior and code inspection in the FAT IFS kit sources). Here’s what I
> think needs to happen:
>
> After creating the stream FO, you MUST clear the FO_STREAM_FILE,
> FO_HANDLE_CREATED & FO_CLEANUP_COMPLETE flags to successfully open this FO
> via a CREATE IRP.
>
> If the CREATE fails, you need to set FO_STREAM_FILE before calling
> ObDereferenctObject.
>
> If the CREATE succeeds, you will need to set FO_HANDLE_CREATED before
> issuing the CLEANUP and set FO_STREAM_FILE before calling
> ObDereferenceObject. Otherwise, things just don’t work as expected.
>
> Comments, mockery and/or derision welcome…
>
> /ted
>
> Attached is an abbreviated, and hopefully useful code snippet for opening a
> file and then successfully closing it in your pre-Create filter path.
>
>
> NTSTATUS
> xxxPreCreateCheck(PDEVICE_OBJECT targetDeviceObject, PIRP Irp) {
> NTSTATUS Status, RetStatus =
> STATUS_SUCCESS;
> UNICODE_STRING fileName;
> PUNICODE_STRING pFOName;
> PFILE_OBJECT fileObject, cloneFileObject;
>
> PIO_STACK_LOCATION irpSp = IoGetCurrentIrpStackLocation(Irp);
> PIO_STACK_LOCATION nxtSp = IoGetNextIrpStackLocation(Irp);
>
> ACCESS_MASK desiredAccess =
> irpSp->Parameters.Create.SecurityContext->DesiredAccess;
> ACCESS_MASK desiredAccessCreate = desiredAccess;
>
> ULONG createOptions = irpSp->Parameters.Create.Options &
> FILE_VALID_OPTION_FLAGS;
> ULONG createDisposition = (irpSp->Parameters.Create.Options >>
> 24) & 0xFF;
>
> // The subject of this excercise (save it)
> fileObject = irpSp->FileObject;
>
> // Make a copy of the callers buffer here 'cause we don’t know
> who’ll free it
> fileName.Length = fileObject->FileName.Length;
> fileName.MaximumLength = fileObject->FileName.MaximumLength;
> fileName.Buffer = ExAllocatePool(PagedPool, fileName.MaximumLength);
>
> // Bag it if no memory available
> if (!fileName.Buffer) {
> return STATUS_INSUFFICIENT_RESOURCES;
> }
> // Copy original contents into temp buffer
> RtlCopyMemory(fileName.Buffer, fileObject->FileName.Buffer,
> fileName.MaximumLength);
>
> // Allocate a new, temporary FO
> #if defined(NT_40)
> // Watch out for the gratuitous CLEANUP IRP!
> cloneFileObject = IoCreateStreamFileObject(fileObject, NULL); #else
> cloneFileObject = IoCreateStreamFileObjectLite(fileObject,
> NULL); #endif
>
> // (Re-)init newly created FO
> // cloneFileObject->Flags &= ~(FO_STREAM_FILE | FO_HANDLE_CREATED |
> FO_CLEANUP_COMPLETE);
> cloneFileObject->Flags = fileObject->Flags;
> cloneFileObject->RelatedFileObject =
> fileObject->RelatedFileObject;
>
> // Give it a name
> pFOName = &cloneFileObject->FileName;
> *pFOName = fileName;
>
> // Initialize embedded event structures
> KeInitializeEvent(&cloneFileObject->Lock, SynchronizationEvent,
> FALSE);
> KeInitializeEvent(&cloneFileObject->Event, NotificationEvent,
> FALSE);
>
> // Make a copy of the create irp (preserve flags)
> RtlCopyMemory(nxtSp, irpSp, sizeof(IO_STACK_LOCATION));
> nxtSp->Control = 0;
> // Use temporary FO and underlying DO
> nxtSp->FileObject = cloneFileObject;
> nxtSp->DeviceObject = DeviceObject;
>
> // Setup completion routine
> IoSetCompletionRoutine (Irp, xxxSimpleEventCompletion,
> &cloneFileObject->Event, TRUE, TRUE, TRUE);
>
> //
> // Ensure that the create disposition requested
> // indicates all of the appropriate “rights” necessary
> // for the given operation.
> //
> if (createDisposition == FILE_SUPERSEDE) {
> desiredAccessCreate |= DELETE;
> }
>
> if ((createDisposition == FILE_OVERWRITE) ||
> (createDisposition == FILE_OVERWRITE_IF)) {
> desiredAccessCreate |= (FILE_WRITE_DATA | FILE_WRITE_EA
> | FILE_WRITE_ATTRIBUTES);
> }
>
> // Setup our desired access
> nxtSp->Parameters.Create.SecurityContext->DesiredAccess =
> desiredAccessCreate & ~SYNCHRONIZE;
> nxtSp->Parameters.Create.ShareAccess = FILE_SHARE_VALID_FLAGS;
> // Specify open only (no create/supersede)
> nxtSp->Parameters.Create.Options = (FILE_OPEN << 24) |
> (createOptions &
> (FILE_OPEN_NO_RECALL | FILE_OPEN_FOR_BACKUP_INTENT |
>
> FILE_COMPLETE_IF_OPLOCKED | FILE_NON_DIRECTORY_FILE |
>
> FILE_DIRECTORY_FILE | FILE_OPEN_REPARSE_POINT)) |
>
> FILE_NO_INTERMEDIATE_BUFFERING;
>
> //
> // Chuck the request to the underlying driver
> //
> Status = IoCallDriver(DeviceObject, Irp);
>
> if (Status == STATUS_PENDING) {
> // Wait for event if pended
> KeWaitForSingleObject(&cloneFileObject->Event,
> Executive, KernelMode, FALSE, 0);
> Status = Irp->IoStatus.Status;
> }
>
> // Now check for file open failures
> if(!NT_SUCCESS(Status) || (Irp->IoStatus.Status ==
> STATUS_REPARSE)) {
> // Restore original IRP parameters
> irpSp->Parameters.Create.SecurityContext->DesiredAccess
> = desiredAccess;
>
> // Free temp (or FSD) buffer if still allocated
> if (pFOName->Buffer) {
> ExFreePool(pFOName->Buffer);
> pFOName->Buffer = NULL;
> pFOName->Length = 0;
> pFOName->MaximumLength = 0;
> }
>
> // Reset FO stuff
> cloneFileObject->Flags |= FO_STREAM_FILE;
> cloneFileObject->RelatedFileObject = NULL;
>
> // Toss temp FO
> ObDereferenceObject(cloneFileObject);
>
> // Nothing has happened yet
> Irp->PendingReturned = FALSE;
>
> // OK to proceed if cannot open file
> return STATUS_SUCCESS;
> }
>
> //
> // OK - we now have an opened FileObject with a valid FsContext
> pointer
> //
>
> // RetStatus = <<<
Insert your desired actions here >>>
>
> //
> //
Done with FileObject
> //
>
> // Free temp (or FSD) buffer if still allocated
> if (pFOName->Buffer) {
> ExFreePool(pFOName->Buffer);
> pFOName->Buffer = NULL;
> pFOName->Length = 0;
> pFOName->MaximumLength = 0;
> }
>
> // Need to light “handle created” flag
> cloneFileObject->Flags |= FO_HANDLE_CREATED;
>
> // Send a CLEANUP down to the FSD
> nxtSp->MajorFunction = IRP_MJ_CLEANUP;
> nxtSp->MinorFunction = 0;
> nxtSp->Flags = 0;
> nxtSp->Control = 0;
> nxtSp->DeviceObject = DeviceObject;
> nxtSp->FileObject = cloneFileObject;
>
> // Init our completion event
> KeInitializeEvent(&cloneFileObject->Event, NotificationEvent,
> FALSE);
>
> // Setup completion routine
> IoSetCompletionRoutine (Irp, xxxSimpleEventCompletion,
> &cloneFileObject->Event, TRUE, TRUE, TRUE);
>
> //
> // Call the FSD to release any share access, cache refs, etc.
> //
> Status = IoCallDriver(DeviceObject, Irp);
>
> //
> // Wait for the cleanup to finish
> //
> if (Status == STATUS_PENDING) {
> KeWaitForSingleObject(&cloneFileObject->Event,
> Executive, KernelMode, FALSE, 0);
> }
>
> // Reset FO stuff
> cloneFileObject->Flags |= FO_STREAM_FILE;
> cloneFileObject->RelatedFileObject = NULL;
>
> // We allocated the FO, now we must free it
> ObDereferenceObject(cloneFileObject);
>
> // Restore original IRP parameters
> irpSp->Parameters.Create.SecurityContext->DesiredAccess =
> desiredAccess;
>
> // Nothing has happened yet
> Irp->PendingReturned = FALSE;
> Irp->IoStatus.Status = STATUS_SUCCESS;
> Irp->IoStatus.Information = 0;
>
> // return STATUS_SUCCESS if it is OK to go ahead and let the caller
> open the file
> return RetStatus;
> }
>
> —
> You are currently subscribed to ntfsd as: xxxxx@bitarmor.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@livevault.com To unsubscribe
> send a blank email to xxxxx@lists.osr.com
>
> —
> You are currently subscribed to ntfsd as: xxxxx@arkivio.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
> —
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

Please correct me if I am wrong, according to the spirit of OSR online’s
paper about IoCancelOpen, the downside of doing this(reusing the same
fileobject) is that if you did any work(read,write, or compression) which
can potentially enable the cache as the first fileobject, then cache manager
will hold a reference to it and the manual CLOSE will not help here . If
the downstream driver fails second open with the same fileobject after you
restore the filename and forward the original CREATE, then the fileobject
can be deleted by the initial caller. Now we would have the cache manager
hold a reference to a deleted fileobject.

Paul

----- Original Message -----
From: “Maxim S. Shatskih”
To: “Windows File Systems Devs Interest List”
Sent: Friday, October 10, 2003 7:51 PM
Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
eateStreamFile

> There is also a following technique to open some file in filter’s
CREATE
> path:
>
> - save the original FileObject->FileName, RelatedFileObject and the
whole
> CREATE stack location
> - set up the new file name and the new CREATE stack location
> - pass the IRP down
> - work with the opened file
> - then handicraft CLEANUP and CLOSE (in the same IRP and the same
stack
> location) and send them down
> - then reset FO_CLEANUP_COMPLETE on the file object
> - then restore the saved CREATE stack location, FileName and
> RelatedFileObject, and pass the original CREATE down at last.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
> ----- Original Message -----
> From: “Paul Mu”
> To: “Windows File Systems Devs Interest List”
> Sent: Saturday, October 11, 2003 3:35 AM
> Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
> eateStreamFile
>
>
> > One thing I know for sure, FO_CLEANUP_COMPLETE does not intefere with
CREATE.
> >
> > Paul
> >
> > -----Original Message-----
> > From: Ted Hess [mailto:xxxxx@livevault.com]
> > Sent: Thursday, October 09, 2003 7:08 AM
> > To: Windows File Systems Devs Interest List
> > Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
> > eateStreamFile
> >
> >
> > Matt -
> >
> > The comment was really a reminder that these flags must be cleared. The
code
> > copies the flags from the original FileObject which does indeed have
these
> > flags absent. Perhaps the totally correct thing to do would be copy the
> > flags AND clear the unwanted ones.
> >
> > Ex: cloneFileObject->Flags = fileObject->Flags & ~(FO_STREAM_FILE |
> > FO_HANDLE_CREATED | FO_CLEANUP_COMPLETE);
> >
> > Also - I’m not sure, but I think the CREATE will fail if any of these
are
> > set.
> >
> > Thanks, /ted
> >
> >
> >
> > -----Original Message-----
> > From: Matthew N. White [mailto:xxxxx@bitarmor.com]
> > Sent: Wednesday, October 08, 2003 7:12 PM
> > To: Windows File Systems Devs Interest List
> > Subject: [ntfsd] RE: Postscript on closing FileObjects created by
> > IoCreateStreamFile
> >
> >
> > Ted,
> > Is there any reason that the line that clears the FO_STREAM_FILE,
> > FO_HANDLE_CREATED, FO_CLEANUP_COMPLETE flags in the code is commented
out?
> > Below you say that you “MUST clear the FO_STREAM_FILE, FO_HANDLE_CREATED
&
> > FO_CLEANUP_COMPLETE flags to successfully open this FO via a CREATE
IRP.”
> > Just wondering about that, since the code just copies the flags over.
> >
> > Matt
> >
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of Ted Hess
> > Sent: Tuesday, October 07, 2003 2:39 PM
> > To: Windows File Systems Devs Interest List
> > Subject: [ntfsd] Postscript on closing FileObjects created by
> > IoCreateStreamFile
> >
> >
> > I’ve been meaning to write up these observations for a couple of months
now
> > but just haven’t had the time. Many thanks to Tony and Nick for their
info
> > and help on this subject back in June when I was struggling with it.
> >
> > I basically followed the scenario outlined on the OSR on-line site and
the
> > latest NT Insider article with a couple of differences which I believe
to be
> > significant. In particular, NTFS seems to be very sensitive to the Flags
> > field in the FileObject.
> >
> > My first attempt essentially followed the steps outlined on the OSR web
> > site. However, I noticed I was leaking FileObjects!! Why was this? Well,
it
> > turns out the the CLEANUP IRP wasn’t really doing anything like it
should
> > have been and the ObDereferenceObject never issued the CLOSE. I was left
> > with a FO and a cache reference. Also, the file sharing state was
whacked
> > 'cause the CLEANUP didn’t reset it.
> >
> > After a bit of hair pulling and experimentation, I discovered that this
> > problem only happened on NTFS and not FAT. FAT does not care about the
same
> > FileObject states as does NTFS (at least from my observations of NTFS
> > behavior and code inspection in the FAT IFS kit sources). Here’s what I
> > think needs to happen:
> >
> > After creating the stream FO, you MUST clear the FO_STREAM_FILE,
> > FO_HANDLE_CREATED & FO_CLEANUP_COMPLETE flags to successfully open this
FO
> > via a CREATE IRP.
> >
> > If the CREATE fails, you need to set FO_STREAM_FILE before calling
> > ObDereferenctObject.
> >
> > If the CREATE succeeds, you will need to set FO_HANDLE_CREATED before
> > issuing the CLEANUP and set FO_STREAM_FILE before calling
> > ObDereferenceObject. Otherwise, things just don’t work as expected.
> >
> > Comments, mockery and/or derision welcome…
> >
> > /ted
> >
> > Attached is an abbreviated, and hopefully useful code snippet for
opening a
> > file and then successfully closing it in your pre-Create filter path.
> >
> >
> > NTSTATUS
> > xxxPreCreateCheck(PDEVICE_OBJECT targetDeviceObject, PIRP Irp) {
> > NTSTATUS Status, RetStatus =
> > STATUS_SUCCESS;
> > UNICODE_STRING fileName;
> > PUNICODE_STRING pFOName;
> > PFILE_OBJECT fileObject, cloneFileObject;
> >
> > PIO_STACK_LOCATION irpSp = IoGetCurrentIrpStackLocation(Irp);
> > PIO_STACK_LOCATION nxtSp = IoGetNextIrpStackLocation(Irp);
> >
> > ACCESS_MASK desiredAccess =
> > irpSp->Parameters.Create.SecurityContext->DesiredAccess;
> > ACCESS_MASK desiredAccessCreate = desiredAccess;
> >
> > ULONG createOptions = irpSp->Parameters.Create.Options &
> > FILE_VALID_OPTION_FLAGS;
> > ULONG createDisposition = (irpSp->Parameters.Create.Options >>
> > 24) & 0xFF;
> >
> > // The subject of this excercise (save it)
> > fileObject = irpSp->FileObject;
> >
> > // Make a copy of the callers buffer here 'cause we don’t know
> > who’ll free it
> > fileName.Length = fileObject->FileName.Length;
> > fileName.MaximumLength = fileObject->FileName.MaximumLength;
> > fileName.Buffer = ExAllocatePool(PagedPool, fileName.MaximumLength);
> >
> > // Bag it if no memory available
> > if (!fileName.Buffer) {
> > return STATUS_INSUFFICIENT_RESOURCES;
> > }
> > // Copy original contents into temp buffer
> > RtlCopyMemory(fileName.Buffer, fileObject->FileName.Buffer,
> > fileName.MaximumLength);
> >
> > // Allocate a new, temporary FO
> > #if defined(NT_40)
> > // Watch out for the gratuitous CLEANUP IRP!
> > cloneFileObject = IoCreateStreamFileObject(fileObject, NULL); #else
> > cloneFileObject = IoCreateStreamFileObjectLite(fileObject,
> > NULL); #endif
> >
> > // (Re-)init newly created FO
> > // cloneFileObject->Flags &= ~(FO_STREAM_FILE | FO_HANDLE_CREATED |
> > FO_CLEANUP_COMPLETE);
> > cloneFileObject->Flags = fileObject->Flags;
> > cloneFileObject->RelatedFileObject =
> > fileObject->RelatedFileObject;
> >
> > // Give it a name
> > pFOName = &cloneFileObject->FileName;
> > *pFOName = fileName;
> >
> > // Initialize embedded event structures
> > KeInitializeEvent(&cloneFileObject->Lock, SynchronizationEvent,
> > FALSE);
> > KeInitializeEvent(&cloneFileObject->Event, NotificationEvent,
> > FALSE);
> >
> > // Make a copy of the create irp (preserve flags)
> > RtlCopyMemory(nxtSp, irpSp, sizeof(IO_STACK_LOCATION));
> > nxtSp->Control = 0;
> > // Use temporary FO and underlying DO
> > nxtSp->FileObject = cloneFileObject;
> > nxtSp->DeviceObject = DeviceObject;
> >
> > // Setup completion routine
> > IoSetCompletionRoutine (Irp, xxxSimpleEventCompletion,
> > &cloneFileObject->Event, TRUE, TRUE, TRUE);
> >
> > //
> > // Ensure that the create disposition requested
> > // indicates all of the appropriate “rights” necessary
> > // for the given operation.
> > //
> > if (createDisposition == FILE_SUPERSEDE) {
> > desiredAccessCreate |= DELETE;
> > }
> >
> > if ((createDisposition == FILE_OVERWRITE) ||
> > (createDisposition == FILE_OVERWRITE_IF)) {
> > desiredAccessCreate |= (FILE_WRITE_DATA | FILE_WRITE_EA
> > | FILE_WRITE_ATTRIBUTES);
> > }
> >
> > // Setup our desired access
> > nxtSp->Parameters.Create.SecurityContext->DesiredAccess =
> > desiredAccessCreate & ~SYNCHRONIZE;
> > nxtSp->Parameters.Create.ShareAccess = FILE_SHARE_VALID_FLAGS;
> > // Specify open only (no create/supersede)
> > nxtSp->Parameters.Create.Options = (FILE_OPEN << 24) |
> > (createOptions &
> > (FILE_OPEN_NO_RECALL | FILE_OPEN_FOR_BACKUP_INTENT |
> >
> > FILE_COMPLETE_IF_OPLOCKED | FILE_NON_DIRECTORY_FILE |
> >
> > FILE_DIRECTORY_FILE | FILE_OPEN_REPARSE_POINT)) |
> >
> > FILE_NO_INTERMEDIATE_BUFFERING;
> >
> > //
> > // Chuck the request to the underlying driver
> > //
> > Status = IoCallDriver(DeviceObject, Irp);
> >
> > if (Status == STATUS_PENDING) {
> > // Wait for event if pended
> > KeWaitForSingleObject(&cloneFileObject->Event,
> > Executive, KernelMode, FALSE, 0);
> > Status = Irp->IoStatus.Status;
> > }
> >
> > // Now check for file open failures
> > if(!NT_SUCCESS(Status) || (Irp->IoStatus.Status ==
> > STATUS_REPARSE)) {
> > // Restore original IRP parameters
> > irpSp->Parameters.Create.SecurityContext->DesiredAccess
> > = desiredAccess;
> >
> > // Free temp (or FSD) buffer if still allocated
> > if (pFOName->Buffer) {
> > ExFreePool(pFOName->Buffer);
> > pFOName->Buffer = NULL;
> > pFOName->Length = 0;
> > pFOName->MaximumLength = 0;
> > }
> >
> > // Reset FO stuff
> > cloneFileObject->Flags |= FO_STREAM_FILE;
> > cloneFileObject->RelatedFileObject = NULL;
> >
> > // Toss temp FO
> > ObDereferenceObject(cloneFileObject);
> >
> > // Nothing has happened yet
> > Irp->PendingReturned = FALSE;
> >
> > // OK to proceed if cannot open file
> > return STATUS_SUCCESS;
> > }
> >
> > //
> > // OK - we now have an opened FileObject with a valid FsContext
> > pointer
> > //
> >
> > // RetStatus = <<<
Insert your desired actions here >>>
> >
> > //
> > //
Done with FileObject
> > //
> >
> > // Free temp (or FSD) buffer if still allocated
> > if (pFOName->Buffer) {
> > ExFreePool(pFOName->Buffer);
> > pFOName->Buffer = NULL;
> > pFOName->Length = 0;
> > pFOName->MaximumLength = 0;
> > }
> >
> > // Need to light “handle created” flag
> > cloneFileObject->Flags |= FO_HANDLE_CREATED;
> >
> > // Send a CLEANUP down to the FSD
> > nxtSp->MajorFunction = IRP_MJ_CLEANUP;
> > nxtSp->MinorFunction = 0;
> > nxtSp->Flags = 0;
> > nxtSp->Control = 0;
> > nxtSp->DeviceObject = DeviceObject;
> > nxtSp->FileObject = cloneFileObject;
> >
> > // Init our completion event
> > KeInitializeEvent(&cloneFileObject->Event, NotificationEvent,
> > FALSE);
> >
> > // Setup completion routine
> > IoSetCompletionRoutine (Irp, xxxSimpleEventCompletion,
> > &cloneFileObject->Event, TRUE, TRUE, TRUE);
> >
> > //
> > // Call the FSD to release any share access, cache refs, etc.
> > //
> > Status = IoCallDriver(DeviceObject, Irp);
> >
> > //
> > // Wait for the cleanup to finish
> > //
> > if (Status == STATUS_PENDING) {
> > KeWaitForSingleObject(&cloneFileObject->Event,
> > Executive, KernelMode, FALSE, 0);
> > }
> >
> > // Reset FO stuff
> > cloneFileObject->Flags |= FO_STREAM_FILE;
> > cloneFileObject->RelatedFileObject = NULL;
> >
> > // We allocated the FO, now we must free it
> > ObDereferenceObject(cloneFileObject);
> >
> > // Restore original IRP parameters
> > irpSp->Parameters.Create.SecurityContext->DesiredAccess =
> > desiredAccess;
> >
> > // Nothing has happened yet
> > Irp->PendingReturned = FALSE;
> > Irp->IoStatus.Status = STATUS_SUCCESS;
> > Irp->IoStatus.Information = 0;
> >
> > // return STATUS_SUCCESS if it is OK to go ahead and let the caller
> > open the file
> > return RetStatus;
> > }
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@bitarmor.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe
> > send a blank email to xxxxx@lists.osr.com
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@arkivio.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@arkivio.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

Yes, but I open my first file object with no caching.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Paul Mu”
To: “Windows File Systems Devs Interest List”
Sent: Saturday, October 11, 2003 11:05 AM
Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
eateStreamFile

> Please correct me if I am wrong, according to the spirit of OSR online’s
> paper about IoCancelOpen, the downside of doing this(reusing the same
> fileobject) is that if you did any work(read,write, or compression) which
> can potentially enable the cache as the first fileobject, then cache manager
> will hold a reference to it and the manual CLOSE will not help here . If
> the downstream driver fails second open with the same fileobject after you
> restore the filename and forward the original CREATE, then the fileobject
> can be deleted by the initial caller. Now we would have the cache manager
> hold a reference to a deleted fileobject.
>
> Paul
>
> ----- Original Message -----
> From: “Maxim S. Shatskih”
> To: “Windows File Systems Devs Interest List”
> Sent: Friday, October 10, 2003 7:51 PM
> Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
> eateStreamFile
>
>
> > There is also a following technique to open some file in filter’s
> CREATE
> > path:
> >
> > - save the original FileObject->FileName, RelatedFileObject and the
> whole
> > CREATE stack location
> > - set up the new file name and the new CREATE stack location
> > - pass the IRP down
> > - work with the opened file
> > - then handicraft CLEANUP and CLOSE (in the same IRP and the same
> stack
> > location) and send them down
> > - then reset FO_CLEANUP_COMPLETE on the file object
> > - then restore the saved CREATE stack location, FileName and
> > RelatedFileObject, and pass the original CREATE down at last.
> >
> > Maxim Shatskih, Windows DDK MVP
> > StorageCraft Corporation
> > xxxxx@storagecraft.com
> > http://www.storagecraft.com
> >
> >
> > ----- Original Message -----
> > From: “Paul Mu”
> > To: “Windows File Systems Devs Interest List”
> > Sent: Saturday, October 11, 2003 3:35 AM
> > Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
> > eateStreamFile
> >
> >
> > > One thing I know for sure, FO_CLEANUP_COMPLETE does not intefere with
> CREATE.
> > >
> > > Paul
> > >
> > > -----Original Message-----
> > > From: Ted Hess [mailto:xxxxx@livevault.com]
> > > Sent: Thursday, October 09, 2003 7:08 AM
> > > To: Windows File Systems Devs Interest List
> > > Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
> > > eateStreamFile
> > >
> > >
> > > Matt -
> > >
> > > The comment was really a reminder that these flags must be cleared. The
> code
> > > copies the flags from the original FileObject which does indeed have
> these
> > > flags absent. Perhaps the totally correct thing to do would be copy the
> > > flags AND clear the unwanted ones.
> > >
> > > Ex: cloneFileObject->Flags = fileObject->Flags & ~(FO_STREAM_FILE |
> > > FO_HANDLE_CREATED | FO_CLEANUP_COMPLETE);
> > >
> > > Also - I’m not sure, but I think the CREATE will fail if any of these
> are
> > > set.
> > >
> > > Thanks, /ted
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Matthew N. White [mailto:xxxxx@bitarmor.com]
> > > Sent: Wednesday, October 08, 2003 7:12 PM
> > > To: Windows File Systems Devs Interest List
> > > Subject: [ntfsd] RE: Postscript on closing FileObjects created by
> > > IoCreateStreamFile
> > >
> > >
> > > Ted,
> > > Is there any reason that the line that clears the FO_STREAM_FILE,
> > > FO_HANDLE_CREATED, FO_CLEANUP_COMPLETE flags in the code is commented
> out?
> > > Below you say that you “MUST clear the FO_STREAM_FILE, FO_HANDLE_CREATED

> &
> > > FO_CLEANUP_COMPLETE flags to successfully open this FO via a CREATE
> IRP.”
> > > Just wondering about that, since the code just copies the flags over.
> > >
> > > Matt
> > >
> > > -----Original Message-----
> > > From: xxxxx@lists.osr.com
> > > [mailto:xxxxx@lists.osr.com] On Behalf Of Ted Hess
> > > Sent: Tuesday, October 07, 2003 2:39 PM
> > > To: Windows File Systems Devs Interest List
> > > Subject: [ntfsd] Postscript on closing FileObjects created by
> > > IoCreateStreamFile
> > >
> > >
> > > I’ve been meaning to write up these observations for a couple of months
> now
> > > but just haven’t had the time. Many thanks to Tony and Nick for their
> info
> > > and help on this subject back in June when I was struggling with it.
> > >
> > > I basically followed the scenario outlined on the OSR on-line site and
> the
> > > latest NT Insider article with a couple of differences which I believe
> to be
> > > significant. In particular, NTFS seems to be very sensitive to the Flags
> > > field in the FileObject.
> > >
> > > My first attempt essentially followed the steps outlined on the OSR web
> > > site. However, I noticed I was leaking FileObjects!! Why was this? Well,
> it
> > > turns out the the CLEANUP IRP wasn’t really doing anything like it
> should
> > > have been and the ObDereferenceObject never issued the CLOSE. I was left
> > > with a FO and a cache reference. Also, the file sharing state was
> whacked
> > > 'cause the CLEANUP didn’t reset it.
> > >
> > > After a bit of hair pulling and experimentation, I discovered that this
> > > problem only happened on NTFS and not FAT. FAT does not care about the
> same
> > > FileObject states as does NTFS (at least from my observations of NTFS
> > > behavior and code inspection in the FAT IFS kit sources). Here’s what I
> > > think needs to happen:
> > >
> > > After creating the stream FO, you MUST clear the FO_STREAM_FILE,
> > > FO_HANDLE_CREATED & FO_CLEANUP_COMPLETE flags to successfully open this
> FO
> > > via a CREATE IRP.
> > >
> > > If the CREATE fails, you need to set FO_STREAM_FILE before calling
> > > ObDereferenctObject.
> > >
> > > If the CREATE succeeds, you will need to set FO_HANDLE_CREATED before
> > > issuing the CLEANUP and set FO_STREAM_FILE before calling
> > > ObDereferenceObject. Otherwise, things just don’t work as expected.
> > >
> > > Comments, mockery and/or derision welcome…
> > >
> > > /ted
> > >
> > > Attached is an abbreviated, and hopefully useful code snippet for
> opening a
> > > file and then successfully closing it in your pre-Create filter path.
> > >
> > >
> > > NTSTATUS
> > > xxxPreCreateCheck(PDEVICE_OBJECT targetDeviceObject, PIRP Irp) {
> > > NTSTATUS Status, RetStatus =
> > > STATUS_SUCCESS;
> > > UNICODE_STRING fileName;
> > > PUNICODE_STRING pFOName;
> > > PFILE_OBJECT fileObject, cloneFileObject;
> > >
> > > PIO_STACK_LOCATION irpSp = IoGetCurrentIrpStackLocation(Irp);
> > > PIO_STACK_LOCATION nxtSp = IoGetNextIrpStackLocation(Irp);
> > >
> > > ACCESS_MASK desiredAccess =
> > > irpSp->Parameters.Create.SecurityContext->DesiredAccess;
> > > ACCESS_MASK desiredAccessCreate = desiredAccess;
> > >
> > > ULONG createOptions = irpSp->Parameters.Create.Options &
> > > FILE_VALID_OPTION_FLAGS;
> > > ULONG createDisposition = (irpSp->Parameters.Create.Options >>
> > > 24) & 0xFF;
> > >
> > > // The subject of this excercise (save it)
> > > fileObject = irpSp->FileObject;
> > >
> > > // Make a copy of the callers buffer here 'cause we don’t know
> > > who’ll free it
> > > fileName.Length = fileObject->FileName.Length;
> > > fileName.MaximumLength = fileObject->FileName.MaximumLength;
> > > fileName.Buffer = ExAllocatePool(PagedPool, fileName.MaximumLength);
> > >
> > > // Bag it if no memory available
> > > if (!fileName.Buffer) {
> > > return STATUS_INSUFFICIENT_RESOURCES;
> > > }
> > > // Copy original contents into temp buffer
> > > RtlCopyMemory(fileName.Buffer, fileObject->FileName.Buffer,
> > > fileName.MaximumLength);
> > >
> > > // Allocate a new, temporary FO
> > > #if defined(NT_40)
> > > // Watch out for the gratuitous CLEANUP IRP!
> > > cloneFileObject = IoCreateStreamFileObject(fileObject, NULL); #else
> > > cloneFileObject = IoCreateStreamFileObjectLite(fileObject,
> > > NULL); #endif
> > >
> > > // (Re-)init newly created FO
> > > // cloneFileObject->Flags &= ~(FO_STREAM_FILE | FO_HANDLE_CREATED |
> > > FO_CLEANUP_COMPLETE);
> > > cloneFileObject->Flags = fileObject->Flags;
> > > cloneFileObject->RelatedFileObject =
> > > fileObject->RelatedFileObject;
> > >
> > > // Give it a name
> > > pFOName = &cloneFileObject->FileName;
> > > *pFOName = fileName;
> > >
> > > // Initialize embedded event structures
> > > KeInitializeEvent(&cloneFileObject->Lock, SynchronizationEvent,
> > > FALSE);
> > > KeInitializeEvent(&cloneFileObject->Event, NotificationEvent,
> > > FALSE);
> > >
> > > // Make a copy of the create irp (preserve flags)
> > > RtlCopyMemory(nxtSp, irpSp, sizeof(IO_STACK_LOCATION));
> > > nxtSp->Control = 0;
> > > // Use temporary FO and underlying DO
> > > nxtSp->FileObject = cloneFileObject;
> > > nxtSp->DeviceObject = DeviceObject;
> > >
> > > // Setup completion routine
> > > IoSetCompletionRoutine (Irp, xxxSimpleEventCompletion,
> > > &cloneFileObject->Event, TRUE, TRUE, TRUE);
> > >
> > > //
> > > // Ensure that the create disposition requested
> > > // indicates all of the appropriate “rights” necessary
> > > // for the given operation.
> > > //
> > > if (createDisposition == FILE_SUPERSEDE) {
> > > desiredAccessCreate |= DELETE;
> > > }
> > >
> > > if ((createDisposition == FILE_OVERWRITE) ||
> > > (createDisposition == FILE_OVERWRITE_IF)) {
> > > desiredAccessCreate |= (FILE_WRITE_DATA | FILE_WRITE_EA
> > > | FILE_WRITE_ATTRIBUTES);
> > > }
> > >
> > > // Setup our desired access
> > > nxtSp->Parameters.Create.SecurityContext->DesiredAccess =
> > > desiredAccessCreate & ~SYNCHRONIZE;
> > > nxtSp->Parameters.Create.ShareAccess = FILE_SHARE_VALID_FLAGS;
> > > // Specify open only (no create/supersede)
> > > nxtSp->Parameters.Create.Options = (FILE_OPEN << 24) |
> > > (createOptions &
> > > (FILE_OPEN_NO_RECALL | FILE_OPEN_FOR_BACKUP_INTENT |
> > >
> > > FILE_COMPLETE_IF_OPLOCKED | FILE_NON_DIRECTORY_FILE |
> > >
> > > FILE_DIRECTORY_FILE | FILE_OPEN_REPARSE_POINT)) |
> > >
> > > FILE_NO_INTERMEDIATE_BUFFERING;
> > >
> > > //
> > > // Chuck the request to the underlying driver
> > > //
> > > Status = IoCallDriver(DeviceObject, Irp);
> > >
> > > if (Status == STATUS_PENDING) {
> > > // Wait for event if pended
> > > KeWaitForSingleObject(&cloneFileObject->Event,
> > > Executive, KernelMode, FALSE, 0);
> > > Status = Irp->IoStatus.Status;
> > > }
> > >
> > > // Now check for file open failures
> > > if(!NT_SUCCESS(Status) || (Irp->IoStatus.Status ==
> > > STATUS_REPARSE)) {
> > > // Restore original IRP parameters
> > > irpSp->Parameters.Create.SecurityContext->DesiredAccess
> > > = desiredAccess;
> > >
> > > // Free temp (or FSD) buffer if still allocated
> > > if (pFOName->Buffer) {
> > > ExFreePool(pFOName->Buffer);
> > > pFOName->Buffer = NULL;
> > > pFOName->Length = 0;
> > > pFOName->MaximumLength = 0;
> > > }
> > >
> > > // Reset FO stuff
> > > cloneFileObject->Flags |= FO_STREAM_FILE;
> > > cloneFileObject->RelatedFileObject = NULL;
> > >
> > > // Toss temp FO
> > > ObDereferenceObject(cloneFileObject);
> > >
> > > // Nothing has happened yet
> > > Irp->PendingReturned = FALSE;
> > >
> > > // OK to proceed if cannot open file
> > > return STATUS_SUCCESS;
> > > }
> > >
> > > //
> > > // OK - we now have an opened FileObject with a valid FsContext
> > > pointer
> > > //
> > >
> > > // RetStatus = <<<
Insert your desired actions here >>>
> > >
> > > //
> > > //
Done with FileObject
> > > //
> > >
> > > // Free temp (or FSD) buffer if still allocated
> > > if (pFOName->Buffer) {
> > > ExFreePool(pFOName->Buffer);
> > > pFOName->Buffer = NULL;
> > > pFOName->Length = 0;
> > > pFOName->MaximumLength = 0;
> > > }
> > >
> > > // Need to light “handle created” flag
> > > cloneFileObject->Flags |= FO_HANDLE_CREATED;
> > >
> > > // Send a CLEANUP down to the FSD
> > > nxtSp->MajorFunction = IRP_MJ_CLEANUP;
> > > nxtSp->MinorFunction = 0;
> > > nxtSp->Flags = 0;
> > > nxtSp->Control = 0;
> > > nxtSp->DeviceObject = DeviceObject;
> > > nxtSp->FileObject = cloneFileObject;
> > >
> > > // Init our completion event
> > > KeInitializeEvent(&cloneFileObject->Event, NotificationEvent,
> > > FALSE);
> > >
> > > // Setup completion routine
> > > IoSetCompletionRoutine (Irp, xxxSimpleEventCompletion,
> > > &cloneFileObject->Event, TRUE, TRUE, TRUE);
> > >
> > > //
> > > // Call the FSD to release any share access, cache refs, etc.
> > > //
> > > Status = IoCallDriver(DeviceObject, Irp);
> > >
> > > //
> > > // Wait for the cleanup to finish
> > > //
> > > if (Status == STATUS_PENDING) {
> > > KeWaitForSingleObject(&cloneFileObject->Event,
> > > Executive, KernelMode, FALSE, 0);
> > > }
> > >
> > > // Reset FO stuff
> > > cloneFileObject->Flags |= FO_STREAM_FILE;
> > > cloneFileObject->RelatedFileObject = NULL;
> > >
> > > // We allocated the FO, now we must free it
> > > ObDereferenceObject(cloneFileObject);
> > >
> > > // Restore original IRP parameters
> > > irpSp->Parameters.Create.SecurityContext->DesiredAccess =
> > > desiredAccess;
> > >
> > > // Nothing has happened yet
> > > Irp->PendingReturned = FALSE;
> > > Irp->IoStatus.Status = STATUS_SUCCESS;
> > > Irp->IoStatus.Information = 0;
> > >
> > > // return STATUS_SUCCESS if it is OK to go ahead and let the caller
> > > open the file
> > > return RetStatus;
> > > }
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@bitarmor.com
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@livevault.com To
> unsubscribe
> > > send a blank email to xxxxx@lists.osr.com
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@arkivio.com
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > >
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@arkivio.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

I see now. But is no caching fileobject(file opened with no intermediate
buffering) really honored for the no-caching all the time, for instance,
for a compressed file.

Paul
----- Original Message -----
From: “Maxim S. Shatskih”
To: “Windows File Systems Devs Interest List”
Sent: Saturday, October 11, 2003 8:54 PM
Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
eateStreamFile

> Yes, but I open my first file object with no caching.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
> ----- Original Message -----
> From: “Paul Mu”
> To: “Windows File Systems Devs Interest List”
> Sent: Saturday, October 11, 2003 11:05 AM
> Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
> eateStreamFile
>
>
> > Please correct me if I am wrong, according to the spirit of OSR
online’s
> > paper about IoCancelOpen, the downside of doing this(reusing the same
> > fileobject) is that if you did any work(read,write, or compression)
which
> > can potentially enable the cache as the first fileobject, then cache
manager
> > will hold a reference to it and the manual CLOSE will not help here .
If
> > the downstream driver fails second open with the same fileobject after
you
> > restore the filename and forward the original CREATE, then the
fileobject
> > can be deleted by the initial caller. Now we would have the cache
manager
> > hold a reference to a deleted fileobject.
> >
> > Paul
> >
> > ----- Original Message -----
> > From: “Maxim S. Shatskih”
> > To: “Windows File Systems Devs Interest List”
> > Sent: Friday, October 10, 2003 7:51 PM
> > Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
> > eateStreamFile
> >
> >
> > > There is also a following technique to open some file in filter’s
> > CREATE
> > > path:
> > >
> > > - save the original FileObject->FileName, RelatedFileObject and
the
> > whole
> > > CREATE stack location
> > > - set up the new file name and the new CREATE stack location
> > > - pass the IRP down
> > > - work with the opened file
> > > - then handicraft CLEANUP and CLOSE (in the same IRP and the same
> > stack
> > > location) and send them down
> > > - then reset FO_CLEANUP_COMPLETE on the file object
> > > - then restore the saved CREATE stack location, FileName and
> > > RelatedFileObject, and pass the original CREATE down at last.
> > >
> > > Maxim Shatskih, Windows DDK MVP
> > > StorageCraft Corporation
> > > xxxxx@storagecraft.com
> > > http://www.storagecraft.com
> > >
> > >
> > > ----- Original Message -----
> > > From: “Paul Mu”
> > > To: “Windows File Systems Devs Interest List”
> > > Sent: Saturday, October 11, 2003 3:35 AM
> > > Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
> > > eateStreamFile
> > >
> > >
> > > > One thing I know for sure, FO_CLEANUP_COMPLETE does not intefere
with
> > CREATE.
> > > >
> > > > Paul
> > > >
> > > > -----Original Message-----
> > > > From: Ted Hess [mailto:xxxxx@livevault.com]
> > > > Sent: Thursday, October 09, 2003 7:08 AM
> > > > To: Windows File Systems Devs Interest List
> > > > Subject: [ntfsd] RE: Postscript on closing FileObjects created by
IoCr
> > > > eateStreamFile
> > > >
> > > >
> > > > Matt -
> > > >
> > > > The comment was really a reminder that these flags must be cleared.
The
> > code
> > > > copies the flags from the original FileObject which does indeed have
> > these
> > > > flags absent. Perhaps the totally correct thing to do would be copy
the
> > > > flags AND clear the unwanted ones.
> > > >
> > > > Ex: cloneFileObject->Flags = fileObject->Flags & ~(FO_STREAM_FILE |
> > > > FO_HANDLE_CREATED | FO_CLEANUP_COMPLETE);
> > > >
> > > > Also - I’m not sure, but I think the CREATE will fail if any of
these
> > are
> > > > set.
> > > >
> > > > Thanks, /ted
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Matthew N. White [mailto:xxxxx@bitarmor.com]
> > > > Sent: Wednesday, October 08, 2003 7:12 PM
> > > > To: Windows File Systems Devs Interest List
> > > > Subject: [ntfsd] RE: Postscript on closing FileObjects created by
> > > > IoCreateStreamFile
> > > >
> > > >
> > > > Ted,
> > > > Is there any reason that the line that clears the FO_STREAM_FILE,
> > > > FO_HANDLE_CREATED, FO_CLEANUP_COMPLETE flags in the code is
commented
> > out?
> > > > Below you say that you “MUST clear the FO_STREAM_FILE,
FO_HANDLE_CREATED
>
> > &
> > > > FO_CLEANUP_COMPLETE flags to successfully open this FO via a CREATE
> > IRP.”
> > > > Just wondering about that, since the code just copies the flags
over.
> > > >
> > > > Matt
> > > >
> > > > -----Original Message-----
> > > > From: xxxxx@lists.osr.com
> > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Ted Hess
> > > > Sent: Tuesday, October 07, 2003 2:39 PM
> > > > To: Windows File Systems Devs Interest List
> > > > Subject: [ntfsd] Postscript on closing FileObjects created by
> > > > IoCreateStreamFile
> > > >
> > > >
> > > > I’ve been meaning to write up these observations for a couple of
months
> > now
> > > > but just haven’t had the time. Many thanks to Tony and Nick for
their
> > info
> > > > and help on this subject back in June when I was struggling with it.
> > > >
> > > > I basically followed the scenario outlined on the OSR on-line site
and
> > the
> > > > latest NT Insider article with a couple of differences which I
believe
> > to be
> > > > significant. In particular, NTFS seems to be very sensitive to the
Flags
> > > > field in the FileObject.
> > > >
> > > > My first attempt essentially followed the steps outlined on the OSR
web
> > > > site. However, I noticed I was leaking FileObjects!! Why was this?
Well,
> > it
> > > > turns out the the CLEANUP IRP wasn’t really doing anything like it
> > should
> > > > have been and the ObDereferenceObject never issued the CLOSE. I was
left
> > > > with a FO and a cache reference. Also, the file sharing state was
> > whacked
> > > > 'cause the CLEANUP didn’t reset it.
> > > >
> > > > After a bit of hair pulling and experimentation, I discovered that
this
> > > > problem only happened on NTFS and not FAT. FAT does not care about
the
> > same
> > > > FileObject states as does NTFS (at least from my observations of
NTFS
> > > > behavior and code inspection in the FAT IFS kit sources). Here’s
what I
> > > > think needs to happen:
> > > >
> > > > After creating the stream FO, you MUST clear the FO_STREAM_FILE,
> > > > FO_HANDLE_CREATED & FO_CLEANUP_COMPLETE flags to successfully open
this
> > FO
> > > > via a CREATE IRP.
> > > >
> > > > If the CREATE fails, you need to set FO_STREAM_FILE before calling
> > > > ObDereferenctObject.
> > > >
> > > > If the CREATE succeeds, you will need to set FO_HANDLE_CREATED
before
> > > > issuing the CLEANUP and set FO_STREAM_FILE before calling
> > > > ObDereferenceObject. Otherwise, things just don’t work as expected.
> > > >
> > > > Comments, mockery and/or derision welcome…
> > > >
> > > > /ted
> > > >
> > > > Attached is an abbreviated, and hopefully useful code snippet for
> > opening a
> > > > file and then successfully closing it in your pre-Create filter
path.
> > > >
> > > >
> > > > NTSTATUS
> > > > xxxPreCreateCheck(PDEVICE_OBJECT targetDeviceObject, PIRP Irp) {
> > > > NTSTATUS Status, RetStatus =
> > > > STATUS_SUCCESS;
> > > > UNICODE_STRING fileName;
> > > > PUNICODE_STRING pFOName;
> > > > PFILE_OBJECT fileObject, cloneFileObject;
> > > >
> > > > PIO_STACK_LOCATION irpSp = IoGetCurrentIrpStackLocation(Irp);
> > > > PIO_STACK_LOCATION nxtSp = IoGetNextIrpStackLocation(Irp);
> > > >
> > > > ACCESS_MASK desiredAccess =
> > > > irpSp->Parameters.Create.SecurityContext->DesiredAccess;
> > > > ACCESS_MASK desiredAccessCreate = desiredAccess;
> > > >
> > > > ULONG createOptions = irpSp->Parameters.Create.Options &
> > > > FILE_VALID_OPTION_FLAGS;
> > > > ULONG createDisposition = (irpSp->Parameters.Create.Options >>
> > > > 24) & 0xFF;
> > > >
> > > > // The subject of this excercise (save it)
> > > > fileObject = irpSp->FileObject;
> > > >
> > > > // Make a copy of the callers buffer here 'cause we don’t know
> > > > who’ll free it
> > > > fileName.Length = fileObject->FileName.Length;
> > > > fileName.MaximumLength = fileObject->FileName.MaximumLength;
> > > > fileName.Buffer = ExAllocatePool(PagedPool, fileName.MaximumLength);
> > > >
> > > > // Bag it if no memory available
> > > > if (!fileName.Buffer) {
> > > > return STATUS_INSUFFICIENT_RESOURCES;
> > > > }
> > > > // Copy original contents into temp buffer
> > > > RtlCopyMemory(fileName.Buffer, fileObject->FileName.Buffer,
> > > > fileName.MaximumLength);
> > > >
> > > > // Allocate a new, temporary FO
> > > > #if defined(NT_40)
> > > > // Watch out for the gratuitous CLEANUP IRP!
> > > > cloneFileObject = IoCreateStreamFileObject(fileObject, NULL); #else
> > > > cloneFileObject = IoCreateStreamFileObjectLite(fileObject,
> > > > NULL); #endif
> > > >
> > > > // (Re-)init newly created FO
> > > > // cloneFileObject->Flags &= ~(FO_STREAM_FILE | FO_HANDLE_CREATED |
> > > > FO_CLEANUP_COMPLETE);
> > > > cloneFileObject->Flags = fileObject->Flags;
> > > > cloneFileObject->RelatedFileObject =
> > > > fileObject->RelatedFileObject;
> > > >
> > > > // Give it a name
> > > > pFOName = &cloneFileObject->FileName;
> > > > *pFOName = fileName;
> > > >
> > > > // Initialize embedded event structures
> > > > KeInitializeEvent(&cloneFileObject->Lock, SynchronizationEvent,
> > > > FALSE);
> > > > KeInitializeEvent(&cloneFileObject->Event, NotificationEvent,
> > > > FALSE);
> > > >
> > > > // Make a copy of the create irp (preserve flags)
> > > > RtlCopyMemory(nxtSp, irpSp, sizeof(IO_STACK_LOCATION));
> > > > nxtSp->Control = 0;
> > > > // Use temporary FO and underlying DO
> > > > nxtSp->FileObject = cloneFileObject;
> > > > nxtSp->DeviceObject = DeviceObject;
> > > >
> > > > // Setup completion routine
> > > > IoSetCompletionRoutine (Irp, xxxSimpleEventCompletion,
> > > > &cloneFileObject->Event, TRUE, TRUE, TRUE);
> > > >
> > > > //
> > > > // Ensure that the create disposition requested
> > > > // indicates all of the appropriate “rights” necessary
> > > > // for the given operation.
> > > > //
> > > > if (createDisposition == FILE_SUPERSEDE) {
> > > > desiredAccessCreate |= DELETE;
> > > > }
> > > >
> > > > if ((createDisposition == FILE_OVERWRITE) ||
> > > > (createDisposition == FILE_OVERWRITE_IF)) {
> > > > desiredAccessCreate |= (FILE_WRITE_DATA | FILE_WRITE_EA
> > > > | FILE_WRITE_ATTRIBUTES);
> > > > }
> > > >
> > > > // Setup our desired access
> > > > nxtSp->Parameters.Create.SecurityContext->DesiredAccess =
> > > > desiredAccessCreate & ~SYNCHRONIZE;
> > > > nxtSp->Parameters.Create.ShareAccess = FILE_SHARE_VALID_FLAGS;
> > > > // Specify open only (no create/supersede)
> > > > nxtSp->Parameters.Create.Options = (FILE_OPEN << 24) |
> > > > (createOptions &
> > > > (FILE_OPEN_NO_RECALL | FILE_OPEN_FOR_BACKUP_INTENT |
> > > >
> > > > FILE_COMPLETE_IF_OPLOCKED | FILE_NON_DIRECTORY_FILE |
> > > >
> > > > FILE_DIRECTORY_FILE | FILE_OPEN_REPARSE_POINT)) |
> > > >
> > > > FILE_NO_INTERMEDIATE_BUFFERING;
> > > >
> > > > //
> > > > // Chuck the request to the underlying driver
> > > > //
> > > > Status = IoCallDriver(DeviceObject, Irp);
> > > >
> > > > if (Status == STATUS_PENDING) {
> > > > // Wait for event if pended
> > > > KeWaitForSingleObject(&cloneFileObject->Event,
> > > > Executive, KernelMode, FALSE, 0);
> > > > Status = Irp->IoStatus.Status;
> > > > }
> > > >
> > > > // Now check for file open failures
> > > > if(!NT_SUCCESS(Status) || (Irp->IoStatus.Status ==
> > > > STATUS_REPARSE)) {
> > > > // Restore original IRP parameters
> > > > irpSp->Parameters.Create.SecurityContext->DesiredAccess
> > > > = desiredAccess;
> > > >
> > > > // Free temp (or FSD) buffer if still allocated
> > > > if (pFOName->Buffer) {
> > > > ExFreePool(pFOName->Buffer);
> > > > pFOName->Buffer = NULL;
> > > > pFOName->Length = 0;
> > > > pFOName->MaximumLength = 0;
> > > > }
> > > >
> > > > // Reset FO stuff
> > > > cloneFileObject->Flags |= FO_STREAM_FILE;
> > > > cloneFileObject->RelatedFileObject = NULL;
> > > >
> > > > // Toss temp FO
> > > > ObDereferenceObject(cloneFileObject);
> > > >
> > > > // Nothing has happened yet
> > > > Irp->PendingReturned = FALSE;
> > > >
> > > > // OK to proceed if cannot open file
> > > > return STATUS_SUCCESS;
> > > > }
> > > >
> > > > //
> > > > // OK - we now have an opened FileObject with a valid FsContext
> > > > pointer
> > > > //
> > > >
> > > > // RetStatus = <<<
Insert your desired actions here >>>
> > > >
> > > > //
> > > > //
Done with FileObject
> > > > //
> > > >
> > > > // Free temp (or FSD) buffer if still allocated
> > > > if (pFOName->Buffer) {
> > > > ExFreePool(pFOName->Buffer);
> > > > pFOName->Buffer = NULL;
> > > > pFOName->Length = 0;
> > > > pFOName->MaximumLength = 0;
> > > > }
> > > >
> > > > // Need to light “handle created” flag
> > > > cloneFileObject->Flags |= FO_HANDLE_CREATED;
> > > >
> > > > // Send a CLEANUP down to the FSD
> > > > nxtSp->MajorFunction = IRP_MJ_CLEANUP;
> > > > nxtSp->MinorFunction = 0;
> > > > nxtSp->Flags = 0;
> > > > nxtSp->Control = 0;
> > > > nxtSp->DeviceObject = DeviceObject;
> > > > nxtSp->FileObject = cloneFileObject;
> > > >
> > > > // Init our completion event
> > > > KeInitializeEvent(&cloneFileObject->Event, NotificationEvent,
> > > > FALSE);
> > > >
> > > > // Setup completion routine
> > > > IoSetCompletionRoutine (Irp, xxxSimpleEventCompletion,
> > > > &cloneFileObject->Event, TRUE, TRUE, TRUE);
> > > >
> > > > //
> > > > // Call the FSD to release any share access, cache refs, etc.
> > > > //
> > > > Status = IoCallDriver(DeviceObject, Irp);
> > > >
> > > > //
> > > > // Wait for the cleanup to finish
> > > > //
> > > > if (Status == STATUS_PENDING) {
> > > > KeWaitForSingleObject(&cloneFileObject->Event,
> > > > Executive, KernelMode, FALSE, 0);
> > > > }
> > > >
> > > > // Reset FO stuff
> > > > cloneFileObject->Flags |= FO_STREAM_FILE;
> > > > cloneFileObject->RelatedFileObject = NULL;
> > > >
> > > > // We allocated the FO, now we must free it
> > > > ObDereferenceObject(cloneFileObject);
> > > >
> > > > // Restore original IRP parameters
> > > > irpSp->Parameters.Create.SecurityContext->DesiredAccess =
> > > > desiredAccess;
> > > >
> > > > // Nothing has happened yet
> > > > Irp->PendingReturned = FALSE;
> > > > Irp->IoStatus.Status = STATUS_SUCCESS;
> > > > Irp->IoStatus.Information = 0;
> > > >
> > > > // return STATUS_SUCCESS if it is OK to go ahead and let the caller
> > > > open the file
> > > > return RetStatus;
> > > > }
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@bitarmor.com
> > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@livevault.com To
> > unsubscribe
> > > > send a blank email to xxxxx@lists.osr.com
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@arkivio.com
> > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@arkivio.com
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@arkivio.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

This file of mine (a very special file) is never compressed, I even fail
FSCTL_SET_COMPRESSION/ENCRYPTION on it.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Paul Mu”
To: “Windows File Systems Devs Interest List”
Sent: Monday, October 13, 2003 8:15 AM
Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
eateStreamFile

> I see now. But is no caching fileobject(file opened with no intermediate
> buffering) really honored for the no-caching all the time, for instance,
> for a compressed file.
>
> Paul
> ----- Original Message -----
> From: “Maxim S. Shatskih”
> To: “Windows File Systems Devs Interest List”
> Sent: Saturday, October 11, 2003 8:54 PM
> Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
> eateStreamFile
>
>
> > Yes, but I open my first file object with no caching.
> >
> > Maxim Shatskih, Windows DDK MVP
> > StorageCraft Corporation
> > xxxxx@storagecraft.com
> > http://www.storagecraft.com
> >
> >
> > ----- Original Message -----
> > From: “Paul Mu”
> > To: “Windows File Systems Devs Interest List”
> > Sent: Saturday, October 11, 2003 11:05 AM
> > Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
> > eateStreamFile
> >
> >
> > > Please correct me if I am wrong, according to the spirit of OSR
> online’s
> > > paper about IoCancelOpen, the downside of doing this(reusing the same
> > > fileobject) is that if you did any work(read,write, or compression)
> which
> > > can potentially enable the cache as the first fileobject, then cache
> manager
> > > will hold a reference to it and the manual CLOSE will not help here .
> If
> > > the downstream driver fails second open with the same fileobject after
> you
> > > restore the filename and forward the original CREATE, then the
> fileobject
> > > can be deleted by the initial caller. Now we would have the cache
> manager
> > > hold a reference to a deleted fileobject.
> > >
> > > Paul
> > >
> > > ----- Original Message -----
> > > From: “Maxim S. Shatskih”
> > > To: “Windows File Systems Devs Interest List”
> > > Sent: Friday, October 10, 2003 7:51 PM
> > > Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
> > > eateStreamFile
> > >
> > >
> > > > There is also a following technique to open some file in filter’s
> > > CREATE
> > > > path:
> > > >
> > > > - save the original FileObject->FileName, RelatedFileObject and
> the
> > > whole
> > > > CREATE stack location
> > > > - set up the new file name and the new CREATE stack location
> > > > - pass the IRP down
> > > > - work with the opened file
> > > > - then handicraft CLEANUP and CLOSE (in the same IRP and the same
> > > stack
> > > > location) and send them down
> > > > - then reset FO_CLEANUP_COMPLETE on the file object
> > > > - then restore the saved CREATE stack location, FileName and
> > > > RelatedFileObject, and pass the original CREATE down at last.
> > > >
> > > > Maxim Shatskih, Windows DDK MVP
> > > > StorageCraft Corporation
> > > > xxxxx@storagecraft.com
> > > > http://www.storagecraft.com
> > > >
> > > >
> > > > ----- Original Message -----
> > > > From: “Paul Mu”
> > > > To: “Windows File Systems Devs Interest List”
> > > > Sent: Saturday, October 11, 2003 3:35 AM
> > > > Subject: [ntfsd] RE: Postscript on closing FileObjects created by IoCr
> > > > eateStreamFile
> > > >
> > > >
> > > > > One thing I know for sure, FO_CLEANUP_COMPLETE does not intefere
> with
> > > CREATE.
> > > > >
> > > > > Paul
> > > > >
> > > > > -----Original Message-----
> > > > > From: Ted Hess [mailto:xxxxx@livevault.com]
> > > > > Sent: Thursday, October 09, 2003 7:08 AM
> > > > > To: Windows File Systems Devs Interest List
> > > > > Subject: [ntfsd] RE: Postscript on closing FileObjects created by
> IoCr
> > > > > eateStreamFile
> > > > >
> > > > >
> > > > > Matt -
> > > > >
> > > > > The comment was really a reminder that these flags must be cleared.
> The
> > > code
> > > > > copies the flags from the original FileObject which does indeed have
> > > these
> > > > > flags absent. Perhaps the totally correct thing to do would be copy
> the
> > > > > flags AND clear the unwanted ones.
> > > > >
> > > > > Ex: cloneFileObject->Flags = fileObject->Flags & ~(FO_STREAM_FILE |
> > > > > FO_HANDLE_CREATED | FO_CLEANUP_COMPLETE);
> > > > >
> > > > > Also - I’m not sure, but I think the CREATE will fail if any of
> these
> > > are
> > > > > set.
> > > > >
> > > > > Thanks, /ted
> > > > >
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Matthew N. White [mailto:xxxxx@bitarmor.com]
> > > > > Sent: Wednesday, October 08, 2003 7:12 PM
> > > > > To: Windows File Systems Devs Interest List
> > > > > Subject: [ntfsd] RE: Postscript on closing FileObjects created by
> > > > > IoCreateStreamFile
> > > > >
> > > > >
> > > > > Ted,
> > > > > Is there any reason that the line that clears the FO_STREAM_FILE,
> > > > > FO_HANDLE_CREATED, FO_CLEANUP_COMPLETE flags in the code is
> commented
> > > out?
> > > > > Below you say that you “MUST clear the FO_STREAM_FILE,
> FO_HANDLE_CREATED
> >
> > > &
> > > > > FO_CLEANUP_COMPLETE flags to successfully open this FO via a CREATE
> > > IRP.”
> > > > > Just wondering about that, since the code just copies the flags
> over.
> > > > >
> > > > > Matt
> > > > >
> > > > > -----Original Message-----
> > > > > From: xxxxx@lists.osr.com
> > > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Ted Hess
> > > > > Sent: Tuesday, October 07, 2003 2:39 PM
> > > > > To: Windows File Systems Devs Interest List
> > > > > Subject: [ntfsd] Postscript on closing FileObjects created by
> > > > > IoCreateStreamFile
> > > > >
> > > > >
> > > > > I’ve been meaning to write up these observations for a couple of
> months
> > > now
> > > > > but just haven’t had the time. Many thanks to Tony and Nick for
> their
> > > info
> > > > > and help on this subject back in June when I was struggling with it.
> > > > >
> > > > > I basically followed the scenario outlined on the OSR on-line site
> and
> > > the
> > > > > latest NT Insider article with a couple of differences which I
> believe
> > > to be
> > > > > significant. In particular, NTFS seems to be very sensitive to the
> Flags
> > > > > field in the FileObject.
> > > > >
> > > > > My first attempt essentially followed the steps outlined on the OSR
> web
> > > > > site. However, I noticed I was leaking FileObjects!! Why was this?
> Well,
> > > it
> > > > > turns out the the CLEANUP IRP wasn’t really doing anything like it
> > > should
> > > > > have been and the ObDereferenceObject never issued the CLOSE. I was
> left
> > > > > with a FO and a cache reference. Also, the file sharing state was
> > > whacked
> > > > > 'cause the CLEANUP didn’t reset it.
> > > > >
> > > > > After a bit of hair pulling and experimentation, I discovered that
> this
> > > > > problem only happened on NTFS and not FAT. FAT does not care about
> the
> > > same
> > > > > FileObject states as does NTFS (at least from my observations of
> NTFS
> > > > > behavior and code inspection in the FAT IFS kit sources). Here’s
> what I
> > > > > think needs to happen:
> > > > >
> > > > > After creating the stream FO, you MUST clear the FO_STREAM_FILE,
> > > > > FO_HANDLE_CREATED & FO_CLEANUP_COMPLETE flags to successfully open
> this
> > > FO
> > > > > via a CREATE IRP.
> > > > >
> > > > > If the CREATE fails, you need to set FO_STREAM_FILE before calling
> > > > > ObDereferenctObject.
> > > > >
> > > > > If the CREATE succeeds, you will need to set FO_HANDLE_CREATED
> before
> > > > > issuing the CLEANUP and set FO_STREAM_FILE before calling
> > > > > ObDereferenceObject. Otherwise, things just don’t work as expected.
> > > > >
> > > > > Comments, mockery and/or derision welcome…
> > > > >
> > > > > /ted
> > > > >
> > > > > Attached is an abbreviated, and hopefully useful code snippet for
> > > opening a
> > > > > file and then successfully closing it in your pre-Create filter
> path.
> > > > >
> > > > >
> > > > > NTSTATUS
> > > > > xxxPreCreateCheck(PDEVICE_OBJECT targetDeviceObject, PIRP Irp) {
> > > > > NTSTATUS Status, RetStatus =
> > > > > STATUS_SUCCESS;
> > > > > UNICODE_STRING fileName;
> > > > > PUNICODE_STRING pFOName;
> > > > > PFILE_OBJECT fileObject, cloneFileObject;
> > > > >
> > > > > PIO_STACK_LOCATION irpSp = IoGetCurrentIrpStackLocation(Irp);
> > > > > PIO_STACK_LOCATION nxtSp = IoGetNextIrpStackLocation(Irp);
> > > > >
> > > > > ACCESS_MASK desiredAccess =
> > > > > irpSp->Parameters.Create.SecurityContext->DesiredAccess;
> > > > > ACCESS_MASK desiredAccessCreate = desiredAccess;
> > > > >
> > > > > ULONG createOptions = irpSp->Parameters.Create.Options &
> > > > > FILE_VALID_OPTION_FLAGS;
> > > > > ULONG createDisposition = (irpSp->Parameters.Create.Options >>
> > > > > 24) & 0xFF;
> > > > >
> > > > > // The subject of this excercise (save it)
> > > > > fileObject = irpSp->FileObject;
> > > > >
> > > > > // Make a copy of the callers buffer here 'cause we don’t know
> > > > > who’ll free it
> > > > > fileName.Length = fileObject->FileName.Length;
> > > > > fileName.MaximumLength = fileObject->FileName.MaximumLength;
> > > > > fileName.Buffer = ExAllocatePool(PagedPool, fileName.MaximumLength);
> > > > >
> > > > > // Bag it if no memory available
> > > > > if (!fileName.Buffer) {
> > > > > return STATUS_INSUFFICIENT_RESOURCES;
> > > > > }
> > > > > // Copy original contents into temp buffer
> > > > > RtlCopyMemory(fileName.Buffer, fileObject->FileName.Buffer,
> > > > > fileName.MaximumLength);
> > > > >
> > > > > // Allocate a new, temporary FO
> > > > > #if defined(NT_40)
> > > > > // Watch out for the gratuitous CLEANUP IRP!
> > > > > cloneFileObject = IoCreateStreamFileObject(fileObject, NULL); #else
> > > > > cloneFileObject = IoCreateStreamFileObjectLite(fileObject,
> > > > > NULL); #endif
> > > > >
> > > > > // (Re-)init newly created FO
> > > > > // cloneFileObject->Flags &= ~(FO_STREAM_FILE | FO_HANDLE_CREATED |
> > > > > FO_CLEANUP_COMPLETE);
> > > > > cloneFileObject->Flags = fileObject->Flags;
> > > > > cloneFileObject->RelatedFileObject =
> > > > > fileObject->RelatedFileObject;
> > > > >
> > > > > // Give it a name
> > > > > pFOName = &cloneFileObject->FileName;
> > > > > *pFOName = fileName;
> > > > >
> > > > > // Initialize embedded event structures
> > > > > KeInitializeEvent(&cloneFileObject->Lock, SynchronizationEvent,
> > > > > FALSE);
> > > > > KeInitializeEvent(&cloneFileObject->Event, NotificationEvent,
> > > > > FALSE);
> > > > >
> > > > > // Make a copy of the create irp (preserve flags)
> > > > > RtlCopyMemory(nxtSp, irpSp, sizeof(IO_STACK_LOCATION));
> > > > > nxtSp->Control = 0;
> > > > > // Use temporary FO and underlying DO
> > > > > nxtSp->FileObject = cloneFileObject;
> > > > > nxtSp->DeviceObject = DeviceObject;
> > > > >
> > > > > // Setup completion routine
> > > > > IoSetCompletionRoutine (Irp, xxxSimpleEventCompletion,
> > > > > &cloneFileObject->Event, TRUE, TRUE, TRUE);
> > > > >
> > > > > //
> > > > > // Ensure that the create disposition requested
> > > > > // indicates all of the appropriate “rights” necessary
> > > > > // for the given operation.
> > > > > //
> > > > > if (createDisposition == FILE_SUPERSEDE) {
> > > > > desiredAccessCreate |= DELETE;
> > > > > }
> > > > >
> > > > > if ((createDisposition == FILE_OVERWRITE) ||
> > > > > (createDisposition == FILE_OVERWRITE_IF)) {
> > > > > desiredAccessCreate |= (FILE_WRITE_DATA | FILE_WRITE_EA
> > > > > | FILE_WRITE_ATTRIBUTES);
> > > > > }
> > > > >
> > > > > // Setup our desired access
> > > > > nxtSp->Parameters.Create.SecurityContext->DesiredAccess =
> > > > > desiredAccessCreate & ~SYNCHRONIZE;
> > > > > nxtSp->Parameters.Create.ShareAccess = FILE_SHARE_VALID_FLAGS;
> > > > > // Specify open only (no create/supersede)
> > > > > nxtSp->Parameters.Create.Options = (FILE_OPEN << 24) |
> > > > > (createOptions &
> > > > > (FILE_OPEN_NO_RECALL | FILE_OPEN_FOR_BACKUP_INTENT |
> > > > >
> > > > > FILE_COMPLETE_IF_OPLOCKED | FILE_NON_DIRECTORY_FILE |
> > > > >
> > > > > FILE_DIRECTORY_FILE | FILE_OPEN_REPARSE_POINT)) |
> > > > >
> > > > > FILE_NO_INTERMEDIATE_BUFFERING;
> > > > >
> > > > > //
> > > > > // Chuck the request to the underlying driver
> > > > > //
> > > > > Status = IoCallDriver(DeviceObject, Irp);
> > > > >
> > > > > if (Status == STATUS_PENDING) {
> > > > > // Wait for event if pended
> > > > > KeWaitForSingleObject(&cloneFileObject->Event,
> > > > > Executive, KernelMode, FALSE, 0);
> > > > > Status = Irp->IoStatus.Status;
> > > > > }
> > > > >
> > > > > // Now check for file open failures
> > > > > if(!NT_SUCCESS(Status) || (Irp->IoStatus.Status ==
> > > > > STATUS_REPARSE)) {
> > > > > // Restore original IRP parameters
> > > > > irpSp->Parameters.Create.SecurityContext->DesiredAccess
> > > > > = desiredAccess;
> > > > >
> > > > > // Free temp (or FSD) buffer if still allocated
> > > > > if (pFOName->Buffer) {
> > > > > ExFreePool(pFOName->Buffer);
> > > > > pFOName->Buffer = NULL;
> > > > > pFOName->Length = 0;
> > > > > pFOName->MaximumLength = 0;
> > > > > }
> > > > >
> > > > > // Reset FO stuff
> > > > > cloneFileObject->Flags |= FO_STREAM_FILE;
> > > > > cloneFileObject->RelatedFileObject = NULL;
> > > > >
> > > > > // Toss temp FO
> > > > > ObDereferenceObject(cloneFileObject);
> > > > >
> > > > > // Nothing has happened yet
> > > > > Irp->PendingReturned = FALSE;
> > > > >
> > > > > // OK to proceed if cannot open file
> > > > > return STATUS_SUCCESS;
> > > > > }
> > > > >
> > > > > //
> > > > > // OK - we now have an opened FileObject with a valid FsContext
> > > > > pointer
> > > > > //
> > > > >
> > > > > // RetStatus = <<<
Insert your desired actions here >>>
> > > > >
> > > > > //
> > > > > //
Done with FileObject
> > > > > //
> > > > >
> > > > > // Free temp (or FSD) buffer if still allocated
> > > > > if (pFOName->Buffer) {
> > > > > ExFreePool(pFOName->Buffer);
> > > > > pFOName->Buffer = NULL;
> > > > > pFOName->Length = 0;
> > > > > pFOName->MaximumLength = 0;
> > > > > }
> > > > >
> > > > > // Need to light “handle created” flag
> > > > > cloneFileObject->Flags |= FO_HANDLE_CREATED;
> > > > >
> > > > > // Send a CLEANUP down to the FSD
> > > > > nxtSp->MajorFunction = IRP_MJ_CLEANUP;
> > > > > nxtSp->MinorFunction = 0;
> > > > > nxtSp->Flags = 0;
> > > > > nxtSp->Control = 0;
> > > > > nxtSp->DeviceObject = DeviceObject;
> > > > > nxtSp->FileObject = cloneFileObject;
> > > > >
> > > > > // Init our completion event
> > > > > KeInitializeEvent(&cloneFileObject->Event, NotificationEvent,
> > > > > FALSE);
> > > > >
> > > > > // Setup completion routine
> > > > > IoSetCompletionRoutine (Irp, xxxSimpleEventCompletion,
> > > > > &cloneFileObject->Event, TRUE, TRUE, TRUE);
> > > > >
> > > > > //
> > > > > // Call the FSD to release any share access, cache refs, etc.
> > > > > //
> > > > > Status = IoCallDriver(DeviceObject, Irp);
> > > > >
> > > > > //
> > > > > // Wait for the cleanup to finish
> > > > > //
> > > > > if (Status == STATUS_PENDING) {
> > > > > KeWaitForSingleObject(&cloneFileObject->Event,
> > > > > Executive, KernelMode, FALSE, 0);
> > > > > }
> > > > >
> > > > > // Reset FO stuff
> > > > > cloneFileObject->Flags |= FO_STREAM_FILE;
> > > > > cloneFileObject->RelatedFileObject = NULL;
> > > > >
> > > > > // We allocated the FO, now we must free it
> > > > > ObDereferenceObject(cloneFileObject);
> > > > >
> > > > > // Restore original IRP parameters
> > > > > irpSp->Parameters.Create.SecurityContext->DesiredAccess =
> > > > > desiredAccess;
> > > > >
> > > > > // Nothing has happened yet
> > > > > Irp->PendingReturned = FALSE;
> > > > > Irp->IoStatus.Status = STATUS_SUCCESS;
> > > > > Irp->IoStatus.Information = 0;
> > > > >
> > > > > // return STATUS_SUCCESS if it is OK to go ahead and let the caller
> > > > > open the file
> > > > > return RetStatus;
> > > > > }
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as: xxxxx@bitarmor.com
> > > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > > >
> > > > >
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as: xxxxx@livevault.com To
> > > unsubscribe
> > > > > send a blank email to xxxxx@lists.osr.com
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as: xxxxx@arkivio.com
> > > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> > > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@arkivio.com
> > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > >
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@arkivio.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com