IRP_MJ_CLEANUP/CLOSE redux

Ok, so here’s the deal. I have a file filter system driver which I am
excercising with an application that does this:

  1. open fileA
    driver sees IRP_MJ_CREATE
  2. read fileA
    driver sees IRP_MJ_READ
  3. write back to fileA
  4. close fileA
    driver sees IRP_MJ_CLEANUP
  5. immediately reopen fileA
    driver sees IRP_MJ_CREATE
    driver sees IRP_MJ_WRITE from 1st open

It appears that the driver receives the IRP_MJ_CREATE from the second Open
*before* it receives the IRP_MJ_WRITE from the first write! In addition, it
doesn’t seem to see an IRP_MJ_CLOSE at all.

Any clues as to what is going on here? How do I get the I/O from the first
open of fileA to complete before the driver goes onto the second?

Thanks.

Neil

Neil,

This looks like normal NT/Windows 2000 behavior to me. Why do you expect
the IRP_MJ_WRITE to occur before the second create? I mean, the cache
manager write’s data behind quickly, but we’re talking order seconds, not
order milliseconds (which is more akin to the time delay between the file
close - the IRP_MJ_CLEANUP and the next open.)

And, of course, since there is an open cache section referencing the file
object, we can’t close it (the reference count is non-zero.)

If you want the I/O to complete, you could either change the mode to
write-through (so data won’t sit around in the cache) or you could always
flush it out when you see the IRP_MJ_CLEANUP (a good ol’
IRP_MJ_FLUSH_BUFFERS should do the trick.)

I hope this helps.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Neil Weicher [mailto:xxxxx@netlib.com]
Sent: Monday, March 27, 2000 7:18 PM
To: File Systems Developers
Subject: [ntfsd] IRP_MJ_CLEANUP/CLOSE redux

Ok, so here’s the deal. I have a file filter system driver which I am
excercising with an application that does this:

  1. open fileA
    driver sees IRP_MJ_CREATE
  2. read fileA
    driver sees IRP_MJ_READ
  3. write back to fileA
  4. close fileA
    driver sees IRP_MJ_CLEANUP
  5. immediately reopen fileA
    driver sees IRP_MJ_CREATE
    driver sees IRP_MJ_WRITE from 1st open

It appears that the driver receives the IRP_MJ_CREATE from the second Open
*before* it receives the IRP_MJ_WRITE from the first write! In addition, it
doesn’t seem to see an IRP_MJ_CLOSE at all.

Any clues as to what is going on here? How do I get the I/O from the first
open of fileA to complete before the driver goes onto the second?

Thanks.

Neil


You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

Tony, why doesn’t his filter see the write into cache? Maybe it’s
occuring via fast I/O?


Dave Cox
Hewlett-Packard Co.
HPSO/SSMO (Santa Barbara)
https://ecardfile.com/id/Dave+Cox

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Monday, March 27, 2000 9:03 PM
To: File Systems Developers
Subject: [ntfsd] RE: IRP_MJ_CLEANUP/CLOSE redux

Neil,

This looks like normal NT/Windows 2000 behavior to me. Why do you expect
the IRP_MJ_WRITE to occur before the second create? I mean, the cache
manager write’s data behind quickly, but we’re talking order seconds, not
order milliseconds (which is more akin to the time delay between the file
close - the IRP_MJ_CLEANUP and the next open.)

And, of course, since there is an open cache section referencing the file
object, we can’t close it (the reference count is non-zero.)

If you want the I/O to complete, you could either change the mode to
write-through (so data won’t sit around in the cache) or you could always
flush it out when you see the IRP_MJ_CLEANUP (a good ol’
IRP_MJ_FLUSH_BUFFERS should do the trick.)

I hope this helps.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Neil Weicher [mailto:xxxxx@netlib.com]
Sent: Monday, March 27, 2000 7:18 PM
To: File Systems Developers
Subject: [ntfsd] IRP_MJ_CLEANUP/CLOSE redux

Ok, so here’s the deal. I have a file filter system driver which I am
excercising with an application that does this:

  1. open fileA
    driver sees IRP_MJ_CREATE
  2. read fileA
    driver sees IRP_MJ_READ
  3. write back to fileA
  4. close fileA
    driver sees IRP_MJ_CLEANUP
  5. immediately reopen fileA
    driver sees IRP_MJ_CREATE
    driver sees IRP_MJ_WRITE from 1st open

It appears that the driver receives the IRP_MJ_CREATE from the second Open
*before* it receives the IRP_MJ_WRITE from the first write! In addition, it
doesn’t seem to see an IRP_MJ_CLOSE at all.

Any clues as to what is going on here? How do I get the I/O from the first
open of fileA to complete before the driver goes onto the second?

Thanks.

Neil


You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)


You are currently subscribed to ntfsd as: david_cox2@hp.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

Dave,

I’d assumed that either (a) it was coming in via fast I/O (and hence there
was no IRP_MJ_WRITE); or (b) it was written via a mapped file. (b) is less
likely unless he wrote a program to do it, but in either case this all
sounds like typical NT behavior to me.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: COX,DAVID (HP-Roseville,ex1) [mailto:david_cox2@hp.com]
Sent: Tuesday, March 28, 2000 7:32 PM
To: File Systems Developers
Subject: [ntfsd] RE: IRP_MJ_CLEANUP/CLOSE redux

Tony, why doesn’t his filter see the write into cache? Maybe it’s
occuring via fast I/O?


Dave Cox
Hewlett-Packard Co.
HPSO/SSMO (Santa Barbara)
https://ecardfile.com/id/Dave+Cox

-----Original Message-----
From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Monday, March 27, 2000 9:03 PM
To: File Systems Developers
Subject: [ntfsd] RE: IRP_MJ_CLEANUP/CLOSE redux

Neil,

This looks like normal NT/Windows 2000 behavior to me. Why do you expect
the IRP_MJ_WRITE to occur before the second create? I mean, the cache
manager write’s data behind quickly, but we’re talking order seconds, not
order milliseconds (which is more akin to the time delay between the file
close - the IRP_MJ_CLEANUP and the next open.)

And, of course, since there is an open cache section referencing the file
object, we can’t close it (the reference count is non-zero.)

If you want the I/O to complete, you could either change the mode to
write-through (so data won’t sit around in the cache) or you could always
flush it out when you see the IRP_MJ_CLEANUP (a good ol’
IRP_MJ_FLUSH_BUFFERS should do the trick.)

I hope this helps.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Neil Weicher [mailto:xxxxx@netlib.com]
Sent: Monday, March 27, 2000 7:18 PM
To: File Systems Developers
Subject: [ntfsd] IRP_MJ_CLEANUP/CLOSE redux

Ok, so here’s the deal. I have a file filter system driver which I am
excercising with an application that does this:

  1. open fileA
    driver sees IRP_MJ_CREATE
  2. read fileA
    driver sees IRP_MJ_READ
  3. write back to fileA
  4. close fileA
    driver sees IRP_MJ_CLEANUP
  5. immediately reopen fileA
    driver sees IRP_MJ_CREATE
    driver sees IRP_MJ_WRITE from 1st open

It appears that the driver receives the IRP_MJ_CREATE from the second Open
*before* it receives the IRP_MJ_WRITE from the first write! In addition, it
doesn’t seem to see an IRP_MJ_CLOSE at all.

Any clues as to what is going on here? How do I get the I/O from the first
open of fileA to complete before the driver goes onto the second?

Thanks.

Neil


You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)


You are currently subscribed to ntfsd as: david_cox2@hp.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)


You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

Hello Tony,

In case of write, not getting IRP_MJ_CLOSE at all, Is it a normal NT
behavior ?

Regards,
Aman.


From: Tony Mason[SMTP:xxxxx@osr.com]
Reply To: File Systems Developers
Sent: Tuesday, March 28, 2000 10:32 AM
To: File Systems Developers
Subject: [ntfsd] RE: IRP_MJ_CLEANUP/CLOSE redux

Neil,

This looks like normal NT/Windows 2000 behavior to me. Why do you expect
the IRP_MJ_WRITE to occur before the second create? I mean, the cache
manager write’s data behind quickly, but we’re talking order seconds, not
order milliseconds (which is more akin to the time delay between the file
close - the IRP_MJ_CLEANUP and the next open.)

And, of course, since there is an open cache section referencing the file
object, we can’t close it (the reference count is non-zero.)

If you want the I/O to complete, you could either change the mode to
write-through (so data won’t sit around in the cache) or you could always
flush it out when you see the IRP_MJ_CLEANUP (a good ol’
IRP_MJ_FLUSH_BUFFERS should do the trick.)

I hope this helps.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Neil Weicher [mailto:xxxxx@netlib.com]
Sent: Monday, March 27, 2000 7:18 PM
To: File Systems Developers
Subject: [ntfsd] IRP_MJ_CLEANUP/CLOSE redux

Ok, so here’s the deal. I have a file filter system driver which I am
excercising with an application that does this:

  1. open fileA
    driver sees IRP_MJ_CREATE
  2. read fileA
    driver sees IRP_MJ_READ
  3. write back to fileA
  4. close fileA
    driver sees IRP_MJ_CLEANUP
  5. immediately reopen fileA
    driver sees IRP_MJ_CREATE
    driver sees IRP_MJ_WRITE from 1st open

It appears that the driver receives the IRP_MJ_CREATE from the second Open
*before* it receives the IRP_MJ_WRITE from the first write! In addition,
it
doesn’t seem to see an IRP_MJ_CLOSE at all.

Any clues as to what is going on here? How do I get the I/O from the
first
open of fileA to complete before the driver goes onto the second?

Thanks.

Neil


You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)


You are currently subscribed to ntfsd as: xxxxx@satyam.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)

It isn’t that one doesn’t get IRP_MJ_CLOSE at all, it is just that you have
to wait until all references to the file object “go away”.

The problem is that the VM system maintains a section object that is used to
memory map the file (and it is memory mapped into the cache, for instance.)
So long as that section object does not “go away” the file object behind it
cannot “go away” either. So, in turn, the section object stays around until
all of the mappings of it go away and any memory containing contents from
that section go away.

I’ve seen this process take (literally) days, because it is all related to
page reclaimation processing within the VM system - so a quiescent system
(such as your development workstation) over a long weekend might not close
those sections until the following week when you start working again. Thus,
the IRP_MJ_CLOSE may - literally - take days to arrive.

But eventually, it does arrive. Just because it is independent of the
handle close, people assume it never happens. That’s just not the case.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com http:

-----Original Message-----
From: Aman_Gupta [mailto:xxxxx@satyam.com]
Sent: Wednesday, March 29, 2000 12:34 AM
To: File Systems Developers; xxxxx@osr.com
Subject: RE: [ntfsd] RE: IRP_MJ_CLEANUP/CLOSE redux

Hello Tony,

In case of write, not getting IRP_MJ_CLOSE at all, Is it a normal NT
behavior ?

Regards,
Aman.

----------
From: Tony Mason[SMTP:xxxxx@osr.com]
Reply To: File Systems Developers
Sent: Tuesday, March 28, 2000 10:32 AM
To: File Systems Developers
Subject: [ntfsd] RE: IRP_MJ_CLEANUP/CLOSE redux

Neil,

This looks like normal NT/Windows 2000 behavior to me. Why do you
expect
the IRP_MJ_WRITE to occur before the second create? I mean, the cache
manager write’s data behind quickly, but we’re talking order seconds, not
order milliseconds (which is more akin to the time delay between the file
close - the IRP_MJ_CLEANUP and the next open.)

And, of course, since there is an open cache section referencing the
file
object, we can’t close it (the reference count is non-zero.)

If you want the I/O to complete, you could either change the mode to

write-through (so data won’t sit around in the cache) or you could always
flush it out when you see the IRP_MJ_CLEANUP (a good ol’
IRP_MJ_FLUSH_BUFFERS should do the trick.)

I hope this helps.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com http:

-----Original Message-----
From: Neil Weicher [mailto:xxxxx@netlib.com mailto:xxxxx]
Sent: Monday, March 27, 2000 7:18 PM
To: File Systems Developers
Subject: [ntfsd] IRP_MJ_CLEANUP/CLOSE redux

Ok, so here’s the deal. I have a file filter system driver which I
am
excercising with an application that does this:

1) open fileA
driver sees IRP_MJ_CREATE
2) read fileA
driver sees IRP_MJ_READ
3) write back to fileA
4) close fileA
driver sees IRP_MJ_CLEANUP
5) immediately reopen fileA
driver sees IRP_MJ_CREATE
driver sees IRP_MJ_WRITE from 1st open

It appears that the driver receives the IRP_MJ_CREATE from the
second Open
before it receives the IRP_MJ_WRITE from the first write! In addition, it

doesn’t seem to see an IRP_MJ_CLOSE at all.

Any clues as to what is going on here? How do I get the I/O from
the first
open of fileA to complete before the driver goes onto the second?

Thanks.

Neil


You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)


You are currently subscribed to ntfsd as: xxxxx@satyam.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)</mailto:xxxxx></http:></http:>

This can be confirmed by shutting down the system and looking at a lot of
CLOSESs commming. Many of them are closes for files that were closed by
applications long time ago.

Inaki.

At 01:48 29/03/00 -0500, you wrote:

It isn’t that one doesn’t get IRP_MJ_CLOSE at all, it is just that you have
to wait until all references to the file object “go away”.

The problem is that the VM system maintains a section object that is used to
memory map the file (and it is memory mapped into the cache, for instance.)
So long as that section object does not “go away” the file object behind it
cannot “go away” either. So, in turn, the section object stays around until
all of the mappings of it go away and any memory containing contents from
that section go away.

I’ve seen this process take (literally) days, because it is all related to
page reclaimation processing within the VM system - so a quiescent system
(such as your development workstation) over a long weekend might not close
those sections until the following week when you start working again. Thus,
the IRP_MJ_CLOSE may - literally - take days to arrive.

But eventually, it does arrive. Just because it is independent of the
handle close, people assume it never happens. That’s just not the case.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com http:
>
>
>-----Original Message-----
>From: Aman_Gupta [mailto:xxxxx@satyam.com]
>Sent: Wednesday, March 29, 2000 12:34 AM
>To: File Systems Developers; xxxxx@osr.com
>Subject: RE: [ntfsd] RE: IRP_MJ_CLEANUP/CLOSE redux
>
>
>
>Hello Tony,
>
>In case of write, not getting IRP_MJ_CLOSE at all, Is it a normal NT
>behavior ?
>
>Regards,
>Aman.
>
> ----------
>From: Tony Mason[SMTP:xxxxx@osr.com]
>Reply To: File Systems Developers
>Sent: Tuesday, March 28, 2000 10:32 AM
>To: File Systems Developers
>Subject: [ntfsd] RE: IRP_MJ_CLEANUP/CLOSE redux
>
> Neil,
>
> This looks like normal NT/Windows 2000 behavior to me. Why do you
>expect
>the IRP_MJ_WRITE to occur before the second create? I mean, the cache
>manager write’s data behind quickly, but we’re talking order seconds, not
>order milliseconds (which is more akin to the time delay between the file
>close - the IRP_MJ_CLEANUP and the next open.)
>
> And, of course, since there is an open cache section referencing the
>file
>object, we can’t close it (the reference count is non-zero.)
>
> If you want the I/O to complete, you could either change the mode to
>
>write-through (so data won’t sit around in the cache) or you could always
>flush it out when you see the IRP_MJ_CLEANUP (a good ol’
>IRP_MJ_FLUSH_BUFFERS should do the trick.)
>
> I hope this helps.
>
> Regards,
>
> Tony
>
> Tony Mason
>Consulting Partner
>OSR Open Systems Resources, Inc.
>http://www.osr.com http:
>
> -----Original Message-----
>From: Neil Weicher [mailto:xxxxx@netlib.com mailto:xxxxx]
>Sent: Monday, March 27, 2000 7:18 PM
>To: File Systems Developers
>Subject: [ntfsd] IRP_MJ_CLEANUP/CLOSE redux
>
>
> Ok, so here’s the deal. I have a file filter system driver which I
>am
>excercising with an application that does this:
>
> 1) open fileA
> driver sees IRP_MJ_CREATE
>2) read fileA
> driver sees IRP_MJ_READ
>3) write back to fileA
>4) close fileA
> driver sees IRP_MJ_CLEANUP
>5) immediately reopen fileA
> driver sees IRP_MJ_CREATE
> driver sees IRP_MJ_WRITE from 1st open
>
> It appears that the driver receives the IRP_MJ_CREATE from the
>second Open
>before it receives the IRP_MJ_WRITE from the first write! In addition, it
>
>doesn’t seem to see an IRP_MJ_CLOSE at all.
>
> Any clues as to what is going on here? How do I get the I/O from
>the first
>open of fileA to complete before the driver goes onto the second?
>
> Thanks.
>
> Neil
>
>
>
> —
>You are currently subscribed to ntfsd as: xxxxx@osr.com
>To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>
> —
>You are currently subscribed to ntfsd as: xxxxx@satyam.com
>To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>
>
>—
>You are currently subscribed to ntfsd as: xxxxx@lander.es
>To unsubscribe send a blank email to $subst(‘Email.Unsub’)
>
>
></mailto:xxxxx></http:></http:>

Hi Tony,

Could you please tell me when the IRP_MJ_CLEANUP for IRP_MJ_WRITE write
will come ? As you mentioned that close may take days and one reference for
that file will remain open for that period then how the other operations
like RENAME will take place as CREATE for rename fails with
STATUS_ACCESS_VIOLATION.

Regards,
Aman.


From: Tony Mason[SMTP:xxxxx@osr.com]
Sent: Wednesday, March 29, 2000 12:18 PM
To: ‘Aman_Gupta’; ntfsd redirect; Tony Mason
Subject: RE: [ntfsd] RE: IRP_MJ_CLEANUP/CLOSE redux

It isn’t that one doesn’t get IRP_MJ_CLOSE at all, it is just that you
have to wait until all references to the file object “go away”.

The problem is that the VM system maintains a section object that is used
to memory map the file (and it is memory mapped into the cache, for
instance.) So long as that section object does not “go away” the file
object behind it cannot “go away” either. So, in turn, the section object
stays around until all of the mappings of it go away and any memory
containing contents from that section go away.

I’ve seen this process take (literally) days, because it is all related to
page reclaimation processing within the VM system - so a quiescent system
(such as your development workstation) over a long weekend might not close
those sections until the following week when you start working again.
Thus, the IRP_MJ_CLOSE may - literally - take days to arrive.

But eventually, it does arrive. Just because it is independent of the
handle close, people assume it never happens. That’s just not the case.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Aman_Gupta [mailto:xxxxx@satyam.com]
Sent: Wednesday, March 29, 2000 12:34 AM
To: File Systems Developers; xxxxx@osr.com
Subject: RE: [ntfsd] RE: IRP_MJ_CLEANUP/CLOSE redux

Hello Tony,

In case of write, not getting IRP_MJ_CLOSE at all, Is it a normal NT
behavior ?

Regards,
Aman.


From: Tony Mason[SMTP:xxxxx@osr.com]
Reply To: File Systems Developers
Sent: Tuesday, March 28, 2000 10:32 AM
To: File Systems Developers
Subject: [ntfsd] RE: IRP_MJ_CLEANUP/CLOSE redux

Neil,

This looks like normal NT/Windows 2000 behavior to me. Why
do you expect
the IRP_MJ_WRITE to occur before the second create? I mean, the
cache
manager write’s data behind quickly, but we’re talking order
seconds, not
order milliseconds (which is more akin to the time delay between the
file
close - the IRP_MJ_CLEANUP and the next open.)

And, of course, since there is an open cache section
referencing the file
object, we can’t close it (the reference count is non-zero.)

If you want the I/O to complete, you could either change the
mode to
write-through (so data won’t sit around in the cache) or you could
always
flush it out when you see the IRP_MJ_CLEANUP (a good ol’
IRP_MJ_FLUSH_BUFFERS should do the trick.)

I hope this helps.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Neil Weicher [mailto:xxxxx@netlib.com]
Sent: Monday, March 27, 2000 7:18 PM
To: File Systems Developers
Subject: [ntfsd] IRP_MJ_CLEANUP/CLOSE redux

Ok, so here’s the deal. I have a file filter system driver
which I am
excercising with an application that does this:

  1. open fileA
    driver sees IRP_MJ_CREATE
  2. read fileA
    driver sees IRP_MJ_READ
  3. write back to fileA
  4. close fileA
    driver sees IRP_MJ_CLEANUP
  5. immediately reopen fileA
    driver sees IRP_MJ_CREATE
    driver sees IRP_MJ_WRITE from 1st open

It appears that the driver receives the IRP_MJ_CREATE from
the second Open
*before* it receives the IRP_MJ_WRITE from the first write! In
addition, it
doesn’t seem to see an IRP_MJ_CLOSE at all.

Any clues as to what is going on here? How do I get the I/O
from the first
open of fileA to complete before the driver goes onto the second?

Thanks.

Neil


You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)


You are currently subscribed to ntfsd as: xxxxx@satyam.com
To unsubscribe send a blank email to $subst(‘Email.Unsub’)