Problem with NAI filter V7.0

Has anyone else out there had inter-op problems with the latest McAfee /
NAI filter?

Their filter (naiavf5x.sys V7.0.0.230) crashes whenever I call
IoCompleteRequest on a CLOSE IRP from a worker thread and they are above
me on the stack! The reason my completion is running on a worker at this
point is due to the fact that my completion was called at elevated
(above PASSIVE) IRQL.

This was never a problem with naifiltr.sys (v6).

If anyone from NAI is reading this and would like a crash-dump…

/ted

I also had an interop problem with NAV (6.0 I believe), where if I
enabled Driver Verifier on just my driver, Verifier would cause a bug
check because it would see that NAVAP.SYS was doing stack switching,
even though I wasn’t actively “verifying” NAVAP.SYS. Not a big deal
(well, possibly) but kind of annoying, because I’d have to uninstall NAV
whenever I wanted to use verifier (always) to test my own drivers. I
also posted a message in this list asking for anyone from Symantec to
comment/help/whatever, but received no response. Perhaps you’ll have
better luck.

Nate

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, June 18, 2003 2:55 PM
To: File Systems Developers
Subject: [ntfsd] Problem with NAI filter V7.0

Has anyone else out there had inter-op problems with the latest McAfee /
NAI filter?

Their filter (naiavf5x.sys V7.0.0.230) crashes whenever I call
IoCompleteRequest on a CLOSE IRP from a worker thread and they are above
me on the stack! The reason my completion is running on a worker at this
point is due to the fact that my completion was called at elevated
(above PASSIVE) IRQL.

This was never a problem with naifiltr.sys (v6).

If anyone from NAI is reading this and would like a crash-dump…

/ted


You are currently subscribed to ntfsd as: xxxxx@powerquest.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Ted,

We talked to someone else recently because they were seeing the same
problem. They had an excellent analysis of the crash, that indicated NAI
was making substantial modifications to the close IRP. We’ve subsequently
been advised that this is resolved in a hot fix for Version 7.0, although I
have no independent confirmation of this.

Regards,

Tony

Tony Mason

Consulting Partner

OSR Open Systems Resources, Inc.

http://www.osr.com

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, June 18, 2003 4:55 PM
To: File Systems Developers
Subject: [ntfsd] Problem with NAI filter V7.0

Has anyone else out there had inter-op problems with the latest McAfee / NAI
filter?

Their filter (naiavf5x.sys V7.0.0.230) crashes whenever I call
IoCompleteRequest on a CLOSE IRP from a worker thread and they are above me
on the stack! The reason my completion is running on a worker at this point
is due to the fact that my completion was called at elevated (above PASSIVE)
IRQL.

This was never a problem with naifiltr.sys (v6).

If anyone from NAI is reading this and would like a crash-dump…

/ted


You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Jesus.

In my opinion (and I believed others have expressed the same views on
this list) it’s a bad idea to delay, pend, alter, or otherwise do
anything complex to or during an IRP_MJ_CLOSE. Cleanup is a much safer
time for playing around since cleanups generally occur in some
nonarbitrary context as a direct result of a user-invoked ZwClose. In
contrast, closes can be thrown down from many places as a result of
somebody (usually Cc or Mm) dereferencing a file object, including from
within the lazy-writer thread. I’ve found it extremely easy to deadlock
the system doing anything complex while holding a close.

  • Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Wednesday, June 18, 2003 2:32 PM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with NAI filter V7.0

Ted,

We talked to someone else recently because they were seeing
the same problem. They had an excellent analysis of the
crash, that indicated NAI was making substantial
modifications to the close IRP. We’ve subsequently been
advised that this is resolved in a hot fix for Version 7.0,
although I have no independent confirmation of this.

Regards,

Tony

Tony Mason

Consulting Partner

OSR Open Systems Resources, Inc.

http://www.osr.com

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, June 18, 2003 4:55 PM
To: File Systems Developers
Subject: [ntfsd] Problem with NAI filter V7.0

Has anyone else out there had inter-op problems with the latest McAfee /
NAI filter?

Their filter (naiavf5x.sys V7.0.0.230) crashes whenever I call
IoCompleteRequest on a CLOSE IRP from a worker thread and they are above
me on the stack! The reason my completion is running on a worker at this
point is due to the fact that my completion was called at elevated
(above PASSIVE) IRQL.

This was never a problem with naifiltr.sys (v6).

If anyone from NAI is reading this and would like a crash-dump…

/ted


You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@nryan.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

FWIW Nick - the NAI filters lock up my system when loaded with the OSR NULL
filter driver set. This filter set doesn’t do much more than set a
completion routine, mark the IRP pending, dispatch it and return
STATUS_PENDING. In the completion routine, if we not running at
PASSIVE_LEVEL, the IoCompleteRequest will be called from a worker thread.
All perfectly legit…

Jesus had nothing to do with it!

/ted

-----Original Message-----
From: Nick Ryan [mailto:xxxxx@nryan.com]
Sent: Thursday, June 19, 2003 2:18 AM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with NAI filter V7.0

Jesus.

In my opinion (and I believed others have expressed the same views on this
list) it’s a bad idea to delay, pend, alter, or otherwise do anything
complex to or during an IRP_MJ_CLOSE. Cleanup is a much safer time for
playing around since cleanups generally occur in some nonarbitrary context
as a direct result of a user-invoked ZwClose. In contrast, closes can be
thrown down from many places as a result of somebody (usually Cc or Mm)
dereferencing a file object, including from within the lazy-writer thread.
I’ve found it extremely easy to deadlock the system doing anything complex
while holding a close.

  • Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
Sent: Wednesday, June 18, 2003 2:32 PM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with NAI filter V7.0

Ted,

We talked to someone else recently because they were seeing
the same problem. They had an excellent analysis of the
crash, that indicated NAI was making substantial
modifications to the close IRP. We’ve subsequently been
advised that this is resolved in a hot fix for Version 7.0,
although I have no independent confirmation of this.

Regards,

Tony

Tony Mason

Consulting Partner

OSR Open Systems Resources, Inc.

http://www.osr.com

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, June 18, 2003 4:55 PM
To: File Systems Developers
Subject: [ntfsd] Problem with NAI filter V7.0

Has anyone else out there had inter-op problems with the latest McAfee / NAI
filter?

Their filter (naiavf5x.sys V7.0.0.230) crashes whenever I call
IoCompleteRequest on a CLOSE IRP from a worker thread and they are above me
on the stack! The reason my completion is running on a worker at this point
is due to the fact that my completion was called at elevated (above PASSIVE)
IRQL.

This was never a problem with naifiltr.sys (v6).

If anyone from NAI is reading this and would like a crash-dump…

/ted


You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@nryan.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@livevault.com To unsubscribe
send a blank email to xxxxx@lists.osr.com

I must confirm this problem with naiavf5x.sys too. We received the dump from
the test field (HSM filter) recently
with the 100% identical situation. I would be interested in getting the
confirmation that the problem is fixed too.

Vladimir Strogov

----- Original Message -----
From: “Ted Hess”
To: “File Systems Developers”
Sent: Thursday, June 19, 2003 7:40 PM
Subject: [ntfsd] RE: Problem with NAI filter V7.0

> FWIW Nick - the NAI filters lock up my system when loaded with the OSR
NULL
> filter driver set. This filter set doesn’t do much more than set a
> completion routine, mark the IRP pending, dispatch it and return
> STATUS_PENDING. In the completion routine, if we not running at
> PASSIVE_LEVEL, the IoCompleteRequest will be called from a worker thread.
> All perfectly legit…
>
> Jesus had nothing to do with it!
>
> /ted
>
> -----Original Message-----
> From: Nick Ryan [mailto:xxxxx@nryan.com]
> Sent: Thursday, June 19, 2003 2:18 AM
> To: File Systems Developers
> Subject: [ntfsd] RE: Problem with NAI filter V7.0
>
>
> Jesus.
>
> In my opinion (and I believed others have expressed the same views on this
> list) it’s a bad idea to delay, pend, alter, or otherwise do anything
> complex to or during an IRP_MJ_CLOSE. Cleanup is a much safer time for
> playing around since cleanups generally occur in some nonarbitrary context
> as a direct result of a user-invoked ZwClose. In contrast, closes can be
> thrown down from many places as a result of somebody (usually Cc or Mm)
> dereferencing a file object, including from within the lazy-writer thread.
> I’ve found it extremely easy to deadlock the system doing anything complex
> while holding a close.
>
> - Nick Ryan
>
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
> > Sent: Wednesday, June 18, 2003 2:32 PM
> > To: File Systems Developers
> > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> >
> >
> > Ted,
> >
> >
> >
> > We talked to someone else recently because they were seeing
> > the same problem. They had an excellent analysis of the
> > crash, that indicated NAI was making substantial
> > modifications to the close IRP. We’ve subsequently been
> > advised that this is resolved in a hot fix for Version 7.0,
> > although I have no independent confirmation of this.
> >
> >
> >
> > Regards,
> >
> >
> >
> > Tony
> >
> >
> >
> > Tony Mason
> >
> > Consulting Partner
> >
> > OSR Open Systems Resources, Inc.
> >
> http://www.osr.com
>
>
>
> -----Original Message-----
> From: Ted Hess [mailto:xxxxx@livevault.com]
> Sent: Wednesday, June 18, 2003 4:55 PM
> To: File Systems Developers
> Subject: [ntfsd] Problem with NAI filter V7.0
>
>
>
> Has anyone else out there had inter-op problems with the latest McAfee /
NAI
> filter?
>
> Their filter (naiavf5x.sys V7.0.0.230) crashes whenever I call
> IoCompleteRequest on a CLOSE IRP from a worker thread and they are above
me
> on the stack! The reason my completion is running on a worker at this
point
> is due to the fact that my completion was called at elevated (above
PASSIVE)
> IRQL.
>
> This was never a problem with naifiltr.sys (v6).
>
> If anyone from NAI is reading this and would like a crash-dump…
>
> /ted
>
> —
> You are currently subscribed to ntfsd as: xxxxx@osr.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@nryan.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe
> send a blank email to xxxxx@lists.osr.com
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@umail.ru
> To unsubscribe send a blank email to xxxxx@lists.osr.com

This issue is fixed. The hot fix can be found on the NAI support portal.

www.nai.com

Hmm… I still think that pending close requests like this is a bad idea
if the completion is being delayed to a system worker thread, since
system worker threads themselves generate and wait on closes. It should
be better if you have a thread of your own dedicated to this purpose
(assuming of course that you don’t block this thread on additional
closes).

  • Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Ted Hess
Sent: Thursday, June 19, 2003 8:41 AM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with NAI filter V7.0

FWIW Nick - the NAI filters lock up my system when loaded
with the OSR NULL filter driver set. This filter set doesn’t
do much more than set a completion routine, mark the IRP
pending, dispatch it and return STATUS_PENDING. In the
completion routine, if we not running at PASSIVE_LEVEL, the
IoCompleteRequest will be called from a worker thread. All
perfectly legit…

Jesus had nothing to do with it!

/ted

-----Original Message-----
From: Nick Ryan [mailto:xxxxx@nryan.com]
Sent: Thursday, June 19, 2003 2:18 AM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with NAI filter V7.0

Jesus.

In my opinion (and I believed others have expressed the same
views on this
list) it’s a bad idea to delay, pend, alter, or otherwise do
anything complex to or during an IRP_MJ_CLOSE. Cleanup is a
much safer time for playing around since cleanups generally
occur in some nonarbitrary context as a direct result of a
user-invoked ZwClose. In contrast, closes can be thrown down
from many places as a result of somebody (usually Cc or Mm)
dereferencing a file object, including from within the
lazy-writer thread. I’ve found it extremely easy to deadlock
the system doing anything complex while holding a close.

  • Nick Ryan

> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
> Sent: Wednesday, June 18, 2003 2:32 PM
> To: File Systems Developers
> Subject: [ntfsd] RE: Problem with NAI filter V7.0
>
>
> Ted,
>
>
>
> We talked to someone else recently because they were seeing
the same
> problem. They had an excellent analysis of the crash, that
indicated
> NAI was making substantial modifications to the close IRP. We’ve
> subsequently been advised that this is resolved in a hot fix for
> Version 7.0, although I have no independent confirmation of this.
>
>
>
> Regards,
>
>
>
> Tony
>
>
>
> Tony Mason
>
> Consulting Partner
>
> OSR Open Systems Resources, Inc.
>
http://www.osr.com

-----Original Message-----
From: Ted Hess [mailto:xxxxx@livevault.com]
Sent: Wednesday, June 18, 2003 4:55 PM
To: File Systems Developers
Subject: [ntfsd] Problem with NAI filter V7.0

Has anyone else out there had inter-op problems with the
latest McAfee / NAI filter?

Their filter (naiavf5x.sys V7.0.0.230) crashes whenever I
call IoCompleteRequest on a CLOSE IRP from a worker thread
and they are above me on the stack! The reason my completion
is running on a worker at this point is due to the fact that
my completion was called at elevated (above PASSIVE) IRQL.

This was never a problem with naifiltr.sys (v6).

If anyone from NAI is reading this and would like a crash-dump…

/ted


You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@nryan.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@livevault.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@nryan.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

We just happen to have a support contract with NAI and… We have found
nothing on their site and phone support knew of nothing.

Do you know of any incident # that might help here?

/ted

-----Original Message-----
From: Fred Walters [mailto:xxxxx@charter.net]
Sent: Thursday, June 19, 2003 1:50 PM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with NAI filter V7.0

This issue is fixed. The hot fix can be found on the NAI support portal.

www.nai.com


You are currently subscribed to ntfsd as: xxxxx@livevault.com To unsubscribe
send a blank email to xxxxx@lists.osr.com

Nick

Certainly a driver managed pool of worker threads is used for this scheme,
otherwise it would not be able to work reliably(especially on
pre- Windows 2000 systems ).

Vladimir

----- Original Message -----
From: “Nick Ryan”
To: “File Systems Developers”
Sent: Thursday, June 19, 2003 10:52 PM
Subject: [ntfsd] RE: Problem with NAI filter V7.0

> Hmm… I still think that pending close requests like this is a bad idea
> if the completion is being delayed to a system worker thread, since
> system worker threads themselves generate and wait on closes. It should
> be better if you have a thread of your own dedicated to this purpose
> (assuming of course that you don’t block this thread on additional
> closes).
>
> - Nick Ryan
>
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of Ted Hess
> > Sent: Thursday, June 19, 2003 8:41 AM
> > To: File Systems Developers
> > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> >
> >
> > FWIW Nick - the NAI filters lock up my system when loaded
> > with the OSR NULL filter driver set. This filter set doesn’t
> > do much more than set a completion routine, mark the IRP
> > pending, dispatch it and return STATUS_PENDING. In the
> > completion routine, if we not running at PASSIVE_LEVEL, the
> > IoCompleteRequest will be called from a worker thread. All
> > perfectly legit…
> >
> > Jesus had nothing to do with it!
> >
> > /ted
> >
> > -----Original Message-----
> > From: Nick Ryan [mailto:xxxxx@nryan.com]
> > Sent: Thursday, June 19, 2003 2:18 AM
> > To: File Systems Developers
> > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> >
> >
> > Jesus.
> >
> > In my opinion (and I believed others have expressed the same
> > views on this
> > list) it’s a bad idea to delay, pend, alter, or otherwise do
> > anything complex to or during an IRP_MJ_CLOSE. Cleanup is a
> > much safer time for playing around since cleanups generally
> > occur in some nonarbitrary context as a direct result of a
> > user-invoked ZwClose. In contrast, closes can be thrown down
> > from many places as a result of somebody (usually Cc or Mm)
> > dereferencing a file object, including from within the
> > lazy-writer thread. I’ve found it extremely easy to deadlock
> > the system doing anything complex while holding a close.
> >
> > - Nick Ryan
> >
> > > -----Original Message-----
> > > From: xxxxx@lists.osr.com
> > > [mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
> > > Sent: Wednesday, June 18, 2003 2:32 PM
> > > To: File Systems Developers
> > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > >
> > >
> > > Ted,
> > >
> > >
> > >
> > > We talked to someone else recently because they were seeing
> > the same
> > > problem. They had an excellent analysis of the crash, that
> > indicated
> > > NAI was making substantial modifications to the close IRP. We’ve
> > > subsequently been advised that this is resolved in a hot fix for
> > > Version 7.0, although I have no independent confirmation of this.
> > >
> > >
> > >
> > > Regards,
> > >
> > >
> > >
> > > Tony
> > >
> > >
> > >
> > > Tony Mason
> > >
> > > Consulting Partner
> > >
> > > OSR Open Systems Resources, Inc.
> > >
> > http://www.osr.com
> >
> >
> >
> > -----Original Message-----
> > From: Ted Hess [mailto:xxxxx@livevault.com]
> > Sent: Wednesday, June 18, 2003 4:55 PM
> > To: File Systems Developers
> > Subject: [ntfsd] Problem with NAI filter V7.0
> >
> >
> >
> > Has anyone else out there had inter-op problems with the
> > latest McAfee / NAI filter?
> >
> > Their filter (naiavf5x.sys V7.0.0.230) crashes whenever I
> > call IoCompleteRequest on a CLOSE IRP from a worker thread
> > and they are above me on the stack! The reason my completion
> > is running on a worker at this point is due to the fact that
> > my completion was called at elevated (above PASSIVE) IRQL.
> >
> > This was never a problem with naifiltr.sys (v6).
> >
> > If anyone from NAI is reading this and would like a crash-dump…
> >
> > /ted
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@osr.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@nryan.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@livevault.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@nryan.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@umail.ru
> To unsubscribe send a blank email to xxxxx@lists.osr.com

We too have seen this under NAI 5. They tinker with the stack counter
and pointer in the IRP and this trashes a well written driver. They say
it is not their issue, but we are 100% sure that it is. We had to patch
our code to fix their problem by restoring the IRP stack counter and
location to where it should be after NAI screws them up.

Jamey

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Strogov
Sent: Thursday, June 19, 2003 9:02 AM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with NAI filter V7.0

I must confirm this problem with naiavf5x.sys too. We received the dump
from
the test field (HSM filter) recently
with the 100% identical situation. I would be interested in getting the
confirmation that the problem is fixed too.

Vladimir Strogov

----- Original Message -----
From: “Ted Hess”
To: “File Systems Developers”
Sent: Thursday, June 19, 2003 7:40 PM
Subject: [ntfsd] RE: Problem with NAI filter V7.0

> FWIW Nick - the NAI filters lock up my system when loaded with the OSR
NULL
> filter driver set. This filter set doesn’t do much more than set a
> completion routine, mark the IRP pending, dispatch it and return
> STATUS_PENDING. In the completion routine, if we not running at
> PASSIVE_LEVEL, the IoCompleteRequest will be called from a worker
thread.
> All perfectly legit…
>
> Jesus had nothing to do with it!
>
> /ted
>
> -----Original Message-----
> From: Nick Ryan [mailto:xxxxx@nryan.com]
> Sent: Thursday, June 19, 2003 2:18 AM
> To: File Systems Developers
> Subject: [ntfsd] RE: Problem with NAI filter V7.0
>
>
> Jesus.
>
> In my opinion (and I believed others have expressed the same views on
this
> list) it’s a bad idea to delay, pend, alter, or otherwise do anything
> complex to or during an IRP_MJ_CLOSE. Cleanup is a much safer time for
> playing around since cleanups generally occur in some nonarbitrary
context
> as a direct result of a user-invoked ZwClose. In contrast, closes can
be
> thrown down from many places as a result of somebody (usually Cc or
Mm)
> dereferencing a file object, including from within the lazy-writer
thread.
> I’ve found it extremely easy to deadlock the system doing anything
complex
> while holding a close.
>
> - Nick Ryan
>
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
> > Sent: Wednesday, June 18, 2003 2:32 PM
> > To: File Systems Developers
> > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> >
> >
> > Ted,
> >
> >
> >
> > We talked to someone else recently because they were seeing
> > the same problem. They had an excellent analysis of the
> > crash, that indicated NAI was making substantial
> > modifications to the close IRP. We’ve subsequently been
> > advised that this is resolved in a hot fix for Version 7.0,
> > although I have no independent confirmation of this.
> >
> >
> >
> > Regards,
> >
> >
> >
> > Tony
> >
> >
> >
> > Tony Mason
> >
> > Consulting Partner
> >
> > OSR Open Systems Resources, Inc.
> >
> http://www.osr.com
>
>
>
> -----Original Message-----
> From: Ted Hess [mailto:xxxxx@livevault.com]
> Sent: Wednesday, June 18, 2003 4:55 PM
> To: File Systems Developers
> Subject: [ntfsd] Problem with NAI filter V7.0
>
>
>
> Has anyone else out there had inter-op problems with the latest McAfee
/
NAI
> filter?
>
> Their filter (naiavf5x.sys V7.0.0.230) crashes whenever I call
> IoCompleteRequest on a CLOSE IRP from a worker thread and they are
above
me
> on the stack! The reason my completion is running on a worker at this
point
> is due to the fact that my completion was called at elevated (above
PASSIVE)
> IRQL.
>
> This was never a problem with naifiltr.sys (v6).
>
> If anyone from NAI is reading this and would like a crash-dump…
>
> /ted
>
> —
> You are currently subscribed to ntfsd as: xxxxx@osr.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@nryan.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@livevault.com To
unsubscribe
> send a blank email to xxxxx@lists.osr.com
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@umail.ru
> To unsubscribe send a blank email to xxxxx@lists.osr.com


You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Is driver managed pool of worker threads a must? Why it is not possible to
use usual system managed worker threads?

-htfv

----- Original Message -----
From: “Vladimir Strogov”
To: “File Systems Developers”
Sent: Thursday, June 19, 2003 10:36 PM
Subject: [ntfsd] RE: Problem with NAI filter V7.0

> Nick
>
> Certainly a driver managed pool of worker threads is used for this scheme,
> otherwise it would not be able to work reliably(especially on
> pre- Windows 2000 systems ).
>
> Vladimir
>
> ----- Original Message -----
> From: “Nick Ryan”
> To: “File Systems Developers”
> Sent: Thursday, June 19, 2003 10:52 PM
> Subject: [ntfsd] RE: Problem with NAI filter V7.0
>
>
> > Hmm… I still think that pending close requests like this is a bad idea
> > if the completion is being delayed to a system worker thread, since
> > system worker threads themselves generate and wait on closes. It should
> > be better if you have a thread of your own dedicated to this purpose
> > (assuming of course that you don’t block this thread on additional
> > closes).
> >
> > - Nick Ryan
> >
> > > -----Original Message-----
> > > From: xxxxx@lists.osr.com
> > > [mailto:xxxxx@lists.osr.com] On Behalf Of Ted Hess
> > > Sent: Thursday, June 19, 2003 8:41 AM
> > > To: File Systems Developers
> > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > >
> > >
> > > FWIW Nick - the NAI filters lock up my system when loaded
> > > with the OSR NULL filter driver set. This filter set doesn’t
> > > do much more than set a completion routine, mark the IRP
> > > pending, dispatch it and return STATUS_PENDING. In the
> > > completion routine, if we not running at PASSIVE_LEVEL, the
> > > IoCompleteRequest will be called from a worker thread. All
> > > perfectly legit…
> > >
> > > Jesus had nothing to do with it!
> > >
> > > /ted
> > >
> > > -----Original Message-----
> > > From: Nick Ryan [mailto:xxxxx@nryan.com]
> > > Sent: Thursday, June 19, 2003 2:18 AM
> > > To: File Systems Developers
> > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > >
> > >
> > > Jesus.
> > >
> > > In my opinion (and I believed others have expressed the same
> > > views on this
> > > list) it’s a bad idea to delay, pend, alter, or otherwise do
> > > anything complex to or during an IRP_MJ_CLOSE. Cleanup is a
> > > much safer time for playing around since cleanups generally
> > > occur in some nonarbitrary context as a direct result of a
> > > user-invoked ZwClose. In contrast, closes can be thrown down
> > > from many places as a result of somebody (usually Cc or Mm)
> > > dereferencing a file object, including from within the
> > > lazy-writer thread. I’ve found it extremely easy to deadlock
> > > the system doing anything complex while holding a close.
> > >
> > > - Nick Ryan
> > >
> > > > -----Original Message-----
> > > > From: xxxxx@lists.osr.com
> > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
> > > > Sent: Wednesday, June 18, 2003 2:32 PM
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > >
> > > >
> > > > Ted,
> > > >
> > > >
> > > >
> > > > We talked to someone else recently because they were seeing
> > > the same
> > > > problem. They had an excellent analysis of the crash, that
> > > indicated
> > > > NAI was making substantial modifications to the close IRP. We’ve
> > > > subsequently been advised that this is resolved in a hot fix for
> > > > Version 7.0, although I have no independent confirmation of this.
> > > >
> > > >
> > > >
> > > > Regards,
> > > >
> > > >
> > > >
> > > > Tony
> > > >
> > > >
> > > >
> > > > Tony Mason
> > > >
> > > > Consulting Partner
> > > >
> > > > OSR Open Systems Resources, Inc.
> > > >
> > > http://www.osr.com
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Ted Hess [mailto:xxxxx@livevault.com]
> > > Sent: Wednesday, June 18, 2003 4:55 PM
> > > To: File Systems Developers
> > > Subject: [ntfsd] Problem with NAI filter V7.0
> > >
> > >
> > >
> > > Has anyone else out there had inter-op problems with the
> > > latest McAfee / NAI filter?
> > >
> > > Their filter (naiavf5x.sys V7.0.0.230) crashes whenever I
> > > call IoCompleteRequest on a CLOSE IRP from a worker thread
> > > and they are above me on the stack! The reason my completion
> > > is running on a worker at this point is due to the fact that
> > > my completion was called at elevated (above PASSIVE) IRQL.
> > >
> > > This was never a problem with naifiltr.sys (v6).
> > >
> > > If anyone from NAI is reading this and would like a crash-dump…
> > >
> > > /ted
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@osr.com
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@nryan.com
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@livevault.com
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@nryan.com
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@umail.ru
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@vba.com.by
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

Alexey

A system worker thread item was a limited resource under versions of NT,
prior to Windows 2000. You could introduce deadlocks (exposed under stress),
using these items in file system filters. So if you are interested in having
a more or less common source code base for file system filters for different
NT versions, it is better
to use your own worker threads pool (if you need such mechanisms).

Windows 2000 and higher can dynamically increase available worker thread
items, but I have not tried using them for such purposes.

Vladimir

----- Original Message -----
From: “Alexey Logachyov”
To: “File Systems Developers”
Sent: Sunday, June 22, 2003 3:04 PM
Subject: [ntfsd] RE: Problem with NAI filter V7.0

> Is driver managed pool of worker threads a must? Why it is not possible to
> use usual system managed worker threads?
>
> -htfv
>
>
>
> ----- Original Message -----
> From: “Vladimir Strogov”
> To: “File Systems Developers”
> Sent: Thursday, June 19, 2003 10:36 PM
> Subject: [ntfsd] RE: Problem with NAI filter V7.0
>
>
> > Nick
> >
> > Certainly a driver managed pool of worker threads is used for this
scheme,
> > otherwise it would not be able to work reliably(especially on
> > pre- Windows 2000 systems ).
> >
> > Vladimir
> >
> > ----- Original Message -----
> > From: “Nick Ryan”
> > To: “File Systems Developers”
> > Sent: Thursday, June 19, 2003 10:52 PM
> > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> >
> >
> > > Hmm… I still think that pending close requests like this is a bad
idea
> > > if the completion is being delayed to a system worker thread, since
> > > system worker threads themselves generate and wait on closes. It
should
> > > be better if you have a thread of your own dedicated to this purpose
> > > (assuming of course that you don’t block this thread on additional
> > > closes).
> > >
> > > - Nick Ryan
> > >
> > > > -----Original Message-----
> > > > From: xxxxx@lists.osr.com
> > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Ted Hess
> > > > Sent: Thursday, June 19, 2003 8:41 AM
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > >
> > > >
> > > > FWIW Nick - the NAI filters lock up my system when loaded
> > > > with the OSR NULL filter driver set. This filter set doesn’t
> > > > do much more than set a completion routine, mark the IRP
> > > > pending, dispatch it and return STATUS_PENDING. In the
> > > > completion routine, if we not running at PASSIVE_LEVEL, the
> > > > IoCompleteRequest will be called from a worker thread. All
> > > > perfectly legit…
> > > >
> > > > Jesus had nothing to do with it!
> > > >
> > > > /ted
> > > >
> > > > -----Original Message-----
> > > > From: Nick Ryan [mailto:xxxxx@nryan.com]
> > > > Sent: Thursday, June 19, 2003 2:18 AM
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > >
> > > >
> > > > Jesus.
> > > >
> > > > In my opinion (and I believed others have expressed the same
> > > > views on this
> > > > list) it’s a bad idea to delay, pend, alter, or otherwise do
> > > > anything complex to or during an IRP_MJ_CLOSE. Cleanup is a
> > > > much safer time for playing around since cleanups generally
> > > > occur in some nonarbitrary context as a direct result of a
> > > > user-invoked ZwClose. In contrast, closes can be thrown down
> > > > from many places as a result of somebody (usually Cc or Mm)
> > > > dereferencing a file object, including from within the
> > > > lazy-writer thread. I’ve found it extremely easy to deadlock
> > > > the system doing anything complex while holding a close.
> > > >
> > > > - Nick Ryan
> > > >
> > > > > -----Original Message-----
> > > > > From: xxxxx@lists.osr.com
> > > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
> > > > > Sent: Wednesday, June 18, 2003 2:32 PM
> > > > > To: File Systems Developers
> > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > >
> > > > >
> > > > > Ted,
> > > > >
> > > > >
> > > > >
> > > > > We talked to someone else recently because they were seeing
> > > > the same
> > > > > problem. They had an excellent analysis of the crash, that
> > > > indicated
> > > > > NAI was making substantial modifications to the close IRP. We’ve
> > > > > subsequently been advised that this is resolved in a hot fix for
> > > > > Version 7.0, although I have no independent confirmation of this.
> > > > >
> > > > >
> > > > >
> > > > > Regards,
> > > > >
> > > > >
> > > > >
> > > > > Tony
> > > > >
> > > > >
> > > > >
> > > > > Tony Mason
> > > > >
> > > > > Consulting Partner
> > > > >
> > > > > OSR Open Systems Resources, Inc.
> > > > >
> > > > http://www.osr.com
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Ted Hess [mailto:xxxxx@livevault.com]
> > > > Sent: Wednesday, June 18, 2003 4:55 PM
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] Problem with NAI filter V7.0
> > > >
> > > >
> > > >
> > > > Has anyone else out there had inter-op problems with the
> > > > latest McAfee / NAI filter?
> > > >
> > > > Their filter (naiavf5x.sys V7.0.0.230) crashes whenever I
> > > > call IoCompleteRequest on a CLOSE IRP from a worker thread
> > > > and they are above me on the stack! The reason my completion
> > > > is running on a worker at this point is due to the fact that
> > > > my completion was called at elevated (above PASSIVE) IRQL.
> > > >
> > > > This was never a problem with naifiltr.sys (v6).
> > > >
> > > > If anyone from NAI is reading this and would like a crash-dump…
> > > >
> > > > /ted
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@osr.com
> > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@nryan.com
> > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@livevault.com
> > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@nryan.com
> > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > >
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@umail.ru
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@vba.com.by
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@umail.ru
> To unsubscribe send a blank email to xxxxx@lists.osr.com

Several years ago OSR wrote an article in NT Insider about using your own
worker threads. Look for it. Works and gives you complete control. System
worker threads were designed for use by file systems.

“Alexey Logachyov” wrote in message news:xxxxx@ntfsd…
>
> Is driver managed pool of worker threads a must? Why it is not possible to
> use usual system managed worker threads?
>
> -htfv
>
>
>
> ----- Original Message -----
> From: “Vladimir Strogov”
> To: “File Systems Developers”
> Sent: Thursday, June 19, 2003 10:36 PM
> Subject: [ntfsd] RE: Problem with NAI filter V7.0
>
>
> > Nick
> >
> > Certainly a driver managed pool of worker threads is used for this
scheme,
> > otherwise it would not be able to work reliably(especially on
> > pre- Windows 2000 systems ).
> >
> > Vladimir
> >
> > ----- Original Message -----
> > From: “Nick Ryan”
> > To: “File Systems Developers”
> > Sent: Thursday, June 19, 2003 10:52 PM
> > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> >
> >
> > > Hmm… I still think that pending close requests like this is a bad
idea
> > > if the completion is being delayed to a system worker thread, since
> > > system worker threads themselves generate and wait on closes. It
should
> > > be better if you have a thread of your own dedicated to this purpose
> > > (assuming of course that you don’t block this thread on additional
> > > closes).
> > >
> > > - Nick Ryan
> > >
> > > > -----Original Message-----
> > > > From: xxxxx@lists.osr.com
> > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Ted Hess
> > > > Sent: Thursday, June 19, 2003 8:41 AM
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > >
> > > >
> > > > FWIW Nick - the NAI filters lock up my system when loaded
> > > > with the OSR NULL filter driver set. This filter set doesn’t
> > > > do much more than set a completion routine, mark the IRP
> > > > pending, dispatch it and return STATUS_PENDING. In the
> > > > completion routine, if we not running at PASSIVE_LEVEL, the
> > > > IoCompleteRequest will be called from a worker thread. All
> > > > perfectly legit…
> > > >
> > > > Jesus had nothing to do with it!
> > > >
> > > > /ted
> > > >
> > > > -----Original Message-----
> > > > From: Nick Ryan [mailto:xxxxx@nryan.com]
> > > > Sent: Thursday, June 19, 2003 2:18 AM
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > >
> > > >
> > > > Jesus.
> > > >
> > > > In my opinion (and I believed others have expressed the same
> > > > views on this
> > > > list) it’s a bad idea to delay, pend, alter, or otherwise do
> > > > anything complex to or during an IRP_MJ_CLOSE. Cleanup is a
> > > > much safer time for playing around since cleanups generally
> > > > occur in some nonarbitrary context as a direct result of a
> > > > user-invoked ZwClose. In contrast, closes can be thrown down
> > > > from many places as a result of somebody (usually Cc or Mm)
> > > > dereferencing a file object, including from within the
> > > > lazy-writer thread. I’ve found it extremely easy to deadlock
> > > > the system doing anything complex while holding a close.
> > > >
> > > > - Nick Ryan
> > > >
> > > > > -----Original Message-----
> > > > > From: xxxxx@lists.osr.com
> > > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
> > > > > Sent: Wednesday, June 18, 2003 2:32 PM
> > > > > To: File Systems Developers
> > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > >
> > > > >
> > > > > Ted,
> > > > >
> > > > >
> > > > >
> > > > > We talked to someone else recently because they were seeing
> > > > the same
> > > > > problem. They had an excellent analysis of the crash, that
> > > > indicated
> > > > > NAI was making substantial modifications to the close IRP. We’ve
> > > > > subsequently been advised that this is resolved in a hot fix for
> > > > > Version 7.0, although I have no independent confirmation of this.
> > > > >
> > > > >
> > > > >
> > > > > Regards,
> > > > >
> > > > >
> > > > >
> > > > > Tony
> > > > >
> > > > >
> > > > >
> > > > > Tony Mason
> > > > >
> > > > > Consulting Partner
> > > > >
> > > > > OSR Open Systems Resources, Inc.
> > > > >
> > > > http://www.osr.com
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Ted Hess [mailto:xxxxx@livevault.com]
> > > > Sent: Wednesday, June 18, 2003 4:55 PM
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] Problem with NAI filter V7.0
> > > >
> > > >
> > > >
> > > > Has anyone else out there had inter-op problems with the
> > > > latest McAfee / NAI filter?
> > > >
> > > > Their filter (naiavf5x.sys V7.0.0.230) crashes whenever I
> > > > call IoCompleteRequest on a CLOSE IRP from a worker thread
> > > > and they are above me on the stack! The reason my completion
> > > > is running on a worker at this point is due to the fact that
> > > > my completion was called at elevated (above PASSIVE) IRQL.
> > > >
> > > > This was never a problem with naifiltr.sys (v6).
> > > >
> > > > If anyone from NAI is reading this and would like a crash-dump…
> > > >
> > > > /ted
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@osr.com
> > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@nryan.com
> > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@livevault.com
> > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@nryan.com
> > > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > > >
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@umail.ru
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@vba.com.by
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
>
>
>
>

Although it’s true 2k&later can spawn additional worker threads, there
are some worker threads that are ‘special’ under Windows and cannot be
cloned, such as the lazy writer thread. Blocking the lazy writer thread
and performing cached I/O on some other file, for example, may deadlock
because the I/O may in turn cause Cc to wait for the lazy writer to free
up pages.

  • Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Strogov
Sent: Sunday, June 22, 2003 8:26 AM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with NAI filter V7.0

Alexey

A system worker thread item was a limited resource under
versions of NT, prior to Windows 2000. You could introduce
deadlocks (exposed under stress), using these items in file
system filters. So if you are interested in having a more or
less common source code base for file system filters for
different NT versions, it is better to use your own worker
threads pool (if you need such mechanisms).

Windows 2000 and higher can dynamically increase available
worker thread items, but I have not tried using them for such
purposes.

Vladimir

----- Original Message -----
From: “Alexey Logachyov”
> To: “File Systems Developers”
> Sent: Sunday, June 22, 2003 3:04 PM
> Subject: [ntfsd] RE: Problem with NAI filter V7.0
>
>
> > Is driver managed pool of worker threads a must? Why it is not
> > possible to use usual system managed worker threads?
> >
> > -htfv
> >
> >
> >
> > ----- Original Message -----
> > From: “Vladimir Strogov”
> > To: “File Systems Developers”
> > Sent: Thursday, June 19, 2003 10:36 PM
> > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> >
> >
> > > Nick
> > >
> > > Certainly a driver managed pool of worker threads is used for this
> scheme,
> > > otherwise it would not be able to work reliably(especially on
> > > pre- Windows 2000 systems ).
> > >
> > > Vladimir
> > >
> > > ----- Original Message -----
> > > From: “Nick Ryan”
> > > To: “File Systems Developers”
> > > Sent: Thursday, June 19, 2003 10:52 PM
> > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > >
> > >
> > > > Hmm… I still think that pending close requests like this is a
> > > > bad
> idea
> > > > if the completion is being delayed to a system worker thread,
> > > > since system worker threads themselves generate and wait on
> > > > closes. It
> should
> > > > be better if you have a thread of your own dedicated to this
> > > > purpose (assuming of course that you don’t block this thread on
> > > > additional closes).
> > > >
> > > > - Nick Ryan
> > > >
> > > > > -----Original Message-----
> > > > > From: xxxxx@lists.osr.com
> > > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Ted Hess
> > > > > Sent: Thursday, June 19, 2003 8:41 AM
> > > > > To: File Systems Developers
> > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > >
> > > > >
> > > > > FWIW Nick - the NAI filters lock up my system when
> loaded with
> > > > > the OSR NULL filter driver set. This filter set
> doesn’t do much
> > > > > more than set a completion routine, mark the IRP pending,
> > > > > dispatch it and return STATUS_PENDING. In the completion
> > > > > routine, if we not running at PASSIVE_LEVEL, the
> > > > > IoCompleteRequest will be called from a worker thread. All
> > > > > perfectly legit…
> > > > >
> > > > > Jesus had nothing to do with it!
> > > > >
> > > > > /ted
> > > > >
> > > > > -----Original Message-----
> > > > > From: Nick Ryan [mailto:xxxxx@nryan.com]
> > > > > Sent: Thursday, June 19, 2003 2:18 AM
> > > > > To: File Systems Developers
> > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > >
> > > > >
> > > > > Jesus.
> > > > >
> > > > > In my opinion (and I believed others have expressed the same
> > > > > views on this
> > > > > list) it’s a bad idea to delay, pend, alter, or otherwise do
> > > > > anything complex to or during an IRP_MJ_CLOSE.
> Cleanup is a much
> > > > > safer time for playing around since cleanups
> generally occur in
> > > > > some nonarbitrary context as a direct result of a
> user-invoked
> > > > > ZwClose. In contrast, closes can be thrown down from
> many places
> > > > > as a result of somebody (usually Cc or Mm)
> dereferencing a file
> > > > > object, including from within the lazy-writer thread.
> I’ve found
> > > > > it extremely easy to deadlock the system doing
> anything complex
> > > > > while holding a close.
> > > > >
> > > > > - Nick Ryan
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: xxxxx@lists.osr.com
> > > > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Tony
> > > > > > Mason
> > > > > > Sent: Wednesday, June 18, 2003 2:32 PM
> > > > > > To: File Systems Developers
> > > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > > >
> > > > > >
> > > > > > Ted,
> > > > > >
> > > > > >
> > > > > >
> > > > > > We talked to someone else recently because they were seeing
> > > > > the same
> > > > > > problem. They had an excellent analysis of the crash, that
> > > > > indicated
> > > > > > NAI was making substantial modifications to the close IRP.
> > > > > > We’ve subsequently been advised that this is
> resolved in a hot
> > > > > > fix for Version 7.0, although I have no independent
> > > > > > confirmation of this.
> > > > > >
> > > > > >
> > > > > >
> > > > > > Regards,
> > > > > >
> > > > > >
> > > > > >
> > > > > > Tony
> > > > > >
> > > > > >
> > > > > >
> > > > > > Tony Mason
> > > > > >
> > > > > > Consulting Partner
> > > > > >
> > > > > > OSR Open Systems Resources, Inc.
> > > > > >
> > > > > http://www.osr.com
> > > > >
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Ted Hess [mailto:xxxxx@livevault.com]
> > > > > Sent: Wednesday, June 18, 2003 4:55 PM
> > > > > To: File Systems Developers
> > > > > Subject: [ntfsd] Problem with NAI filter V7.0
> > > > >
> > > > >
> > > > >
> > > > > Has anyone else out there had inter-op problems with
> the latest
> > > > > McAfee / NAI filter?
> > > > >
> > > > > Their filter (naiavf5x.sys V7.0.0.230) crashes
> whenever I call
> > > > > IoCompleteRequest on a CLOSE IRP from a worker thread
> and they
> > > > > are above me on the stack! The reason my completion
> is running
> > > > > on a worker at this point is due to the fact that my
> completion
> > > > > was called at elevated (above PASSIVE) IRQL.
> > > > >
> > > > > This was never a problem with naifiltr.sys (v6).
> > > > >
> > > > > If anyone from NAI is reading this and would like a
> > > > > crash-dump…
> > > > >
> > > > > /ted
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as: xxxxx@osr.com To
> > > > > unsubscribe send a blank email to
> > > > > xxxxx@lists.osr.com
> > > > >
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as: xxxxx@nryan.com To
> > > > > unsubscribe send a blank email to
> > > > > xxxxx@lists.osr.com
> > > > >
> > > > >
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as:
> xxxxx@livevault.com To
> > > > > unsubscribe send a blank email to
> > > > > xxxxx@lists.osr.com
> > > > >
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as: xxxxx@nryan.com To
> > > > > unsubscribe send a blank email to
> > > > > xxxxx@lists.osr.com
> > > > >
> > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as:
> > > > xxxxx@umail.ru To unsubscribe send a blank email to
> > > > xxxxx@lists.osr.com
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@vba.com.by To
> > > unsubscribe send a blank email to xxxxx@lists.osr.com
> > >
> >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as:
> xxxxx@umail.ru To
> > unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@nryan.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

Yes, certainly it is true (deadlocks are possible when blocking the lazy
writer thread and performing cached IO), but the discussed case was mainly
about devising schemes of post-processing requests at the PASSIVE_LEVEL
instead of the elevated IRQL (there is no blocking of the lazy writer
thread, for example).

Vladimir

----- Original Message -----
From: “Nick Ryan”
To: “File Systems Developers”
Sent: Sunday, June 22, 2003 11:37 PM
Subject: [ntfsd] RE: Problem with NAI filter V7.0

> Although it’s true 2k&later can spawn additional worker threads, there
> are some worker threads that are ‘special’ under Windows and cannot be
> cloned, such as the lazy writer thread. Blocking the lazy writer thread
> and performing cached I/O on some other file, for example, may deadlock
> because the I/O may in turn cause Cc to wait for the lazy writer to free
> up pages.
>
> - Nick Ryan
>
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Strogov
> > Sent: Sunday, June 22, 2003 8:26 AM
> > To: File Systems Developers
> > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> >
> >
> > Alexey
> >
> > A system worker thread item was a limited resource under
> > versions of NT, prior to Windows 2000. You could introduce
> > deadlocks (exposed under stress), using these items in file
> > system filters. So if you are interested in having a more or
> > less common source code base for file system filters for
> > different NT versions, it is better to use your own worker
> > threads pool (if you need such mechanisms).
> >
> > Windows 2000 and higher can dynamically increase available
> > worker thread items, but I have not tried using them for such
> > purposes.
> >
> > Vladimir
> >
> >
> > ----- Original Message -----
> > From: “Alexey Logachyov”
> > To: “File Systems Developers”
> > Sent: Sunday, June 22, 2003 3:04 PM
> > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> >
> >
> > > Is driver managed pool of worker threads a must? Why it is not
> > > possible to use usual system managed worker threads?
> > >
> > > -htfv
> > >
> > >
> > >
> > > ----- Original Message -----
> > > From: “Vladimir Strogov”
> > > To: “File Systems Developers”
> > > Sent: Thursday, June 19, 2003 10:36 PM
> > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > >
> > >
> > > > Nick
> > > >
> > > > Certainly a driver managed pool of worker threads is used for this
> > scheme,
> > > > otherwise it would not be able to work reliably(especially on
> > > > pre- Windows 2000 systems ).
> > > >
> > > > Vladimir
> > > >
> > > > ----- Original Message -----
> > > > From: “Nick Ryan”
> > > > To: “File Systems Developers”
> > > > Sent: Thursday, June 19, 2003 10:52 PM
> > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > >
> > > >
> > > > > Hmm… I still think that pending close requests like this is a
> > > > > bad
> > idea
> > > > > if the completion is being delayed to a system worker thread,
> > > > > since system worker threads themselves generate and wait on
> > > > > closes. It
> > should
> > > > > be better if you have a thread of your own dedicated to this
> > > > > purpose (assuming of course that you don’t block this thread on
> > > > > additional closes).
> > > > >
> > > > > - Nick Ryan
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: xxxxx@lists.osr.com
> > > > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Ted Hess
> > > > > > Sent: Thursday, June 19, 2003 8:41 AM
> > > > > > To: File Systems Developers
> > > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > > >
> > > > > >
> > > > > > FWIW Nick - the NAI filters lock up my system when
> > loaded with
> > > > > > the OSR NULL filter driver set. This filter set
> > doesn’t do much
> > > > > > more than set a completion routine, mark the IRP pending,
> > > > > > dispatch it and return STATUS_PENDING. In the completion
> > > > > > routine, if we not running at PASSIVE_LEVEL, the
> > > > > > IoCompleteRequest will be called from a worker thread. All
> > > > > > perfectly legit…
> > > > > >
> > > > > > Jesus had nothing to do with it!
> > > > > >
> > > > > > /ted
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Nick Ryan [mailto:xxxxx@nryan.com]
> > > > > > Sent: Thursday, June 19, 2003 2:18 AM
> > > > > > To: File Systems Developers
> > > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > > >
> > > > > >
> > > > > > Jesus.
> > > > > >
> > > > > > In my opinion (and I believed others have expressed the same
> > > > > > views on this
> > > > > > list) it’s a bad idea to delay, pend, alter, or otherwise do
> > > > > > anything complex to or during an IRP_MJ_CLOSE.
> > Cleanup is a much
> > > > > > safer time for playing around since cleanups
> > generally occur in
> > > > > > some nonarbitrary context as a direct result of a
> > user-invoked
> > > > > > ZwClose. In contrast, closes can be thrown down from
> > many places
> > > > > > as a result of somebody (usually Cc or Mm)
> > dereferencing a file
> > > > > > object, including from within the lazy-writer thread.
> > I’ve found
> > > > > > it extremely easy to deadlock the system doing
> > anything complex
> > > > > > while holding a close.
> > > > > >
> > > > > > - Nick Ryan
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: xxxxx@lists.osr.com
> > > > > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Tony
> > > > > > > Mason
> > > > > > > Sent: Wednesday, June 18, 2003 2:32 PM
> > > > > > > To: File Systems Developers
> > > > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > > > >
> > > > > > >
> > > > > > > Ted,
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > We talked to someone else recently because they were seeing
> > > > > > the same
> > > > > > > problem. They had an excellent analysis of the crash, that
> > > > > > indicated
> > > > > > > NAI was making substantial modifications to the close IRP.
> > > > > > > We’ve subsequently been advised that this is
> > resolved in a hot
> > > > > > > fix for Version 7.0, although I have no independent
> > > > > > > confirmation of this.
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Regards,
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Tony
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Tony Mason
> > > > > > >
> > > > > > > Consulting Partner
> > > > > > >
> > > > > > > OSR Open Systems Resources, Inc.
> > > > > > >
> > > > > > http://www.osr.com
> > > > > >
> > > > > >
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Ted Hess [mailto:xxxxx@livevault.com]
> > > > > > Sent: Wednesday, June 18, 2003 4:55 PM
> > > > > > To: File Systems Developers
> > > > > > Subject: [ntfsd] Problem with NAI filter V7.0
> > > > > >
> > > > > >
> > > > > >
> > > > > > Has anyone else out there had inter-op problems with
> > the latest
> > > > > > McAfee / NAI filter?
> > > > > >
> > > > > > Their filter (naiavf5x.sys V7.0.0.230) crashes
> > whenever I call
> > > > > > IoCompleteRequest on a CLOSE IRP from a worker thread
> > and they
> > > > > > are above me on the stack! The reason my completion
> > is running
> > > > > > on a worker at this point is due to the fact that my
> > completion
> > > > > > was called at elevated (above PASSIVE) IRQL.
> > > > > >
> > > > > > This was never a problem with naifiltr.sys (v6).
> > > > > >
> > > > > > If anyone from NAI is reading this and would like a
> > > > > > crash-dump…
> > > > > >
> > > > > > /ted
> > > > > >
> > > > > > —
> > > > > > You are currently subscribed to ntfsd as: xxxxx@osr.com To
> > > > > > unsubscribe send a blank email to
> > > > > > xxxxx@lists.osr.com
> > > > > >
> > > > > >
> > > > > > —
> > > > > > You are currently subscribed to ntfsd as: xxxxx@nryan.com To
> > > > > > unsubscribe send a blank email to
> > > > > > xxxxx@lists.osr.com
> > > > > >
> > > > > >
> > > > > >
> > > > > > —
> > > > > > You are currently subscribed to ntfsd as:
> > xxxxx@livevault.com To
> > > > > > unsubscribe send a blank email to
> > > > > > xxxxx@lists.osr.com
> > > > > >
> > > > > >
> > > > > > —
> > > > > > You are currently subscribed to ntfsd as: xxxxx@nryan.com To
> > > > > > unsubscribe send a blank email to
> > > > > > xxxxx@lists.osr.com
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as:
> > > > > xxxxx@umail.ru To unsubscribe send a blank email to
> > > > > xxxxx@lists.osr.com
> > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@vba.com.by To
> > > > unsubscribe send a blank email to xxxxx@lists.osr.com
> > > >
> > >
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as:
> > xxxxx@umail.ru To
> > > unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@nryan.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@umail.ru
> To unsubscribe send a blank email to xxxxx@lists.osr.com

Yes, there is blocking of the lazy writer thread in the case that
originally spawned this discussion. What was being discussed was the
technique of blocking the completion processing of IRP_MJ_CLOSE (for
some undefined purpose). I mentioned that one should be careful what one
does blocking file closes because the lazy writer thread generates and
blocks on file closes itself. Cc uses the lazy writer thread to cleanup
its shared cache maps when their reference counts go to zero, and in the
process of doing this Cc invokes ObDereferenceObject on its captive file
objects, which generates a close internally.

  • Nick Ryan

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Strogov
Sent: Sunday, June 22, 2003 1:35 PM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with NAI filter V7.0

Yes, certainly it is true (deadlocks are possible when
blocking the lazy writer thread and performing cached IO),
but the discussed case was mainly about devising schemes of
post-processing requests at the PASSIVE_LEVEL instead of the
elevated IRQL (there is no blocking of the lazy writer
thread, for example).

Vladimir

----- Original Message -----
From: “Nick Ryan”
> To: “File Systems Developers”
> Sent: Sunday, June 22, 2003 11:37 PM
> Subject: [ntfsd] RE: Problem with NAI filter V7.0
>
>
> > Although it’s true 2k&later can spawn additional worker
> threads, there
> > are some worker threads that are ‘special’ under Windows
> and cannot be
> > cloned, such as the lazy writer thread. Blocking the lazy writer
> > thread and performing cached I/O on some other file, for
> example, may
> > deadlock because the I/O may in turn cause Cc to wait for the lazy
> > writer to free up pages.
> >
> > - Nick Ryan
> >
> > > -----Original Message-----
> > > From: xxxxx@lists.osr.com
> > > [mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir
> > > Strogov
> > > Sent: Sunday, June 22, 2003 8:26 AM
> > > To: File Systems Developers
> > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > >
> > >
> > > Alexey
> > >
> > > A system worker thread item was a limited resource under
> versions of
> > > NT, prior to Windows 2000. You could introduce deadlocks (exposed
> > > under stress), using these items in file system filters.
> So if you
> > > are interested in having a more or less common source
> code base for
> > > file system filters for different NT versions, it is
> better to use
> > > your own worker threads pool (if you need such mechanisms).
> > >
> > > Windows 2000 and higher can dynamically increase available worker
> > > thread items, but I have not tried using them for such purposes.
> > >
> > > Vladimir
> > >
> > >
> > > ----- Original Message -----
> > > From: “Alexey Logachyov”
> > > To: “File Systems Developers”
> > > Sent: Sunday, June 22, 2003 3:04 PM
> > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > >
> > >
> > > > Is driver managed pool of worker threads a must? Why it is not
> > > > possible to use usual system managed worker threads?
> > > >
> > > > -htfv
> > > >
> > > >
> > > >
> > > > ----- Original Message -----
> > > > From: “Vladimir Strogov”
> > > > To: “File Systems Developers”
> > > > Sent: Thursday, June 19, 2003 10:36 PM
> > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > >
> > > >
> > > > > Nick
> > > > >
> > > > > Certainly a driver managed pool of worker threads is used for
> > > > > this
> > > scheme,
> > > > > otherwise it would not be able to work reliably(especially on
> > > > > pre- Windows 2000 systems ).
> > > > >
> > > > > Vladimir
> > > > >
> > > > > ----- Original Message -----
> > > > > From: “Nick Ryan”
> > > > > To: “File Systems Developers”
> > > > > Sent: Thursday, June 19, 2003 10:52 PM
> > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > >
> > > > >
> > > > > > Hmm… I still think that pending close requests
> like this is
> > > > > > a bad
> > > idea
> > > > > > if the completion is being delayed to a system
> worker thread,
> > > > > > since system worker threads themselves generate and wait on
> > > > > > closes. It
> > > should
> > > > > > be better if you have a thread of your own
> dedicated to this
> > > > > > purpose (assuming of course that you don’t block
> this thread
> > > > > > on additional closes).
> > > > > >
> > > > > > - Nick Ryan
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: xxxxx@lists.osr.com
> > > > > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Ted
> > > > > > > Hess
> > > > > > > Sent: Thursday, June 19, 2003 8:41 AM
> > > > > > > To: File Systems Developers
> > > > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > > > >
> > > > > > >
> > > > > > > FWIW Nick - the NAI filters lock up my system when
> > > loaded with
> > > > > > > the OSR NULL filter driver set. This filter set
> > > doesn’t do much
> > > > > > > more than set a completion routine, mark the IRP pending,
> > > > > > > dispatch it and return STATUS_PENDING. In the completion
> > > > > > > routine, if we not running at PASSIVE_LEVEL, the
> > > > > > > IoCompleteRequest will be called from a worker
> thread. All
> > > > > > > perfectly legit…
> > > > > > >
> > > > > > > Jesus had nothing to do with it!
> > > > > > >
> > > > > > > /ted
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Nick Ryan [mailto:xxxxx@nryan.com]
> > > > > > > Sent: Thursday, June 19, 2003 2:18 AM
> > > > > > > To: File Systems Developers
> > > > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > > > >
> > > > > > >
> > > > > > > Jesus.
> > > > > > >
> > > > > > > In my opinion (and I believed others have
> expressed the same
> > > > > > > views on this
> > > > > > > list) it’s a bad idea to delay, pend, alter, or
> otherwise do
> > > > > > > anything complex to or during an IRP_MJ_CLOSE.
> > > Cleanup is a much
> > > > > > > safer time for playing around since cleanups
> > > generally occur in
> > > > > > > some nonarbitrary context as a direct result of a
> > > user-invoked
> > > > > > > ZwClose. In contrast, closes can be thrown down from
> > > many places
> > > > > > > as a result of somebody (usually Cc or Mm)
> > > dereferencing a file
> > > > > > > object, including from within the lazy-writer thread.
> > > I’ve found
> > > > > > > it extremely easy to deadlock the system doing
> > > anything complex
> > > > > > > while holding a close.
> > > > > > >
> > > > > > > - Nick Ryan
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: xxxxx@lists.osr.com
> > > > > > > > [mailto:xxxxx@lists.osr.com] On
> Behalf Of Tony
> > > > > > > > Mason
> > > > > > > > Sent: Wednesday, June 18, 2003 2:32 PM
> > > > > > > > To: File Systems Developers
> > > > > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > > > > >
> > > > > > > >
> > > > > > > > Ted,
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > We talked to someone else recently because they were
> > > > > > > > seeing
> > > > > > > the same
> > > > > > > > problem. They had an excellent analysis of the crash,
> > > > > > > > that
> > > > > > > indicated
> > > > > > > > NAI was making substantial modifications to the
> close IRP.
> > > > > > > > We’ve subsequently been advised that this is
> > > resolved in a hot
> > > > > > > > fix for Version 7.0, although I have no independent
> > > > > > > > confirmation of this.
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Tony
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Tony Mason
> > > > > > > >
> > > > > > > > Consulting Partner
> > > > > > > >
> > > > > > > > OSR Open Systems Resources, Inc.
> > > > > > > >
> > > > > > > http://www.osr.com
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Ted Hess [mailto:xxxxx@livevault.com]
> > > > > > > Sent: Wednesday, June 18, 2003 4:55 PM
> > > > > > > To: File Systems Developers
> > > > > > > Subject: [ntfsd] Problem with NAI filter V7.0
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > Has anyone else out there had inter-op problems with
> > > the latest
> > > > > > > McAfee / NAI filter?
> > > > > > >
> > > > > > > Their filter (naiavf5x.sys V7.0.0.230) crashes
> > > whenever I call
> > > > > > > IoCompleteRequest on a CLOSE IRP from a worker thread
> > > and they
> > > > > > > are above me on the stack! The reason my completion
> > > is running
> > > > > > > on a worker at this point is due to the fact that my
> > > completion
> > > > > > > was called at elevated (above PASSIVE) IRQL.
> > > > > > >
> > > > > > > This was never a problem with naifiltr.sys (v6).
> > > > > > >
> > > > > > > If anyone from NAI is reading this and would like a
> > > > > > > crash-dump…
> > > > > > >
> > > > > > > /ted
> > > > > > >
> > > > > > > —
> > > > > > > You are currently subscribed to ntfsd as:
> xxxxx@osr.com To
> > > > > > > unsubscribe send a blank email to
> > > > > > > xxxxx@lists.osr.com
> > > > > > >
> > > > > > >
> > > > > > > —
> > > > > > > You are currently subscribed to ntfsd as:
> xxxxx@nryan.com To
> > > > > > > unsubscribe send a blank email to
> > > > > > > xxxxx@lists.osr.com
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > —
> > > > > > > You are currently subscribed to ntfsd as:
> > > xxxxx@livevault.com To
> > > > > > > unsubscribe send a blank email to
> > > > > > > xxxxx@lists.osr.com
> > > > > > >
> > > > > > >
> > > > > > > —
> > > > > > > You are currently subscribed to ntfsd as:
> xxxxx@nryan.com To
> > > > > > > unsubscribe send a blank email to
> > > > > > > xxxxx@lists.osr.com
> > > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > —
> > > > > > You are currently subscribed to ntfsd as:
> > > > > > xxxxx@umail.ru To unsubscribe send a
> blank email to
> > > > > > xxxxx@lists.osr.com
> > > > >
> > > > >
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as: xxxxx@vba.com.by To
> > > > > unsubscribe send a blank email to
> > > > > xxxxx@lists.osr.com
> > > > >
> > > >
> > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as:
> > > xxxxx@umail.ru To
> > > > unsubscribe send a blank email to
> xxxxx@lists.osr.com
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@nryan.com To
> > > unsubscribe send a blank email to xxxxx@lists.osr.com
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as:
> xxxxx@umail.ru To
> > unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@nryan.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>

Because there are a limited number of worker threads. And, since you are
under the FSD, if an FSD has them all used up, your disk driver will
deadlock waiting for one while the FSD is waiting for the disk to
complete.

USE SYSTEM THREADS!

Jamey

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Alexey Logachyov
Sent: Sunday, June 22, 2003 4:04 AM
To: File Systems Developers
Subject: [ntfsd] RE: Problem with NAI filter V7.0

Is driver managed pool of worker threads a must? Why it is not possible
to
use usual system managed worker threads?

-htfv

----- Original Message -----
From: “Vladimir Strogov”
To: “File Systems Developers”
Sent: Thursday, June 19, 2003 10:36 PM
Subject: [ntfsd] RE: Problem with NAI filter V7.0

> Nick
>
> Certainly a driver managed pool of worker threads is used for this
scheme,
> otherwise it would not be able to work reliably(especially on
> pre- Windows 2000 systems ).
>
> Vladimir
>
> ----- Original Message -----
> From: “Nick Ryan”
> To: “File Systems Developers”
> Sent: Thursday, June 19, 2003 10:52 PM
> Subject: [ntfsd] RE: Problem with NAI filter V7.0
>
>
> > Hmm… I still think that pending close requests like this is a bad
idea
> > if the completion is being delayed to a system worker thread, since
> > system worker threads themselves generate and wait on closes. It
should
> > be better if you have a thread of your own dedicated to this purpose
> > (assuming of course that you don’t block this thread on additional
> > closes).
> >
> > - Nick Ryan
> >
> > > -----Original Message-----
> > > From: xxxxx@lists.osr.com
> > > [mailto:xxxxx@lists.osr.com] On Behalf Of Ted Hess
> > > Sent: Thursday, June 19, 2003 8:41 AM
> > > To: File Systems Developers
> > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > >
> > >
> > > FWIW Nick - the NAI filters lock up my system when loaded
> > > with the OSR NULL filter driver set. This filter set doesn’t
> > > do much more than set a completion routine, mark the IRP
> > > pending, dispatch it and return STATUS_PENDING. In the
> > > completion routine, if we not running at PASSIVE_LEVEL, the
> > > IoCompleteRequest will be called from a worker thread. All
> > > perfectly legit…
> > >
> > > Jesus had nothing to do with it!
> > >
> > > /ted
> > >
> > > -----Original Message-----
> > > From: Nick Ryan [mailto:xxxxx@nryan.com]
> > > Sent: Thursday, June 19, 2003 2:18 AM
> > > To: File Systems Developers
> > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > >
> > >
> > > Jesus.
> > >
> > > In my opinion (and I believed others have expressed the same
> > > views on this
> > > list) it’s a bad idea to delay, pend, alter, or otherwise do
> > > anything complex to or during an IRP_MJ_CLOSE. Cleanup is a
> > > much safer time for playing around since cleanups generally
> > > occur in some nonarbitrary context as a direct result of a
> > > user-invoked ZwClose. In contrast, closes can be thrown down
> > > from many places as a result of somebody (usually Cc or Mm)
> > > dereferencing a file object, including from within the
> > > lazy-writer thread. I’ve found it extremely easy to deadlock
> > > the system doing anything complex while holding a close.
> > >
> > > - Nick Ryan
> > >
> > > > -----Original Message-----
> > > > From: xxxxx@lists.osr.com
> > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Tony Mason
> > > > Sent: Wednesday, June 18, 2003 2:32 PM
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > >
> > > >
> > > > Ted,
> > > >
> > > >
> > > >
> > > > We talked to someone else recently because they were seeing
> > > the same
> > > > problem. They had an excellent analysis of the crash, that
> > > indicated
> > > > NAI was making substantial modifications to the close IRP.
We’ve
> > > > subsequently been advised that this is resolved in a hot fix for
> > > > Version 7.0, although I have no independent confirmation of
this.
> > > >
> > > >
> > > >
> > > > Regards,
> > > >
> > > >
> > > >
> > > > Tony
> > > >
> > > >
> > > >
> > > > Tony Mason
> > > >
> > > > Consulting Partner
> > > >
> > > > OSR Open Systems Resources, Inc.
> > > >
> > > http://www.osr.com
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Ted Hess [mailto:xxxxx@livevault.com]
> > > Sent: Wednesday, June 18, 2003 4:55 PM
> > > To: File Systems Developers
> > > Subject: [ntfsd] Problem with NAI filter V7.0
> > >
> > >
> > >
> > > Has anyone else out there had inter-op problems with the
> > > latest McAfee / NAI filter?
> > >
> > > Their filter (naiavf5x.sys V7.0.0.230) crashes whenever I
> > > call IoCompleteRequest on a CLOSE IRP from a worker thread
> > > and they are above me on the stack! The reason my completion
> > > is running on a worker at this point is due to the fact that
> > > my completion was called at elevated (above PASSIVE) IRQL.
> > >
> > > This was never a problem with naifiltr.sys (v6).
> > >
> > > If anyone from NAI is reading this and would like a crash-dump…
> > >
> > > /ted
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@osr.com
> > > To unsubscribe send a blank email to
xxxxx@lists.osr.com
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@nryan.com
> > > To unsubscribe send a blank email to
xxxxx@lists.osr.com
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@livevault.com
> > > To unsubscribe send a blank email to
xxxxx@lists.osr.com
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@nryan.com
> > > To unsubscribe send a blank email to
xxxxx@lists.osr.com
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@umail.ru
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@vba.com.by
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>


You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Yes, there is no blocking in the dispatch handler, but the lazy writer
performs this operation in a synchronous (blocking)
manner and effectively a delay in the completion processing blocks the lazy
writer thread for some time.

But the technique of post-processing requests by the file system filter (in
a more or less general manner) with the help of the driver managed pool of
worker
threads is absolutely legitimate. One should be careful with such things,
certainly.

Vladimir

----- Original Message -----
From: “Nick Ryan”
To: “File Systems Developers”
Sent: Monday, June 23, 2003 3:41 AM
Subject: [ntfsd] RE: Problem with NAI filter V7.0

> Yes, there is blocking of the lazy writer thread in the case that
> originally spawned this discussion. What was being discussed was the
> technique of blocking the completion processing of IRP_MJ_CLOSE (for
> some undefined purpose). I mentioned that one should be careful what one
> does blocking file closes because the lazy writer thread generates and
> blocks on file closes itself. Cc uses the lazy writer thread to cleanup
> its shared cache maps when their reference counts go to zero, and in the
> process of doing this Cc invokes ObDereferenceObject on its captive file
> objects, which generates a close internally.
>
> - Nick Ryan
>
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir Strogov
> > Sent: Sunday, June 22, 2003 1:35 PM
> > To: File Systems Developers
> > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> >
> >
> >
> > Yes, certainly it is true (deadlocks are possible when
> > blocking the lazy writer thread and performing cached IO),
> > but the discussed case was mainly about devising schemes of
> > post-processing requests at the PASSIVE_LEVEL instead of the
> > elevated IRQL (there is no blocking of the lazy writer
> > thread, for example).
> >
> > Vladimir
> >
> > ----- Original Message -----
> > From: “Nick Ryan”
> > To: “File Systems Developers”
> > Sent: Sunday, June 22, 2003 11:37 PM
> > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> >
> >
> > > Although it’s true 2k&later can spawn additional worker
> > threads, there
> > > are some worker threads that are ‘special’ under Windows
> > and cannot be
> > > cloned, such as the lazy writer thread. Blocking the lazy writer
> > > thread and performing cached I/O on some other file, for
> > example, may
> > > deadlock because the I/O may in turn cause Cc to wait for the lazy
> > > writer to free up pages.
> > >
> > > - Nick Ryan
> > >
> > > > -----Original Message-----
> > > > From: xxxxx@lists.osr.com
> > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Vladimir
> > > > Strogov
> > > > Sent: Sunday, June 22, 2003 8:26 AM
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > >
> > > >
> > > > Alexey
> > > >
> > > > A system worker thread item was a limited resource under
> > versions of
> > > > NT, prior to Windows 2000. You could introduce deadlocks (exposed
> > > > under stress), using these items in file system filters.
> > So if you
> > > > are interested in having a more or less common source
> > code base for
> > > > file system filters for different NT versions, it is
> > better to use
> > > > your own worker threads pool (if you need such mechanisms).
> > > >
> > > > Windows 2000 and higher can dynamically increase available worker
> > > > thread items, but I have not tried using them for such purposes.
> > > >
> > > > Vladimir
> > > >
> > > >
> > > > ----- Original Message -----
> > > > From: “Alexey Logachyov”
> > > > To: “File Systems Developers”
> > > > Sent: Sunday, June 22, 2003 3:04 PM
> > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > >
> > > >
> > > > > Is driver managed pool of worker threads a must? Why it is not
> > > > > possible to use usual system managed worker threads?
> > > > >
> > > > > -htfv
> > > > >
> > > > >
> > > > >
> > > > > ----- Original Message -----
> > > > > From: “Vladimir Strogov”
> > > > > To: “File Systems Developers”
> > > > > Sent: Thursday, June 19, 2003 10:36 PM
> > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > >
> > > > >
> > > > > > Nick
> > > > > >
> > > > > > Certainly a driver managed pool of worker threads is used for
> > > > > > this
> > > > scheme,
> > > > > > otherwise it would not be able to work reliably(especially on
> > > > > > pre- Windows 2000 systems ).
> > > > > >
> > > > > > Vladimir
> > > > > >
> > > > > > ----- Original Message -----
> > > > > > From: “Nick Ryan”
> > > > > > To: “File Systems Developers”
> > > > > > Sent: Thursday, June 19, 2003 10:52 PM
> > > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > > >
> > > > > >
> > > > > > > Hmm… I still think that pending close requests
> > like this is
> > > > > > > a bad
> > > > idea
> > > > > > > if the completion is being delayed to a system
> > worker thread,
> > > > > > > since system worker threads themselves generate and wait on
> > > > > > > closes. It
> > > > should
> > > > > > > be better if you have a thread of your own
> > dedicated to this
> > > > > > > purpose (assuming of course that you don’t block
> > this thread
> > > > > > > on additional closes).
> > > > > > >
> > > > > > > - Nick Ryan
> > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: xxxxx@lists.osr.com
> > > > > > > > [mailto:xxxxx@lists.osr.com] On Behalf Of Ted
> > > > > > > > Hess
> > > > > > > > Sent: Thursday, June 19, 2003 8:41 AM
> > > > > > > > To: File Systems Developers
> > > > > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > > > > >
> > > > > > > >
> > > > > > > > FWIW Nick - the NAI filters lock up my system when
> > > > loaded with
> > > > > > > > the OSR NULL filter driver set. This filter set
> > > > doesn’t do much
> > > > > > > > more than set a completion routine, mark the IRP pending,
> > > > > > > > dispatch it and return STATUS_PENDING. In the completion
> > > > > > > > routine, if we not running at PASSIVE_LEVEL, the
> > > > > > > > IoCompleteRequest will be called from a worker
> > thread. All
> > > > > > > > perfectly legit…
> > > > > > > >
> > > > > > > > Jesus had nothing to do with it!
> > > > > > > >
> > > > > > > > /ted
> > > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Nick Ryan [mailto:xxxxx@nryan.com]
> > > > > > > > Sent: Thursday, June 19, 2003 2:18 AM
> > > > > > > > To: File Systems Developers
> > > > > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > > > > >
> > > > > > > >
> > > > > > > > Jesus.
> > > > > > > >
> > > > > > > > In my opinion (and I believed others have
> > expressed the same
> > > > > > > > views on this
> > > > > > > > list) it’s a bad idea to delay, pend, alter, or
> > otherwise do
> > > > > > > > anything complex to or during an IRP_MJ_CLOSE.
> > > > Cleanup is a much
> > > > > > > > safer time for playing around since cleanups
> > > > generally occur in
> > > > > > > > some nonarbitrary context as a direct result of a
> > > > user-invoked
> > > > > > > > ZwClose. In contrast, closes can be thrown down from
> > > > many places
> > > > > > > > as a result of somebody (usually Cc or Mm)
> > > > dereferencing a file
> > > > > > > > object, including from within the lazy-writer thread.
> > > > I’ve found
> > > > > > > > it extremely easy to deadlock the system doing
> > > > anything complex
> > > > > > > > while holding a close.
> > > > > > > >
> > > > > > > > - Nick Ryan
> > > > > > > >
> > > > > > > > > -----Original Message-----
> > > > > > > > > From: xxxxx@lists.osr.com
> > > > > > > > > [mailto:xxxxx@lists.osr.com] On
> > Behalf Of Tony
> > > > > > > > > Mason
> > > > > > > > > Sent: Wednesday, June 18, 2003 2:32 PM
> > > > > > > > > To: File Systems Developers
> > > > > > > > > Subject: [ntfsd] RE: Problem with NAI filter V7.0
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Ted,
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > We talked to someone else recently because they were
> > > > > > > > > seeing
> > > > > > > > the same
> > > > > > > > > problem. They had an excellent analysis of the crash,
> > > > > > > > > that
> > > > > > > > indicated
> > > > > > > > > NAI was making substantial modifications to the
> > close IRP.
> > > > > > > > > We’ve subsequently been advised that this is
> > > > resolved in a hot
> > > > > > > > > fix for Version 7.0, although I have no independent
> > > > > > > > > confirmation of this.
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Tony
> > > > > > > > >
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Tony Mason
> > > > > > > > >
> > > > > > > > > Consulting Partner
> > > > > > > > >
> > > > > > > > > OSR Open Systems Resources, Inc.
> > > > > > > > >
> > > > > > > > http://www.osr.com
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > -----Original Message-----
> > > > > > > > From: Ted Hess [mailto:xxxxx@livevault.com]
> > > > > > > > Sent: Wednesday, June 18, 2003 4:55 PM
> > > > > > > > To: File Systems Developers
> > > > > > > > Subject: [ntfsd] Problem with NAI filter V7.0
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > Has anyone else out there had inter-op problems with
> > > > the latest
> > > > > > > > McAfee / NAI filter?
> > > > > > > >
> > > > > > > > Their filter (naiavf5x.sys V7.0.0.230) crashes
> > > > whenever I call
> > > > > > > > IoCompleteRequest on a CLOSE IRP from a worker thread
> > > > and they
> > > > > > > > are above me on the stack! The reason my completion
> > > > is running
> > > > > > > > on a worker at this point is due to the fact that my
> > > > completion
> > > > > > > > was called at elevated (above PASSIVE) IRQL.
> > > > > > > >
> > > > > > > > This was never a problem with naifiltr.sys (v6).
> > > > > > > >
> > > > > > > > If anyone from NAI is reading this and would like a
> > > > > > > > crash-dump…
> > > > > > > >
> > > > > > > > /ted
> > > > > > > >
> > > > > > > > —
> > > > > > > > You are currently subscribed to ntfsd as:
> > xxxxx@osr.com To
> > > > > > > > unsubscribe send a blank email to
> > > > > > > > xxxxx@lists.osr.com
> > > > > > > >
> > > > > > > >
> > > > > > > > —
> > > > > > > > You are currently subscribed to ntfsd as:
> > xxxxx@nryan.com To
> > > > > > > > unsubscribe send a blank email to
> > > > > > > > xxxxx@lists.osr.com
> > > > > > > >
> > > > > > > >
> > > > > > > >
> > > > > > > > —
> > > > > > > > You are currently subscribed to ntfsd as:
> > > > xxxxx@livevault.com To
> > > > > > > > unsubscribe send a blank email to
> > > > > > > > xxxxx@lists.osr.com
> > > > > > > >
> > > > > > > >
> > > > > > > > —
> > > > > > > > You are currently subscribed to ntfsd as:
> > xxxxx@nryan.com To
> > > > > > > > unsubscribe send a blank email to
> > > > > > > > xxxxx@lists.osr.com
> > > > > > > >
> > > > > > >
> > > > > > >
> > > > > > >
> > > > > > > —
> > > > > > > You are currently subscribed to ntfsd as:
> > > > > > > xxxxx@umail.ru To unsubscribe send a
> > blank email to
> > > > > > > xxxxx@lists.osr.com
> > > > > >
> > > > > >
> > > > > >
> > > > > > —
> > > > > > You are currently subscribed to ntfsd as: xxxxx@vba.com.by To
> > > > > > unsubscribe send a blank email to
> > > > > > xxxxx@lists.osr.com
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as:
> > > > xxxxx@umail.ru To
> > > > > unsubscribe send a blank email to
> > xxxxx@lists.osr.com
> > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@nryan.com To
> > > > unsubscribe send a blank email to xxxxx@lists.osr.com
> > > >
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as:
> > xxxxx@umail.ru To
> > > unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@nryan.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@umail.ru
> To unsubscribe send a blank email to xxxxx@lists.osr.com

> Although it’s true 2k&later can spawn additional worker threads,
there

are some worker threads that are ‘special’ under Windows and cannot
be
cloned, such as the lazy writer thread. Blocking the lazy writer
thread
and performing cached I/O on some other file, for example, may
deadlock

There is no “lazy writer thread”, Cc uses ExQueueWorkItem for a lazy
writer.
So, using ExQueueWorkItem from below the filesystem (in another
ExQueueWorkItem callback) can possibly cause a deadlock inside the
work item thread pool implementation. So, write your own pool.

Max