Deadlock with Other filter driver softwares like AV software

OSR_Community_User · August 3, 2005, 8:42am

Hi All,

We have a filter driver which gets deadlocked with AV software’s like
Symantec. The sequence of steps that cause dead lock are given below:

1… CREATE (IRP_MJ_CREATE) request of the other application reaches AV
driver. (note that Our filter driver does not block CREATE requests)
2… AV driver blocks the other applications CRATE request and initiates
file scan. The AV scan is happening in the context of other application
process context.
3… AV drivers READ (IRP_MJ_READ) request reaches our filter driver (by
this time the AV CREATE request is success which has been passed through our
filter driver as our filter driver is not blocking create request)
4… Our filter driver will block the READ request of AV driver and informs
our user mode application about the file being accessed.
5… Now our user mode application tries to read the file as part of that
it generates CREATE request (IRP_MJ_CREATE) (which will pass through our
filter driver as this request is from our application) and this request is
seen by the AV AV driver and as the scan on that file is not finished, (AV
drivers READ request is blocked by our filter driver for validating the
file) AV is holding this request, which leads to dead lock.

Could any one guide me in finding solution to this. I guess this problem
should have been seen by many.

Help is highly appreciated.

Thanks,

Kedar.

OSR_Community_User · August 3, 2005, 9:07am

Kedar,

You have posted this problem already some time ago.
Again, I can tell you that the problem is in blocking
read requests. You cannot do this. While it will work
with pure OS (read: no FS filters) it will most
likely deadlock with any filter above yours.

I’m afraid you must redesign your filter.

L.

OSR_Community_User · August 3, 2005, 9:29am

Hi Ladislav Zezula,

But how does blocking in CREATE request help in this case.
Even in CREATE the deadlock happens right?

Am I wrong.

Thanks,
Kedar.

“Ladislav Zezula” wrote in message news:xxxxx@ntfsd…
> Kedar,
>
> You have posted this problem already some time ago.
> Again, I can tell you that the problem is in blocking
> read requests. You cannot do this. While it will work
> with pure OS (read: no FS filters) it will most
> likely deadlock with any filter above yours.
>
> I’m afraid you must redesign your filter.
>
> L.
>
>

OSR_Community_User · August 3, 2005, 9:49am

AV software will probably not hold a lock in the create path,
but surely will do it in the read path.
(maybe I’m wrong, but yet I haven’t encounter such case).

You must only ensure that your usermode handler will
not open the file while it’s already processed by the driver.

L.

OSR_Community_User · August 3, 2005, 10:12am

Hi Ladislav Zezula,

I do not understand what you mean by
“your usermode handler will
not open the file while it’s already processed by the driver”

When our driver block a create request and communicates to application about
the file access our user mode application has to open the file to validate.

Do you say that the above approach is wrong.

Thanks,
Kedar.

“Ladislav Zezula” wrote in message news:xxxxx@ntfsd…
> AV software will probably not hold a lock in the create path,
> but surely will do it in the read path.
> (maybe I’m wrong, but yet I haven’t encounter such case).
>
> You must only ensure that your usermode handler will
> not open the file while it’s already processed by the driver.
>
> L.
>

OSR_Community_User · August 3, 2005, 10:40am

> When our driver block a create request and communicates to application

about the file access our user mode application has to open the file to
validate.

If you do this, the follwing scenario is to happen:

Someone opens file C:\File.ext
Your filter catches the file open (doesn’t it ?)
Your filter notifies the user mode about a file being open
Your user mode opens the file (as you say, it needs to do so)
Your filter catches the file open.

You can see that the point 5 is the same situation like
point 2. In point 5, your filter is about to notify user mode,
the user mode is about to check the file etc.

Do you say that the above approach is wrong.

You must prevent the endless loop shown
in the above scenario.

L.

Slava_Imameyev · August 3, 2005, 10:49am

Read the file in step 4 in your filter driver using FileObject contained in
request and send buffer( or sequence of buffers ) to user mode application.
I think this will help you.

“kedar” wrote in message news:xxxxx@ntfsd…
> Hi All,
>
>
>
>
> We have a filter driver which gets deadlocked with AV software’s like
> Symantec. The sequence of steps that cause dead lock are given below:
>
>
>
>
>
> 1… CREATE (IRP_MJ_CREATE) request of the other application reaches AV
> driver. (note that Our filter driver does not block CREATE requests)
> 2… AV driver blocks the other applications CRATE request and initiates
> file scan. The AV scan is happening in the context of other application
> process context.
> 3… AV drivers READ (IRP_MJ_READ) request reaches our filter driver (by
> this time the AV CREATE request is success which has been passed through
> our filter driver as our filter driver is not blocking create request)
> 4… Our filter driver will block the READ request of AV driver and
> informs our user mode application about the file being accessed.
> 5… Now our user mode application tries to read the file as part of that
> it generates CREATE request (IRP_MJ_CREATE) (which will pass through our
> filter driver as this request is from our application) and this request is
> seen by the AV AV driver and as the scan on that file is not finished, (AV
> drivers READ request is blocked by our filter driver for validating the
> file) AV is holding this request, which leads to dead lock.
>
>
> Could any one guide me in finding solution to this. I guess this problem
> should have been seen by many.
>
>
>
> Help is highly appreciated.
>
>
>
> Thanks,
>
> Kedar.
>
>
>

Slava_Imameyev · August 3, 2005, 10:54am

to avoid infinite loop it is sufficient to get current process using
PsGetCurrentProcess() and compare it with the ‘worker’ user mode application
used for communication with the driver

“Ladislav Zezula” wrote in message news:xxxxx@ntfsd…
>> When our driver block a create request and communicates to application
>> about the file access our user mode application has to open the file to
>> validate.
>
> If you do this, the follwing scenario is to happen:
>
> 1) Someone opens file C:\File.ext
> 2) Your filter catches the file open (doesn’t it ?)
> 3) Your filter notifies the user mode about a file being open
> 4) Your user mode opens the file (as you say, it needs to do so)
> 5) Your filter catches the file open.
>
> You can see that the point 5 is the same situation like
> point 2. In point 5, your filter is about to notify user mode,
> the user mode is about to check the file etc.
>
>> Do you say that the above approach is wrong.
>
> You must prevent the endless loop shown
> in the above scenario.
>
> L.
>
>

Slava_Imameyev · August 3, 2005, 10:58am

You can use IoBuildAsynchronousFsdRequest ( IRP_MJ_READ… ) or
IoBuildSynchronousFsdRequest( IRP_MJ_READ… )

“Slava Imameyev” wrote in message news:xxxxx@ntfsd…
> Read the file in step 4 in your filter driver using FileObject contained
> in request and send buffer( or sequence of buffers ) to user mode
> application. I think this will help you.
>
> “kedar” wrote in message news:xxxxx@ntfsd…
>> Hi All,
>>
>>
>>
>>
>> We have a filter driver which gets deadlocked with AV software’s like
>> Symantec. The sequence of steps that cause dead lock are given below:
>>
>>
>>
>>
>>
>> 1… CREATE (IRP_MJ_CREATE) request of the other application reaches AV
>> driver. (note that Our filter driver does not block CREATE requests)
>> 2… AV driver blocks the other applications CRATE request and initiates
>> file scan. The AV scan is happening in the context of other application
>> process context.
>> 3… AV drivers READ (IRP_MJ_READ) request reaches our filter driver (by
>> this time the AV CREATE request is success which has been passed through
>> our filter driver as our filter driver is not blocking create request)
>> 4… Our filter driver will block the READ request of AV driver and
>> informs our user mode application about the file being accessed.
>> 5… Now our user mode application tries to read the file as part of that
>> it generates CREATE request (IRP_MJ_CREATE) (which will pass through our
>> filter driver as this request is from our application) and this request
>> is seen by the AV AV driver and as the scan on that file is not finished,
>> (AV drivers READ request is blocked by our filter driver for validating
>> the file) AV is holding this request, which leads to dead lock.
>>
>>
>> Could any one guide me in finding solution to this. I guess this problem
>> should have been seen by many.
>>
>>
>>
>> Help is highly appreciated.
>>
>>
>>
>> Thanks,
>>
>> Kedar.
>>
>>
>>
>
>
>

OSR_Community_User · August 3, 2005, 11:50am

Hi Ladislav Zezula,

We identify the requests that come from our request and our filter will not
block them and act as passthrough for these requests.

If this is the case you say that there won’t be any problem?

Thanks,
Kedar.

“Ladislav Zezula” wrote in message news:xxxxx@ntfsd…
>> When our driver block a create request and communicates to application
>> about the file access our user mode application has to open the file to
>> validate.
>
> If you do this, the follwing scenario is to happen:
>
> 1) Someone opens file C:\File.ext
> 2) Your filter catches the file open (doesn’t it ?)
> 3) Your filter notifies the user mode about a file being open
> 4) Your user mode opens the file (as you say, it needs to do so)
> 5) Your filter catches the file open.
>
> You can see that the point 5 is the same situation like
> point 2. In point 5, your filter is about to notify user mode,
> the user mode is about to check the file etc.
>
>> Do you say that the above approach is wrong.
>
> You must prevent the endless loop shown
> in the above scenario.
>
> L.
>
>

OSR_Community_User · August 3, 2005, 2:30pm

> We identify the requests that come from our request and our filter will

not block them and act as passthrough for these requests.

Maybe yes. Try and you will see.

L.

Lyndon_J_Clarke-2 · August 3, 2005, 3:19pm

>> We identify the requests that come from our request and our filter will

> not block them and act as passthrough for these requests.

Perhaps and yet perhaps not. Consider for a moment interoperation with
another product written like your product …

Someone opens file C:\File.ext
Filter A catches the file open
Filter A notifies its user mode A about a file being opened
User mode A opens the file
Filter B catches the file open
Filter B notifies user mode B about a file being opened
User mode B opens the file
Filter A catches the file open
Filter A notifies its user mode A about a file being opened
User mode A opens the file
Filter B catches the file open
Filter B notifies user mode B about a file being opened
User mode B opens the file
Filter A catches the open
…

Cheers
Lyndon

OSR_Community_User · August 4, 2005, 11:00am

Hi Ladislav Zezula,

(maybe I’m wrong, but yet I haven’t encounter such case).
Does this mean you never had situation where your driver waits for the
response from the application.

Thanks,
Kedar.

“Ladislav Zezula” wrote in message news:xxxxx@ntfsd…
> AV software will probably not hold a lock in the create path,
> but surely will do it in the read path.
> (maybe I’m wrong, but yet I haven’t encounter such case).
>
> You must only ensure that your usermode handler will
> not open the file while it’s already processed by the driver.
>
> L.
>

OSR_Community_User · August 4, 2005, 11:49am

I did that a long time ago and ever since then I’ve warned people about
the dangers of blocking on ANY operations from a Win32 application.
Down that road lies deadlock…

In addition, when I talk about re-entrancy detection I always suggest
that the person implementing ask themselves “what happens if a filter
above and/or below me does the same thing. Will it work?” If the
answer is “no” then the technique is flawed.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of kedar
Sent: Thursday, August 04, 2005 11:00 AM
To: ntfsd redirect
Subject: Re:[ntfsd] Re:Deadlock with Other filter driver softwares like
AV software

Hi Ladislav Zezula,

(maybe I’m wrong, but yet I haven’t encounter such case).
Does this mean you never had situation where your driver waits for the
response from the application.

Thanks,
Kedar.

“Ladislav Zezula” wrote in message news:xxxxx@ntfsd…
> AV software will probably not hold a lock in the create path,
> but surely will do it in the read path.
> (maybe I’m wrong, but yet I haven’t encounter such case).
>
> You must only ensure that your usermode handler will
> not open the file while it’s already processed by the driver.
>
> L.
>

—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · August 4, 2005, 1:07pm

> Does this mean you never had situation where your driver waits for the

response from the application.

We have a user mode service that
may receive the notification from the driver.
Everytime, when the driver is about to cal user mode,
it tries to acquire a resource protecting the user mode call.
If it fails to acquire the resource, we check the process
originating the IRP and if it is the user mode service, we complete
the create request with STATUS_ACCESS_DENIED.

We don’t hold any locks during passing the CREATE request
to the lower driver.

L.

OSR_Community_User · August 5, 2005, 12:46am

Hi Ladislav Zezula,

Were you calling the user mode function from the kernel driver?

We too do not hold any locks while passing the create request to lower
driver.

Thanks,
Kedar.

“Ladislav Zezula” wrote in message news:xxxxx@ntfsd…
>> Does this mean you never had situation where your driver waits for the
>> response from the application.
>
> We have a user mode service that
> may receive the notification from the driver. Everytime, when the driver
> is about to cal user mode,
> it tries to acquire a resource protecting the user mode call.
> If it fails to acquire the resource, we check the process
> originating the IRP and if it is the user mode service, we complete
> the create request with STATUS_ACCESS_DENIED.
>
> We don’t hold any locks during passing the CREATE request
> to the lower driver.
>
> L.
>
>

OSR_Community_User · August 5, 2005, 12:53am

> Were you calling the user mode function from the kernel driver?

Yes.

L.

OSR_Community_User · August 5, 2005, 12:56am

> Were you calling the user mode function from the kernel driver?

Yes. Of course not directly.

L.

OSR_Community_User · August 8, 2005, 11:56am

Hi Ladislav Zezula,

I am not able to understand. Could you please explain me how you were
calling User mode function and how it was not causing the deadlock.

Thanks,
Kedar.

“Ladislav Zezula” wrote in message news:xxxxx@ntfsd…
>> Were you calling the user mode function from the kernel driver?
>
> Yes. Of course not directly.
>
> L.
>
>

OSR_Community_User · August 13, 2005, 11:15am

> I am not able to understand. Could you please explain me how

you were calling User mode function and how it was not causing
the deadlock.

Sorry fot the late answer, I was out of country for a week.

By “Calling the user mode”, I mean to send a LPC message from the
driver to the user mode application. To prevent deadlock, the
driver will not send the message if the IRP_MJ_CREATE calling
process ID is out user mode service.

L.