RE: Tony,please.Question about Reference Counting art icle in NT Insider?

NTFSD
1

Alexi,

Indeed, were someone to implement their filter in the fashion you described,
they would cause problems - but that is because they are improperly
implemented.

Imagine we use your scenario, but that second IRP is for a query information
call. In that case, the I/O Manager allocates the file object on the stack.
Since this is the FIRST file object to be used for I/O, it backs the section
object. That leads to horrible problems because it is not a REAL file
object, but has now been used to back the section object.

So, I argue that your model is defective on the surface because it wouldn’t
work without reference counting. Thus, I don’t think we need to make an
already defective model work.

You hit this on the head - use stream file objects for such I/O.

Welcome to the truly hideous world of filter-to-filter interactions. To
think of it another way - JOB SECURITY!

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Alexey Logachyov [mailto:xxxxx@vba.com.by]
Sent: Friday, August 29, 2003 6:02 PM
To: File Systems Developers
Subject: [ntfsd] RE: Tony,please.Question about Reference Counting article
in NT Insider?

This is not what I was originally going to write, but while writing I’ve got
an idea of the following scenario:

  1. A file is opened: file object created, IRP_MJ_CREATE IRP sent, completed,
    reference count incremented and equals to 1. As caching is not yet
    initialized both DataSection and ImageSection pointers are NULL.

  2. The same on-disk stream is opened: a new file object is created and
    IRP_MJ_CREATE IRP is sent. As file is not yet opened, we cannot increment
    reference counter. So we pass the request down.

  3. Lower filter uses the file object in the IRP to first query some
    information. So it saves IRP state, passes request to FSD, catches it
    completion, and uses opened file to do whatever processing it wants. After
    that it closes file which causes IRP_MJ_CLOSE IRP be sent to the top of the
    stack (I believe this is the scenario OP described).

  4. During IRP_MJ_CLOSE processing we decrement reference counter and
    effectively drop our context from the table which is bad as we still have
    file stream opened.

  5. Lower filter restores IRP and passes it down. FSD opens the file and
    completes IRP. In IRP_MJ_CREATE IRP completion routine we create a new
    context for open file as we don’t have a context for this FsContext in the
    table.

  6. One of the object is closed: reference counter is decremented and
    appropriate context is removed from our table (remember that DataSection and
    ImageSection are still NULL).

  7. Now, the on-disk stream can be modified unseen through the file object
    left opened.

I hope I described everything clear enough. So, seems like there a desigh
flaw here. The lower filter should either use streaming file objects for
accessing file before it is fully opened or send request to lower device
object only. On the other hand, reference counting algo described in NT
Insider assumes that underlying FSFDs/FSD use only stream file objects for
accessing data internally.

One solution may be to have a list of *all* file objects (not only stream
file objects like in the article) for which we have seen I/Os, and decrement
reference counter only when file object is found in this list. In our
scenario we would have one of the following behaviour. In step 3, if any
request is sent to the top of driver stack, we will increment reference
count of our context record because the file object was not seen yet. If we
did not see any I/O, we will not dereference our context record in step 4 as
the file object is not in list.

However, there’s no cure against bad programmer. Lower filter driver may
send requests to the top of driver stack (which will cause our context
record to be referenced), but send IRP_MJ_CLOSE immediately to the
underlying FSD. That’s queer, I know, but possible. In that case we will end
up with extra reference floating around which is definitely no good.

Well, seems like the more I think about all this stuff the more
possibilities and (what’s worse) problems arise. Enough for today, I leave
this for community to chew over.

-htfv

----- Original Message -----
From: “Tony Mason”
To: “File Systems Developers”
Sent: Friday, August 29, 2003 7:18 PM
Subject: [ntfsd] RE: Tony,please.Question about Reference Counting article
in NT Insider?

> I must admit, I am not sure I understand the scenario to which you
> refer, although I suppose it is possible for an IRP_MJ_CLOSE to arrive
> prior to
the
> completion of an IRP_MJ_CREATE (atypical, but certainly allowed). In
> that case when you do the lookup (in IRP_MJ_CLOSE) for that FsContext
> value,
you
> will not find an entry and will skip it (not tracked so there is no
> change in state). When the IRP_MJ_CREATE completes, you would then
> perform a lookup, the entry would not be found so you would create the
> new entry.
>
> So, is there an issue you have seen here, or are you merely trying to
> improve your understanding?
>
> Regards,
>
> Tony
>
> Tony Mason
> Consulting Partner
> OSR Open Systems Resources, Inc.
> http://www.osr.com
>
> Hope to see you at the next OSR file systems class in San Jose, CA
September
> 16, 2002!
>
> -----Original Message-----
> From: ecore [mailto:xxxxx@hotmail.com]
> Sent: Friday, August 29, 2003 11:40 AM
> To: File Systems Developers
> Subject: [ntfsd] Tony,please.Question about Reference Counting article
> in
NT
> Insider?
>
> At first,excuse me for ask tony in this direct way.:slight_smile:
>
> According to description in Reference Counting article in
> NT Insider,I have below question:
> When does IRP_MJ_CREATE increase the reference count?In
> it’s Completion Routing?
> We know,an IRP_MJ_CREATE will lead to an IRP_MJ_CLOSE(not
> the same fileobject,but the same FCB)recursively,then
> the IRP_MJ_CLOSE will occur before the completion of the
> IRP_MJ_CREATE. The reference count will decrease at first,it perhaps
> will drop to zero.
> At this Scenario,Can I sure the
> (DataSection!=NULL&&ImageSection!=NULL)?
> If the DataSection==NULL&&ImageSection==NULL,oh,Last
> Close.I delete the context of per-file,Wrong!
> anyone can explain it?maybe at this scenario,I can sure
> the two section will not be NULL?
> thanks you,especially for Tony.
>
> —
> You are currently subscribed to ntfsd as: xxxxx@osr.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
> —
> You are currently subscribed to ntfsd as: xxxxx@vba.com.by
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>

—
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

2

Oh,God!That complex and hideous scenario is not the Scenario I refer.
My description is,for these file objects with the same FCB,there is the
scenario:
1.File Object A IRP_MJ_CREATE Completed,so I increase an reference
count;So,the count=1;
2.File Object B IRP_MJ_CREATE reached,not completed,when I pass it to the
FSD,will it
lead to the IRP_MJ_CLOSE for File Object A?
I think it will.Right???
3.So,in File Object A IRP_MJ_CLOSE,I decrease an reference count.So,the
Count=0;
I delete the Entry from my record table.

OK,in File Object B IRP_MJ_CREATE Completion,I will find no entry for the
FCB,so I create
a new entry for the FCB.BUT,my entry isnot so simple,it needs many
infomation;So,I can’t delete
the entry at this clock:
File Object B IRP_MJ_CREATE reached(not completed),I find the entry for
fileobject B’s FCB.
But,at the Completion routine(for File Object B IRP_MJ_CREATE),the entry
have already deleted.

Oh,if 2 is right.I have to guarantee the entry will not be deleted by
other ways.