Re: STATUS_PENDING in IRP_MJ_CREATE

Dear all:

A long time ago, Wed, 22 Aug 2001, Dejan Maksimovic posted an eMail (and
this is a follow-up to that message). No others responded as far as I can
tell.

Dejan Maksimovic wrote:


Although I’ve NEVER encountered a STATUS_PENDING return value in
IRP_MJ_CREATE, as I see it’s possible.


Unlike Dejan Maksimovic, I *have* seen a STATUS_PENDING return value in
IRP_MJ_CREATE.

As a newbie, I have two questions:

a) How is this possible? I thought IRP_MJ_CREATE were synchronous.

b) How can I tell what is causing this?

Ralph Shnelvar

On Wed, 22 Aug 2001 22:53:52 +0200, you wrote:

Hi,

Although I’ve NEVER encountered a STATUS_PENDING return value in
IRP_MJ_CREATE, as I see it’s possible.
Now, this poses a small problem.
For local drives, Irp->IoStatus.Information will contain
FILE_OPENED, FILE_CREATED etc. the information about what was actually
done, so if the CreateDisposition was FILE_OPEN_IF, I can know what
really happened.
On network drives, and network paths, this is not true, however,
and this field is always zeroed!
So, the only way to handle this was to split the Irp - first
send a changed Irp with CreateDisposition set to FILE_OPEN and then
FILE_CREATE, if FILE_OPEN failed. To do this, I returned
STATUS_MORE_PROCESSING_REQUIRED in completion routine, if the
Irp->IoStatus.Status says the operation failed, and resend the Irp in
the dispatch routine, with FILE_CREATE.
However, in case when STATUS_PENDING is returned, this won’t
work, and what’s worse the system would lockup:-)
Now, has anyone been down this road before, and can tell me the
right way to handle this?
My idea would be to:

  • Try FILE_OPEN
  • In dispatch return the status IoCallDriver returned.
  • In completion, if the Irp->IoStatus.Status is successful, just
    return it
  • If not, then setup a work item, and return
    STATUS_MORE_PROCESSING_REQUIRED
  • In the work routine, resend the Irp with FILE_CREATE this
    time.

Does anyone see any pitfalls in this, or is there anything else
I should worry about?


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

The I/O model in Windows NT (and 2000, and XP) is *asynchronous*. Callers
may request anything they want. If they call via the I/O Manager, they will
get what they asked for. If they bypass the I/O Manager and call an FSD
directly, they get whatever the FSD provides.

I can envision designs where an FSD might choose to post EVERY I/O
operation, regardless of its type. That, by itself, is enough for it to
return STATUS_PENDING.

Or suppose I’m in a filter driver. I get a call back in my completion
routine. I code logic that posts the IRP if I am called back at elevated
IRQL. In my dispatch logic I always return STATUS_PENDING (because I have
to either make the I/O synchronous or return STATUS_PENDING in my dispatch
entry point if I might post the IRP. Otherwise I run into horrible
problems.) This isn’t the ONLY architecture, but it is VALID.

There are certain things everyone should keep in mind:

* Drivers are free to implement ANY I/O operation asynchronously (there are
an odd set where posting it acts as a true success code, mind you.) It
doesn’t matter if the caller sets the DO_THIS_SYNCHRONOUSLY_OR_ELSE bit (or
whatever bit you believe means "do this synchronously because that’s what
*I* want you to do.) Callers must be prepared to handle this - after all,
the I/O Manager does!

* ANY Completion Routine may be called at DISPATCH_LEVEL.

I suppose it would be nice if we could codify all of these inherent
assumptions (I found another one recently that people didn’t know - Fast I/O
is an OPTIONAL interface. You cannot implement something ONLY via the fast
I/O path - you have to implement it via the IRP path and then the fast I/O
path.)

Ah, too much to do and too little time to do it. Off the soapbox now (it’s
been one of those days…)

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: Ralph Shnelvar [mailto:xxxxx@dos32.com]
Sent: Wednesday, November 28, 2001 6:27 PM
To: File Systems Developers
Subject: [ntfsd] Re: STATUS_PENDING in IRP_MJ_CREATE

Dear all:

A long time ago, Wed, 22 Aug 2001, Dejan Maksimovic posted an eMail (and
this is a follow-up to that message). No others responded as far as I can
tell.

Dejan Maksimovic wrote:


Although I’ve NEVER encountered a STATUS_PENDING return value in
IRP_MJ_CREATE, as I see it’s possible.


Unlike Dejan Maksimovic, I *have* seen a STATUS_PENDING return value in
IRP_MJ_CREATE.

As a newbie, I have two questions:

a) How is this possible? I thought IRP_MJ_CREATE were synchronous.

b) How can I tell what is causing this?

Ralph Shnelvar

On Wed, 22 Aug 2001 22:53:52 +0200, you wrote:

Hi,

Although I’ve NEVER encountered a STATUS_PENDING return value in
IRP_MJ_CREATE, as I see it’s possible.
Now, this poses a small problem.
For local drives, Irp->IoStatus.Information will contain
FILE_OPENED, FILE_CREATED etc. the information about what was actually
done, so if the CreateDisposition was FILE_OPEN_IF, I can know what
really happened.
On network drives, and network paths, this is not true, however,
and this field is always zeroed!
So, the only way to handle this was to split the Irp - first
send a changed Irp with CreateDisposition set to FILE_OPEN and then
FILE_CREATE, if FILE_OPEN failed. To do this, I returned
STATUS_MORE_PROCESSING_REQUIRED in completion routine, if the
Irp->IoStatus.Status says the operation failed, and resend the Irp in
the dispatch routine, with FILE_CREATE.
However, in case when STATUS_PENDING is returned, this won’t
work, and what’s worse the system would lockup:-)
Now, has anyone been down this road before, and can tell me the
right way to handle this?
My idea would be to:

  • Try FILE_OPEN
  • In dispatch return the status IoCallDriver returned.
  • In completion, if the Irp->IoStatus.Status is successful, just
    return it
  • If not, then setup a work item, and return
    STATUS_MORE_PROCESSING_REQUIRED
  • In the work routine, resend the Irp with FILE_CREATE this
    time.

Does anyone see any pitfalls in this, or is there anything else
I should worry about?


You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Dear Tony:

On Wed, 28 Nov 2001 22:43:39 -0500, you wrote:

The I/O model in Windows NT (and 2000, and XP) is *asynchronous*. Callers
may request anything they want. If they call via the I/O Manager, they will
get what they asked for. If they bypass the I/O Manager and call an FSD
directly, they get whatever the FSD provides.

Three more newbie questions:

a)
I am still confused. There are places in various documentation that says
“IRP_MJ_CREATE is inherently synchronous.”

What does that mean, if anything?

b)
Is there any way to determine what component of the OS marked the IRP
pending?

c)
Even a more basic question. I see that a FSFD can, uh, mark an IRP pending
by doing a IoMarkIrpPending(irp). Is this the only way to mark an IRP
pending? Does this routine do anything more than setting
IRP->PendingReturned ?

[snip]

Ah, too much to do and too little time to do it. Off the soapbox now (it’s
been one of those days…)

Gee, every day is one of those days for me.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Ralph Shnelvar


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

> >The I/O model in Windows NT (and 2000, and XP) is *asynchronous*. Callers

>may request anything they want. If they call via the I/O Manager, they will
>get what they asked for. If they bypass the I/O Manager and call an FSD
>directly, they get whatever the FSD provides.

FAT does return STATUS_PENDING is some cases, though I haven’t been
able to
see it in real world.
I’ll make a small filter that does nothing but pend every IRP, so if
anyone
wants to test their drivers against STATUS_PENDING, let me know.

I am still confused. There are places in various documentation that says
“IRP_MJ_CREATE is inherently synchronous.”

What does that mean, if anything?

Nothing except that it will be seen as synchronous to the caller -
not to
every filter or FSD.

Is there any way to determine what component of the OS marked the IRP
pending?

I’m afraid not.

Even a more basic question. I see that a FSFD can, uh, mark an IRP pending by
doing a IoMarkIrpPending(irp). Is this the only way to mark an IRP pending?
Does this routine do anything more than setting
IRP->PendingReturned ?

There was an article in NT Insider June 2001 Edition, IIRC, which
explains
IRP pending. Try to find it on OSR site (I think it’s available).
IoMarkIrpPending by itself is nothing - and with Driver Verifier,
you should
get a bug check, I think, if you only do that.
The way to do it is usually:

  • IoMarkIrpPending
  • return STATUS_PENDING from your dispatch routine
    That’s enough for pending and IRP.
    However, most of the time, if one returns STATUS_PENDING in dispatch
    that
    means he needs to do extra work in the completion routine or in a worker
    thread,
    so you will see the completion routine returning
    STATUS_MORE_PROCESSING_REQUIRED,
    but this doesn’t have much with pending directly (FYI, not pending IRPs,
    and
    returning STATUS_MORE_PROCESSING_REQUIRED is the easiest way to filter
    out files,
    in case you want to hide them).


Kind regards, Dejan M. CEO Alfa Co. www.alfasp.com
E-mail: xxxxx@alfasp.com
ICQ#: 56570367
Alfa File Monitor - File monitoring system for Win32 developers.
Alfa File Protector - File protection and hiding system for Win32
developers.


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

This seems to be a very misunderstood concept.
What Tony said is right - one can return STATUS_PENDING for practically
any IRP including IRP_MJ_CREATE, though there is the small subset for
which it means something a bit different. We won’t go into the
exception.

However when the docs (or anybody else) says “create is synchronous” it
means this: the i/o manager waits for the create complete when the
create is issued via ZwCreateFile (or IoCreateFile or the new
IoCreateFileSpecifyDeviceObjectHint). If any driver in the stack returns
STATUS_PENDING, these APIs wait for the IRP to complete before
returning.

This is also true for a few other IRPs.

Folks on this alias - most of whom write filters that sit between the
i/o manager & the file systems,
will always have to deal with STATUS_PENDING appropriately.

-----Original Message-----
From: Ralph Shnelvar [mailto:xxxxx@dos32.com]
Sent: Thursday, November 29, 2001 6:06 AM
To: File Systems Developers
Subject: [ntfsd] Re: STATUS_PENDING in IRP_MJ_CREATE

Dear Tony:

On Wed, 28 Nov 2001 22:43:39 -0500, you wrote:

The I/O model in Windows NT (and 2000, and XP) is *asynchronous*.
Callers may request anything they want. If they call via the I/O
Manager, they will get what they asked for. If they bypass the I/O
Manager and call an FSD directly, they get whatever the FSD provides.

Three more newbie questions:

a)
I am still confused. There are places in various documentation that
says “IRP_MJ_CREATE is inherently synchronous.”

What does that mean, if anything?

b)
Is there any way to determine what component of the OS marked the IRP
pending?

c)
Even a more basic question. I see that a FSFD can, uh, mark an IRP
pending by doing a IoMarkIrpPending(irp). Is this the only way to mark
an IRP pending? Does this routine do anything more than setting
IRP->PendingReturned ?

[snip]

Ah, too much to do and too little time to do it. Off the soapbox now
(it’s been one of those days…)

Gee, every day is one of those days for me.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

Ralph Shnelvar


You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

Microsoft says that FSF drivers should not return STATUS_PENDING from
IRP_MJ_CREATE. I also heard an explanation that FS drivers have to be in the
context of the process which requested this operation. As for me I have
never seen STATUS_PENDING returned from IRP_MJ_CREATE (though I have seen it
returned from IRP_MJ_CLOSE).

Alexey Logachyov
xxxxx@vba.com.by
VirusBlokAda ltd.
http://www.vba.com.by

----- Original Message -----
From: “Ravisankar Pudipeddi”
To: “File Systems Developers”
Sent: Friday, November 30, 2001 6:14 AM
Subject: [ntfsd] Re: STATUS_PENDING in IRP_MJ_CREATE

> This seems to be a very misunderstood concept.
> What Tony said is right - one can return STATUS_PENDING for practically
> any IRP including IRP_MJ_CREATE, though there is the small subset for
> which it means something a bit different. We won’t go into the
> exception.
>
> However when the docs (or anybody else) says “create is synchronous” it
> means this: the i/o manager waits for the create complete when the
> create is issued via ZwCreateFile (or IoCreateFile or the new
> IoCreateFileSpecifyDeviceObjectHint). If any driver in the stack returns
> STATUS_PENDING, these APIs wait for the IRP to complete before
> returning.
>
> This is also true for a few other IRPs.
>
> Folks on this alias - most of whom write filters that sit between the
> i/o manager & the file systems,
> will always have to deal with STATUS_PENDING appropriately.
>
> -----Original Message-----
> From: Ralph Shnelvar [mailto:xxxxx@dos32.com]
> Sent: Thursday, November 29, 2001 6:06 AM
> To: File Systems Developers
> Subject: [ntfsd] Re: STATUS_PENDING in IRP_MJ_CREATE
>
>
> Dear Tony:
>
> On Wed, 28 Nov 2001 22:43:39 -0500, you wrote:
>
> >The I/O model in Windows NT (and 2000, and XP) is asynchronous.
> >Callers may request anything they want. If they call via the I/O
> >Manager, they will get what they asked for. If they bypass the I/O
> >Manager and call an FSD directly, they get whatever the FSD provides.
>
> Three more newbie questions:
>
> a)
> I am still confused. There are places in various documentation that
> says “IRP_MJ_CREATE is inherently synchronous.”
>
> What does that mean, if anything?
>
>
> b)
> Is there any way to determine what component of the OS marked the IRP
> pending?
>
>
> c)
> Even a more basic question. I see that a FSFD can, uh, mark an IRP
> pending by doing a IoMarkIrpPending(irp). Is this the only way to mark
> an IRP pending? Does this routine do anything more than setting
> IRP->PendingReturned ?
>
>
> >
> [snip]
>
> >
> >Ah, too much to do and too little time to do it. Off the soapbox now
> >(it’s been one of those days…)
>
> Gee, every day is one of those days for me.
>
> >
> >Regards,
> >
> >Tony
> >
> >Tony Mason
> >Consulting Partner
> >OSR Open Systems Resources, Inc.
> >http://www.osr.com
> >
>
> Ralph Shnelvar
>
> —
> You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
> To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com
>
> —
> You are currently subscribed to ntfsd as: xxxxx@vba.com.by
> To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com

> Microsoft says that FSF drivers should not return STATUS_PENDING from

IRP_MJ_CREATE.

Keyword: “should not” - not “must not”. Which means some drivers might do
it.

I also heard an explanation that FS drivers have to be in the context of the
process which requested this operation.

I assume this is because security check might return wrong results, as the
caller has different rights from the thread the IRP is processed in.

As for me I have never seen STATUS_PENDING returned from IRP_MJ_CREATE (though
I have seen it returned from IRP_MJ_CLOSE).

STATUS_PENDING is a common return from IRP_MJ_CLOSE, and I rarely saw any
other return code, frankly.


Kind regards, Dejan M. CEO Alfa Co. www.alfasp.com
E-mail: xxxxx@alfasp.com
ICQ#: 56570367
Alfa File Monitor - File monitoring system for Win32 developers.
Alfa File Protector - File protection and hiding system for Win32 developers.


You are currently subscribed to ntfsd as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to leave-ntfsd-$subst(‘Recip.MemberIDChar’)@lists.osr.com