Driver Verifier and low resources testing

When testing low resources response, does verifier at any point “free”
resources to permit the driver to function or does it just hog them? Or …
what needs to be done to “free” Verifier garnished resources. Case in point:
I set a 5 retry loop to map a user buffer for kernel mode using
MmProbeandLock pages. With LR enabled in Verifier the loop would iterate 5
times and the driver return an error status. Is 5 retries sufficient? 100?
200? When/or how does Verifier “free” the resources it stole?

How do I know it was Verifier? Users without Verifier enabled for LR were
not seeing the same status returned.

You should probably fail the request, or enusre that you have enough
resources up front to process it rather than retrying until memory
happens to show up.

-p

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Thursday, June 10, 2004 11:26 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] Driver Verifier and low resources testing

When testing low resources response, does verifier at any point “free”
resources to permit the driver to function or does it just hog them? Or

what needs to be done to “free” Verifier garnished resources. Case in
point:
I set a 5 retry loop to map a user buffer for kernel mode using
MmProbeandLock pages. With LR enabled in Verifier the loop would iterate
5 times and the driver return an error status. Is 5 retries sufficient?
100?
200? When/or how does Verifier “free” the resources it stole?

How do I know it was Verifier? Users without Verifier enabled for LR
were not seeing the same status returned.


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

You’re really suggesting failing the request. Check for resources and fail,
or try the allocation and fail. Right now I fail the request after 5 blind
retries, which did not seem to be enough to make Verifier happy, if it could
be made happy. I take it then there is no way to eventually succeed the
request, whether or not I check for the resources prior to requesting the
resources?

And a corollary question … So I check for resource availability … what’s
to guarantee that Verifier won’t garnish the resources and not allow them to
be allocated when I request them?


Gary G. Little
Seagate Technologies, LLC

“Peter Wieland” wrote in message
news:xxxxx@ntdev…
You should probably fail the request, or enusre that you have enough
resources up front to process it rather than retrying until memory
happens to show up.

-p

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Thursday, June 10, 2004 11:26 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] Driver Verifier and low resources testing

When testing low resources response, does verifier at any point “free”
resources to permit the driver to function or does it just hog them? Or

what needs to be done to “free” Verifier garnished resources. Case in
point:
I set a 5 retry loop to map a user buffer for kernel mode using
MmProbeandLock pages. With LR enabled in Verifier the loop would iterate
5 times and the driver return an error status. Is 5 retries sufficient?
100?
200? When/or how does Verifier “free” the resources it stole?

How do I know it was Verifier? Users without Verifier enabled for LR
were not seeing the same status returned.


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Well, if you fail the request “cleanly” driver verifier should be
happy as can be. The point with this low-resource testing is only to
assure the driver does not mess anything up when resources are low.
And if you have an absolute “must succeed” scenario maybe you should
design the interface in a way that allows you to preallocate
everything you’ll need when starting up.
If you can’t do that you can never assume you’ll get anything :wink:

what’s to guarantee that Verifier won’t garnish the resources and not
allow them to be allocated when I request them?

Nothing. Just look what you need, try allocate everything, if anything
fails, release everythig allocated so far and fail the request.
Do this before you do any other steps to process the request.
I think that’s what was ment by “enusre that you have enough resources
up front”. Either that or preallocating at an early point like device-
startup.

Regards,

Paul Groke

“Gary G. Little”
Gesendet von: xxxxx@lists.osr.com
10.06.2004 22:08
Bitte antworten an “Windows System Software Devs Interest List”

An: “Windows System Software Devs Interest List”

Kopie:
Thema: Re:[ntdev] Driver Verifier and low resources testing

You’re really suggesting failing the request. Check for resources and
fail,
or try the allocation and fail. Right now I fail the request after 5 blind
retries, which did not seem to be enough to make Verifier happy, if it
could
be made happy. I take it then there is no way to eventually succeed the
request, whether or not I check for the resources prior to requesting the
resources?

And a corollary question … So I check for resource availability …
what’s
to guarantee that Verifier won’t garnish the resources and not allow them
to
be allocated when I request them?


Gary G. Little
Seagate Technologies, LLC

“Peter Wieland” wrote in message
news:xxxxx@ntdev…
You should probably fail the request, or enusre that you have enough
resources up front to process it rather than retrying until memory
happens to show up.

-p

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Thursday, June 10, 2004 11:26 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] Driver Verifier and low resources testing

When testing low resources response, does verifier at any point “free”
resources to permit the driver to function or does it just hog them? Or

what needs to be done to “free” Verifier garnished resources. Case in
point:
I set a 5 retry loop to map a user buffer for kernel mode using
MmProbeandLock pages. With LR enabled in Verifier the loop would iterate
5 times and the driver return an error status. Is 5 retries sufficient?
100?
200? When/or how does Verifier “free” the resources it stole?

How do I know it was Verifier? Users without Verifier enabled for LR
were not seeing the same status returned.


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

You need to decide if you have to handle the request or if it’s okay to
fail it in low memory.

If you have to handle it, then you need to ensure you’ll have enough
memory to process it. That probably involves having enough of all the
various objects you allocate around that you can process one request at
a time combined with a serialization mechanism that queues outstanding
I/O until some resources (the preallocated ones or some others) become
available.

The lower part of the storage stack does this to ensure forward progress
on read/write requests when memory is low - if it didn’t then an attempt
to page out memory to make more pages could fail due to low memory …
that would be bad. Classpnp has to have irps, srbs and MDLs allocated
ahead of time for this, scsiport has to keep srb_data blocks, scatter
gather lists, system VA windows and map registers around to do the same.
It’s mangable because paging I/O is always <= the maximum tranfer size
of the adapter so we know how much memory we need for a single request.

When I say “ensure you have enough resources available” I mean that you
should allocate the memory. How else would you make sure there’s enough
memroy when you want it.

-p

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Thursday, June 10, 2004 1:09 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Driver Verifier and low resources testing

You’re really suggesting failing the request. Check for resources and
fail, or try the allocation and fail. Right now I fail the request after
5 blind retries, which did not seem to be enough to make Verifier happy,
if it could be made happy. I take it then there is no way to eventually
succeed the request, whether or not I check for the resources prior to
requesting the resources?

And a corollary question … So I check for resource availability …
what’s to guarantee that Verifier won’t garnish the resources and not
allow them to be allocated when I request them?


Gary G. Little
Seagate Technologies, LLC

“Peter Wieland” wrote in message
news:xxxxx@ntdev…
You should probably fail the request, or enusre that you have enough
resources up front to process it rather than retrying until memory
happens to show up.

-p

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Thursday, June 10, 2004 11:26 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] Driver Verifier and low resources testing

When testing low resources response, does verifier at any point “free”
resources to permit the driver to function or does it just hog them? Or

what needs to be done to “free” Verifier garnished resources. Case in
point:
I set a 5 retry loop to map a user buffer for kernel mode using
MmProbeandLock pages. With LR enabled in Verifier the loop would iterate
5 times and the driver return an error status. Is 5 retries sufficient?
100?
200? When/or how does Verifier “free” the resources it stole?

How do I know it was Verifier? Users without Verifier enabled for LR
were not seeing the same status returned.


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Have to be careful here in doing this. I suddenly found myself trying to
“fix” the driver so Verifier would be happy, when the real intent is to make
sure that realworld incidents of low resources are handled properly.
Talking with the app folks the decision is the driver will return a low
resources status and the application will then make the decision to retry or
not to retry. Since I’m mapping a buffer received from the application to
kernel space, and avoiding a copy, it’s difficult to pre-allocate the
buffers required by all of the app(s).

My original thought was what is required to “reset” Verifier to release
resources for use, and then apply that to properly handling true low
resources circumstances. Just waiting in a loop is not the right way, as was
pointed out.

However … this does point out a fault in the current design and will be
rectified in the next iteration — putting the IRP on a queue and then
pending it until resources are available or the IRP can be properly handled.


Gary G. Little
Seagate Technologies, LLC

“Peter Wieland” wrote in message
news:xxxxx@ntdev…
You need to decide if you have to handle the request or if it’s okay to
fail it in low memory.

If you have to handle it, then you need to ensure you’ll have enough
memory to process it. That probably involves having enough of all the
various objects you allocate around that you can process one request at
a time combined with a serialization mechanism that queues outstanding
I/O until some resources (the preallocated ones or some others) become
available.

The lower part of the storage stack does this to ensure forward progress
on read/write requests when memory is low - if it didn’t then an attempt
to page out memory to make more pages could fail due to low memory …
that would be bad. Classpnp has to have irps, srbs and MDLs allocated
ahead of time for this, scsiport has to keep srb_data blocks, scatter
gather lists, system VA windows and map registers around to do the same.
It’s mangable because paging I/O is always <= the maximum tranfer size
of the adapter so we know how much memory we need for a single request.

When I say “ensure you have enough resources available” I mean that you
should allocate the memory. How else would you make sure there’s enough
memroy when you want it.

-p

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Thursday, June 10, 2004 1:09 PM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Driver Verifier and low resources testing

You’re really suggesting failing the request. Check for resources and
fail, or try the allocation and fail. Right now I fail the request after
5 blind retries, which did not seem to be enough to make Verifier happy,
if it could be made happy. I take it then there is no way to eventually
succeed the request, whether or not I check for the resources prior to
requesting the resources?

And a corollary question … So I check for resource availability …
what’s to guarantee that Verifier won’t garnish the resources and not
allow them to be allocated when I request them?


Gary G. Little
Seagate Technologies, LLC

“Peter Wieland” wrote in message
news:xxxxx@ntdev…
You should probably fail the request, or enusre that you have enough
resources up front to process it rather than retrying until memory
happens to show up.

-p

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Gary G. Little
Sent: Thursday, June 10, 2004 11:26 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] Driver Verifier and low resources testing

When testing low resources response, does verifier at any point “free”
resources to permit the driver to function or does it just hog them? Or

what needs to be done to “free” Verifier garnished resources. Case in
point:
I set a 5 retry loop to map a user buffer for kernel mode using
MmProbeandLock pages. With LR enabled in Verifier the loop would iterate
5 times and the driver return an error status. Is 5 retries sufficient?
100?
200? When/or how does Verifier “free” the resources it stole?

How do I know it was Verifier? Users without Verifier enabled for LR
were not seeing the same status returned.


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com