Any ideas about this topic?
Regards,
Ray Yang
xxxxx@ybwork.com
Why do you need this?
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
----- Original Message -----
From: “Ray Yang”
To: “Windows System Software Devs Interest List”
Sent: Saturday, May 08, 2004 7:50 PM
Subject: [ntdev] Any way to know if a buffer is paged or non-paged pool?
> Any ideas about this topic?
>
> Regards,
> Ray Yang
> xxxxx@ybwork.com
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
My filter driver processes some IRPs and get the data when these IRPs
finish. Some of data buffer are in the paged pool and some in the non-paged
pool.
But, my IRP completion processing routine runs in PASSIVE_LEVEL or
DISPATCH_LEVEL. So, to make this routine to get data in paged pool even if
it’s running at DISPATCH_LEVEL, I need to lock (MmProbeAndLockPages) paged
pool buffer before issuing the IRP to the lower driver.
I’ve tried to lock all buffer (including paged and non-paged). But it seems
that if I lock a non-paged buffer at PASSIVE_LEVEL, OS crashes in
MmUnlockPages.
Then, another question: is it safe to lock a buffer with MmProbeAndLockPages
for more than 1 times? E.g., the upper driver locks a buffer with
MmProbeAndLockPages, then my filter driver locks it too. Is this ok?
Regards,
Ray Yang
xxxxx@ybwork.com
----- Original Message -----
From: “Maxim S. Shatskih”
To: “Windows System Software Devs Interest List”
Sent: Sunday, May 09, 2004 1:09 AM
Subject: Re: [ntdev] Any way to know if a buffer is paged or non-paged pool?
> Why do you need this?
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
> ----- Original Message -----
> From: “Ray Yang”
> To: “Windows System Software Devs Interest List”
> Sent: Saturday, May 08, 2004 7:50 PM
> Subject: [ntdev] Any way to know if a buffer is paged or non-paged pool?
>
>
> > Any ideas about this topic?
> >
> > Regards,
> > Ray Yang
> > xxxxx@ybwork.com
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@ybwork.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
- Consider moving all data to nonpaged pool
- Call IoQueueWorkItem from the completion routine, the work item is
allowed to touch pageable memory.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
----- Original Message -----
From: “Ray Yang”
To: “Windows System Software Devs Interest List”
Sent: Sunday, May 09, 2004 4:33 AM
Subject: Re: [ntdev] Any way to know if a buffer is paged or non-paged pool?
> My filter driver processes some IRPs and get the data when these IRPs
> finish. Some of data buffer are in the paged pool and some in the non-paged
> pool.
>
> But, my IRP completion processing routine runs in PASSIVE_LEVEL or
> DISPATCH_LEVEL. So, to make this routine to get data in paged pool even if
> it’s running at DISPATCH_LEVEL, I need to lock (MmProbeAndLockPages) paged
> pool buffer before issuing the IRP to the lower driver.
>
> I’ve tried to lock all buffer (including paged and non-paged). But it seems
> that if I lock a non-paged buffer at PASSIVE_LEVEL, OS crashes in
> MmUnlockPages.
>
> Then, another question: is it safe to lock a buffer with MmProbeAndLockPages
> for more than 1 times? E.g., the upper driver locks a buffer with
> MmProbeAndLockPages, then my filter driver locks it too. Is this ok?
>
> Regards,
> Ray Yang
> xxxxx@ybwork.com
> ----- Original Message -----
> From: “Maxim S. Shatskih”
> To: “Windows System Software Devs Interest List”
> Sent: Sunday, May 09, 2004 1:09 AM
> Subject: Re: [ntdev] Any way to know if a buffer is paged or non-paged pool?
>
>
> > Why do you need this?
> >
> > Maxim Shatskih, Windows DDK MVP
> > StorageCraft Corporation
> > xxxxx@storagecraft.com
> > http://www.storagecraft.com
> >
> >
> > ----- Original Message -----
> > From: “Ray Yang”
> > To: “Windows System Software Devs Interest List”
> > Sent: Saturday, May 08, 2004 7:50 PM
> > Subject: [ntdev] Any way to know if a buffer is paged or non-paged pool?
> >
> >
> > > Any ideas about this topic?
> > >
> > > Regards,
> > > Ray Yang
> > > xxxxx@ybwork.com
> > >
> > > —
> > > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> > >
> > > You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@ybwork.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
Maxim,
Thanks for you reply.
I’ve found the reason of the OS crash - I misunderstood the behavior of IO
Manager and set a *wrong* length for memory copy :).
Anyway, your idea is a good solution for normal drivers, good.
Well, there is still no answer for another question about multiple lock of a
buffer. I know MS *just* suggests the highest-level driver to do this. But,
is it stupid for an intermediate driver to do this?
Regards,
Ray Yang
xxxxx@ybwork.com
----- Original Message -----
From: “Maxim S. Shatskih”
To: “Windows System Software Devs Interest List”
Sent: Sunday, May 09, 2004 9:20 PM
Subject: Re: [ntdev] Any way to know if a buffer is paged or non-paged pool?
> 1) Consider moving all data to nonpaged pool
> 2) Call IoQueueWorkItem from the completion routine, the work item is
> allowed to touch pageable memory.
>
> Maxim Shatskih, Windows DDK MVP
> StorageCraft Corporation
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
>
> ----- Original Message -----
> From: “Ray Yang”
> To: “Windows System Software Devs Interest List”
> Sent: Sunday, May 09, 2004 4:33 AM
> Subject: Re: [ntdev] Any way to know if a buffer is paged or non-paged
pool?
>
>
> > My filter driver processes some IRPs and get the data when these IRPs
> > finish. Some of data buffer are in the paged pool and some in the
non-paged
> > pool.
> >
> > But, my IRP completion processing routine runs in PASSIVE_LEVEL or
> > DISPATCH_LEVEL. So, to make this routine to get data in paged pool even
if
> > it’s running at DISPATCH_LEVEL, I need to lock (MmProbeAndLockPages)
paged
> > pool buffer before issuing the IRP to the lower driver.
> >
> > I’ve tried to lock all buffer (including paged and non-paged). But it
seems
> > that if I lock a non-paged buffer at PASSIVE_LEVEL, OS crashes in
> > MmUnlockPages.
> >
> > Then, another question: is it safe to lock a buffer with
MmProbeAndLockPages
> > for more than 1 times? E.g., the upper driver locks a buffer with
> > MmProbeAndLockPages, then my filter driver locks it too. Is this ok?
> >
> > Regards,
> > Ray Yang
> > xxxxx@ybwork.com
> > ----- Original Message -----
> > From: “Maxim S. Shatskih”
> > To: “Windows System Software Devs Interest List”
> > Sent: Sunday, May 09, 2004 1:09 AM
> > Subject: Re: [ntdev] Any way to know if a buffer is paged or non-paged
pool?
> >
> >
> > > Why do you need this?
> > >
> > > Maxim Shatskih, Windows DDK MVP
> > > StorageCraft Corporation
> > > xxxxx@storagecraft.com
> > > http://www.storagecraft.com
> > >
> > >
> > > ----- Original Message -----
> > > From: “Ray Yang”
> > > To: “Windows System Software Devs Interest List”
> > > Sent: Saturday, May 08, 2004 7:50 PM
> > > Subject: [ntdev] Any way to know if a buffer is paged or non-paged
pool?
> > >
> > >
> > > > Any ideas about this topic?
> > > >
> > > > Regards,
> > > > Ray Yang
> > > > xxxxx@ybwork.com
> > > >
> > > > —
> > > > Questions? First check the Kernel Driver FAQ at
> > > http://www.osronline.com/article.cfm?id=256
> > > >
> > > > You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> > > > To unsubscribe send a blank email to
xxxxx@lists.osr.com
> > >
> > >
> > > —
> > > Questions? First check the Kernel Driver FAQ at
> > http://www.osronline.com/article.cfm?id=256
> > >
> > > You are currently subscribed to ntdev as: xxxxx@ybwork.com
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> > >
> >
> >
> > —
> > Questions? First check the Kernel Driver FAQ at
> http://www.osronline.com/article.cfm?id=256
> >
> > You are currently subscribed to ntdev as: xxxxx@storagecraft.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
> —
> Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256
>
> You are currently subscribed to ntdev as: xxxxx@ybwork.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
Ray Yang wrote:
Well, there is still no answer for another question about multiple lock of a
buffer. I know MS *just* suggests the highest-level driver to do this. But,
is it stupid for an intermediate driver to do this?
Hmmmm… Not sure what you’re looking for as an answer here.
Going down the stack, drivers should be calling
MmGetSystemAddressForMdlSafe(…) – This does the mapping if necessary
and returns the already existing mapped kernel virtual address. If you
hav an MDL, the pages are already probed and locked, right?
In terms of your literal question “Is it safe to probe and lock a buffer
multiple times” the answer is “Yes”. Probing certainly hurts nothing.
And locking the buffer merely increments the reference count on
the page. This implies, of course, that you must unlock as many times
as you lock.
HTH,
Peter
OSR
My experience with multiple probe and locks is that it is not counted,
at least it was not in Nt3.51/NT4?
A driver maintained list of locked virtual memory address/range was a
solution.
ned
PeterGV wrote:
Ray Yang wrote:
>
> Well, there is still no answer for another question about multiple
> lock of a
> buffer. I know MS *just* suggests the highest-level driver to do
> this. But,
> is it stupid for an intermediate driver to do this?
>Hmmmm… Not sure what you’re looking for as an answer here.
Going down the stack, drivers should be calling
MmGetSystemAddressForMdlSafe(…) – This does the mapping if
necessary and returns the already existing mapped kernel virtual
address. If you hav an MDL, the pages are already probed and locked,
right?In terms of your literal question “Is it safe to probe and lock a
buffer multiple times” the answer is “Yes”. Probing certainly hurts
nothing. And locking the buffer merely increments the
reference count on the page. This implies, of course, that you must
unlock as many times as you lock.HTH,
Peter
OSR
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256You are currently subscribed to ntdev as: xxxxx@colorts.com.au
To unsubscribe send a blank email to xxxxx@lists.osr.com
“ned kacavenda” wrote in message
news:xxxxx@ntdev…
> My experience with multiple probe and locks is that it is not counted,
> at least it was not in Nt3.51/NT4?
> A driver maintained list of locked virtual memory address/range was a
> solution.
>
I’m sorry, but one of us is having a memory failure.
As far back as I can remember, MmProbeAndLockPages has incremented the
ReferenceCount field. This MUST be the case… otherwise, how would the
memory manager know when it was safe to release a page? It has always been
possible for multiple drivers to have I/O in progress to the same physical
page of memory, right? Without reference counting, how could Mm keep track
of this?
In ANY case, I can tell you for sure that it works that way back at least to
Win2K,
Peter
OSR
> As far back as *I* can remember, MmProbeAndLockPages has incremented the
ReferenceCount field. This MUST be the case… otherwise, how would the
memory manager know when it was safe to release a page? It has always been
In some NT versions (dunno what), MmProbeAndLockPages does NOT increment the
refcounts if it is sure that the memory is from NPP.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
Peter,
A scenario where locking seemed not to have been reference counted is
where an user application
queues severnal I/O requests for same virtual memory range. The
operation was done via ioctl,
METHOD_NEITHER, with a structure being passed in to driver that
described i/o operation,
including the memory virtual address and length.
All queued requests had to have memory locked prior to IRP being queued
in internal list.
In some cases memory ranges being locked by each IRP requests were
unique and in other
cases same memory range may be referred by a number of queued IRPs.
Dynamics of I/O
operation also meant that request yet to come in may also deal with
memory already locked
down by queued IRPS.
As IRPS’s were processed and completed, the associated virtual memory
was unlocked.
If all IRPs referred to distinct memory blocks there was no problem.
If multiple
IRPs were dealing with same vritual memory ranges the system there were
problems.
I do not recall exactly whether we had system bluescreen or whether DMA
chains
could not be built (this was in days of NT3.51) but the solution was to
manage locking
in the driver by internally reference count locked memory.
Incidentally, this has helped in another case where very large memory
block of 260MB
in size was part of I/O operation on two devices in same device driver.
Locking would not succeed on account of insufficient resources.
Solution was to
have a common list for locked memory ranges.
Current version of Win2K and XP make it almost impossible to lock down
large amounts
of virtual memory, ie 100’s of megabytes.
Another thing that got us into strife with Win2K as opposed to 3.51 and
4 was that
if virtual memory range was described by multiple VADs the
MmProbeAndLockPages()
would fail. Attempt to use MmMprobeAndLockSelectedPages() would work but
then MmUnlockPages() would bluescreen.
You may be right about refrence counting, maybe we did not investigate
the issue
properly. As a side issue, how is refrence counting done? one per-page
basis
or some other?
ned
Peter Viscarola wrote:
“ned kacavenda” wrote in message
>news:xxxxx@ntdev…
>
>
>>My experience with multiple probe and locks is that it is not counted,
>>at least it was not in Nt3.51/NT4?
>>A driver maintained list of locked virtual memory address/range was a
>>solution.
>>
>>
>>
>
>I’m sorry, but one of us is having a memory failure.
>
>As far back as I can remember, MmProbeAndLockPages has incremented the
>ReferenceCount field. This MUST be the case… otherwise, how would the
>memory manager know when it was safe to release a page? It has always been
>possible for multiple drivers to have I/O in progress to the same physical
>page of memory, right? Without reference counting, how could Mm keep track
>of this?
>
>In ANY case, I can tell you for sure that it works that way back at least to
>Win2K,
>
>Peter
>OSR
>
>
>
>—
>Questions? First check the Kernel Driver FAQ at http://www.osronline.com/article.cfm?id=256
>
>You are currently subscribed to ntdev as: xxxxx@colorts.com.au
>To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
>
> properly. As a side issue, how is refrence counting done? one per-page
In the PFN database indexed by the physical page number. Mis-counting causes
PFN_LIST_CORRUPT BSOD.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com