PageFault after pinning the range.

Hello everyone,

I am using the pinning interface provided by cache manager to map a view of
a file.The problem is even after I have mapped an offset and pinned it I
still get a pagefault in the range while accessing it, this results in a
bugcheck PAGE_FAULT_IN_NON_PAGED_AREA.This happens when I am pinning a range
of 3-4 MB.If a smaller range 5-6KB is specified there is no problem.The
valid data length is equals the end of file.

One thing which I had observed when the crash happens is that the BCB which
we obtain contains a list of child BCB ,these child BCB corresponds to the
range,length & the corresponding virtual address(at offset 0x70 I
think),the virtual address for which the pagefault is caused does not have
an associated BCB,instead the Offset for which pagefault is caused has been
mapped to a different address.So is there an upper limit to the range which
can be mapped I am describing the steps which I am doing to achieve it.
The steps I perform are in a system thread
1.Initiate caching for a file using CcInitializeCacheMap.
2.MapOffset in the file say from 1024*4096 ,of length 4096 CcPinRead with
PIN_WAIT .
3.touch the pointer by a memcpy.
4.Unpin the BCB.
5.MapOffset in the file say from 0 ,of length 4096*1000 CcPinRead with
PIN_WAIT .
6.Touch the pointer obtained by a memset ((PCHAR)pBuffer,4096*1000);//there
are no overruns believe me.
Now,I start getting paging I/O read the read offset > 4096*1000 on the
device object associated a call stack shows that I have caused a pagefault
for a memory location well under (PCHAR)pBuffer+4096*1000 after some time
we have a crash.

HELP!

Thank you for reading the mail.
regards,
Johri.

The cache manager manages files in the system cache in 256K aligned
views. Pinned regions cannot span more than one 256K view, therefore,
you cannot pin 3-4MB at a time. The most you can pin is 256K starting
at a 256K-aligned offset in the file.

As a side note, using the CcPinXxx interface to map a range of a file
does not ensure that the pages remain resident in memory. While those
pages are pinned, you are guaranteed that the file range will stay
mapped into the system cache virtual address space, but the memory
manager can page out the physical pages as the system’s memory demand
requires.

Thanks,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Amritanshu
Sent: Wednesday, March 17, 2004 5:21 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] PageFault after pinning the range.

Hello everyone,

I am using the pinning interface provided by cache manager to map a view
of
a file.The problem is even after I have mapped an offset and pinned it
I
still get a pagefault in the range while accessing it, this results in
a
bugcheck PAGE_FAULT_IN_NON_PAGED_AREA.This happens when I am pinning a
range
of 3-4 MB.If a smaller range 5-6KB is specified there is no problem.The
valid data length is equals the end of file.

One thing which I had observed when the crash happens is that the BCB
which
we obtain contains a list of child BCB ,these child BCB corresponds to
the
range,length & the corresponding virtual address(at offset 0x70 I
think),the virtual address for which the pagefault is caused does not
have
an associated BCB,instead the Offset for which pagefault is caused has
been
mapped to a different address.So is there an upper limit to the range
which
can be mapped I am describing the steps which I am doing to achieve it.
The steps I perform are in a system thread
1.Initiate caching for a file using CcInitializeCacheMap.
2.MapOffset in the file say from 1024*4096 ,of length 4096 CcPinRead
with
PIN_WAIT .
3.touch the pointer by a memcpy.
4.Unpin the BCB.
5.MapOffset in the file say from 0 ,of length 4096*1000 CcPinRead with
PIN_WAIT .
6.Touch the pointer obtained by a memset
((PCHAR)pBuffer,4096*1000);//there
are no overruns believe me.
Now,I start getting paging I/O read the read offset > 4096*1000 on the
device object associated a call stack shows that I have caused a
pagefault
for a memory location well under (PCHAR)pBuffer+4096*1000 after some
time
we have a crash.

HELP!

Thank you for reading the mail.
regards,
Johri.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Molly,
Thank you very much for replying I had also concluded that 0x40000 was the
biggest range that can be mapped consistently.
But I have a few questions
1.Maybe I am wrong but the IFS kit which I have does not mention it so if
there is another resource then do let me know.I have been looking
around for some information for some time now.
2.Secondly perhaps the call could have looked at the arguments and said
dude you are not doing the right thing therefore I fail the request by
returning a false.I tried it with checked build version of windows 2000
maybe an assert will help.But it doesn’t do that.So I concluded some of the
drivers from Microsoft can use the call for boundaries gr8er then 256 KB or
something identical all they have to do is understand the BCB.I am new to
windows development but if this is the way things have to go then we all
will be interacting a lot ;).

with best regards,
Johri.


“Molly Brown” wrote in message
news:xxxxx@ntfsd…
The cache manager manages files in the system cache in 256K aligned
views. Pinned regions cannot span more than one 256K view, therefore,
you cannot pin 3-4MB at a time. The most you can pin is 256K starting
at a 256K-aligned offset in the file.

As a side note, using the CcPinXxx interface to map a range of a file
does not ensure that the pages remain resident in memory. While those
pages are pinned, you are guaranteed that the file range will stay
mapped into the system cache virtual address space, but the memory
manager can page out the physical pages as the system’s memory demand
requires.

Thanks,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Amritanshu
Sent: Wednesday, March 17, 2004 5:21 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] PageFault after pinning the range.

Hello everyone,

I am using the pinning interface provided by cache manager to map a view
of
a file.The problem is even after I have mapped an offset and pinned it
I
still get a pagefault in the range while accessing it, this results in
a
bugcheck PAGE_FAULT_IN_NON_PAGED_AREA.This happens when I am pinning a
range
of 3-4 MB.If a smaller range 5-6KB is specified there is no problem.The
valid data length is equals the end of file.

One thing which I had observed when the crash happens is that the BCB
which
we obtain contains a list of child BCB ,these child BCB corresponds to
the
range,length & the corresponding virtual address(at offset 0x70 I
think),the virtual address for which the pagefault is caused does not
have
an associated BCB,instead the Offset for which pagefault is caused has
been
mapped to a different address.So is there an upper limit to the range
which
can be mapped I am describing the steps which I am doing to achieve it.
The steps I perform are in a system thread
1.Initiate caching for a file using CcInitializeCacheMap.
2.MapOffset in the file say from 10244096 ,of length 4096 CcPinRead
with
PIN_WAIT .
3.touch the pointer by a memcpy.
4.Unpin the BCB.
5.MapOffset in the file say from 0 ,of length 4096
1000 CcPinRead with
PIN_WAIT .
6.Touch the pointer obtained by a memset
((PCHAR)pBuffer,40961000);//there
are no overruns believe me.
Now,I start getting paging I/O read the read offset > 4096
1000 on the
device object associated a call stack shows that I have caused a
pagefault
for a memory location well under (PCHAR)pBuffer+4096*1000 after some
time
we have a crash.

HELP!

Thank you for reading the mail.
regards,
Johri.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

I can’t find where this fact is documented either :). I’ll file a bug
to get this point added to the documentation.

Yes, adding more asserts/bugchecks is a good idea, but that’s not going
to help you for Windows 2000 development. The file system samples which
ship as part of the IFS Kit will be your best reference. If you are
going to be writing a file system, you may want to look into taking the
file system class which OSR offers. I think you will be able to find
lots of people on this list who will say that it’s a helpful class.

Event the Microsoft drivers must live with this same limitation. The
BCB structure returned to the driver calling the Cc APIs are opaque to
everyone but the cache manager.

Thanks,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Amritanshu
Sent: Wednesday, March 17, 2004 10:34 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] PageFault after pinning the range.

Molly,
Thank you very much for replying I had also concluded that 0x40000 was
the biggest range that can be mapped consistently.
But I have a few questions
1.Maybe I am wrong but the IFS kit which I have does not mention it so
if there is another resource then do let me know.I have been looking
around for some information for some time now.
2.Secondly perhaps the call could have looked at the arguments and said
dude you are not doing the right thing therefore I fail the request by
returning a false.I tried it with checked build version of windows 2000
maybe an assert will help.But it doesn’t do that.So I concluded some of
the drivers from Microsoft can use the call for boundaries gr8er then
256 KB or something identical all they have to do is understand the
BCB.I am new to windows development but if this is the way things have
to go then we all will be interacting a lot ;).

with best regards,
Johri.


“Molly Brown” wrote in message
news:xxxxx@ntfsd…
The cache manager manages files in the system cache in 256K aligned
views. Pinned regions cannot span more than one 256K view, therefore,
you cannot pin 3-4MB at a time. The most you can pin is 256K starting
at a 256K-aligned offset in the file.

As a side note, using the CcPinXxx interface to map a range of a file
does not ensure that the pages remain resident in memory. While those
pages are pinned, you are guaranteed that the file range will stay
mapped into the system cache virtual address space, but the memory
manager can page out the physical pages as the system’s memory demand
requires.

Thanks,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Amritanshu
Sent: Wednesday, March 17, 2004 5:21 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] PageFault after pinning the range.

Hello everyone,

I am using the pinning interface provided by cache manager to map a view
of a file.The problem is even after I have mapped an offset and pinned
it I still get a pagefault in the range while accessing it, this results
in a bugcheck PAGE_FAULT_IN_NON_PAGED_AREA.This happens when I am
pinning a range of 3-4 MB.If a smaller range 5-6KB is specified there
is no problem.The valid data length is equals the end of file.

One thing which I had observed when the crash happens is that the BCB
which we obtain contains a list of child BCB ,these child BCB
corresponds to the range,length & the corresponding virtual address(at
offset 0x70 I think),the virtual address for which the pagefault is
caused does not have an associated BCB,instead the Offset for which
pagefault is caused has been mapped to a different address.So is there
an upper limit to the range which can be mapped I am describing the
steps which I am doing to achieve it.
The steps I perform are in a system thread 1.Initiate caching for a file
using CcInitializeCacheMap.
2.MapOffset in the file say from 10244096 ,of length 4096 CcPinRead
with PIN_WAIT .
3.touch the pointer by a memcpy.
4.Unpin the BCB.
5.MapOffset in the file say from 0 ,of length 4096
1000 CcPinRead with
PIN_WAIT .
6.Touch the pointer obtained by a memset
((PCHAR)pBuffer,40961000);//there
are no overruns believe me.
Now,I start getting paging I/O read the read offset > 4096
1000 on the
device object associated a call stack shows that I have caused a
pagefault for a memory location well under (PCHAR)pBuffer+4096*1000
after some time we have a crash.

HELP!

Thank you for reading the mail.
regards,
Johri.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Hello Molly,
I am back to bother you again I just needed to clarify one thing, in your
first reply you said that
cache manager maps view aligned to 256K boundary so does that mean that a
view at offset =
256k-4096 of length 8192(i.e… a view spanning two boundaries) cannot be
mapped.
And this restriction will be there for MDL Interface as well right.
thanking you for your patience,
with best regards,
Johri.


“Molly Brown” wrote in message
news:xxxxx@ntfsd…
I can’t find where this fact is documented either :). I’ll file a bug
to get this point added to the documentation.

Yes, adding more asserts/bugchecks is a good idea, but that’s not going
to help you for Windows 2000 development. The file system samples which
ship as part of the IFS Kit will be your best reference. If you are
going to be writing a file system, you may want to look into taking the
file system class which OSR offers. I think you will be able to find
lots of people on this list who will say that it’s a helpful class.

Event the Microsoft drivers must live with this same limitation. The
BCB structure returned to the driver calling the Cc APIs are opaque to
everyone but the cache manager.

Thanks,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Amritanshu
Sent: Wednesday, March 17, 2004 10:34 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] PageFault after pinning the range.

Molly,
Thank you very much for replying I had also concluded that 0x40000 was
the biggest range that can be mapped consistently.
But I have a few questions
1.Maybe I am wrong but the IFS kit which I have does not mention it so
if there is another resource then do let me know.I have been looking
around for some information for some time now.
2.Secondly perhaps the call could have looked at the arguments and said
dude you are not doing the right thing therefore I fail the request by
returning a false.I tried it with checked build version of windows 2000
maybe an assert will help.But it doesn’t do that.So I concluded some of
the drivers from Microsoft can use the call for boundaries gr8er then
256 KB or something identical all they have to do is understand the
BCB.I am new to windows development but if this is the way things have
to go then we all will be interacting a lot ;).

with best regards,
Johri.
_______________________________
“Molly Brown” wrote in message
news:xxxxx@ntfsd…
The cache manager manages files in the system cache in 256K aligned
views. Pinned regions cannot span more than one 256K view, therefore,
you cannot pin 3-4MB at a time. The most you can pin is 256K starting
at a 256K-aligned offset in the file.

As a side note, using the CcPinXxx interface to map a range of a file
does not ensure that the pages remain resident in memory. While those
pages are pinned, you are guaranteed that the file range will stay
mapped into the system cache virtual address space, but the memory
manager can page out the physical pages as the system’s memory demand
requires.

Thanks,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Amritanshu
Sent: Wednesday, March 17, 2004 5:21 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] PageFault after pinning the range.

Hello everyone,

I am using the pinning interface provided by cache manager to map a view
of a file.The problem is even after I have mapped an offset and pinned
it I still get a pagefault in the range while accessing it, this results
in a bugcheck PAGE_FAULT_IN_NON_PAGED_AREA.This happens when I am
pinning a range of 3-4 MB.If a smaller range 5-6KB is specified there
is no problem.The valid data length is equals the end of file.

One thing which I had observed when the crash happens is that the BCB
which we obtain contains a list of child BCB ,these child BCB
corresponds to the range,length & the corresponding virtual address(at
offset 0x70 I think),the virtual address for which the pagefault is
caused does not have an associated BCB,instead the Offset for which
pagefault is caused has been mapped to a different address.So is there
an upper limit to the range which can be mapped I am describing the
steps which I am doing to achieve it.
The steps I perform are in a system thread 1.Initiate caching for a file
using CcInitializeCacheMap.
2.MapOffset in the file say from 10244096 ,of length 4096 CcPinRead
with PIN_WAIT .
3.touch the pointer by a memcpy.
4.Unpin the BCB.
5.MapOffset in the file say from 0 ,of length 4096
1000 CcPinRead with
PIN_WAIT .
6.Touch the pointer obtained by a memset
((PCHAR)pBuffer,40961000);//there
are no overruns believe me.
Now,I start getting paging I/O read the read offset > 4096
1000 on the
device object associated a call stack shows that I have caused a
pagefault for a memory location well under (PCHAR)pBuffer+4096*1000
after some time we have a crash.

HELP!

Thank you for reading the mail.
regards,
Johri.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Yes, a view starting at offset 256k-4096 for a length of 8192 cannot be
mapped. The cache manager’s views to the file are 256K in size and
start at each 256K boundary (offset 0, 256K, 512K, etc.).

The MDL interfaces can span the cache manager’s 256K view boundaries
since a MdlChain is returned. If the range specified crosses more than
one view, more than one Mdl will be returned to describe the file region
(represented by the MdlChain).

I remembered a couple other useful references on the cache manager since
yesterday. The first is an OSR white paper on the cache manager which
has been in the IFS Kit since the Windows XP IFS Kit release. (Look
under the “Help | OSR White Papers” subfolders on the Start Menu where
your development kit is installed.) This focuses on the APIs
themselves. Mark Russinovich wrote a good article called “Inside the
Cache Manager” a few years back which gives a good architectural view of
how the cache manager works. You can find this article at
http://www.win2000mag.com/articles/print.cfm?ArticleID=3864. And of
course the “Inside Windows 2000” book has a good description as well.

Thanks,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Amritanshu
Sent: Thursday, March 18, 2004 11:03 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] PageFault after pinning the range.

Hello Molly,
I am back to bother you again I just needed to clarify one thing, in
your first reply you said that cache manager maps view aligned to 256K
boundary so does that mean that a view at offset =
256k-4096 of length 8192(i.e… a view spanning two boundaries) cannot be
mapped.
And this restriction will be there for MDL Interface as well right.
thanking you for your patience,
with best regards,
Johri.


“Molly Brown” wrote in message
news:xxxxx@ntfsd…
I can’t find where this fact is documented either :). I’ll file a bug
to get this point added to the documentation.

Yes, adding more asserts/bugchecks is a good idea, but that’s not going
to help you for Windows 2000 development. The file system samples which
ship as part of the IFS Kit will be your best reference. If you are
going to be writing a file system, you may want to look into taking the
file system class which OSR offers. I think you will be able to find
lots of people on this list who will say that it’s a helpful class.

Event the Microsoft drivers must live with this same limitation. The
BCB structure returned to the driver calling the Cc APIs are opaque to
everyone but the cache manager.

Thanks,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Amritanshu
Sent: Wednesday, March 17, 2004 10:34 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] PageFault after pinning the range.

Molly,
Thank you very much for replying I had also concluded that 0x40000 was
the biggest range that can be mapped consistently.
But I have a few questions
1.Maybe I am wrong but the IFS kit which I have does not mention it so
if there is another resource then do let me know.I have been looking
around for some information for some time now.
2.Secondly perhaps the call could have looked at the arguments and said
dude you are not doing the right thing therefore I fail the request by
returning a false.I tried it with checked build version of windows 2000
maybe an assert will help.But it doesn’t do that.So I concluded some of
the drivers from Microsoft can use the call for boundaries gr8er then
256 KB or something identical all they have to do is understand the
BCB.I am new to windows development but if this is the way things have
to go then we all will be interacting a lot ;).

with best regards,
Johri.
_______________________________
“Molly Brown” wrote in message
news:xxxxx@ntfsd…
The cache manager manages files in the system cache in 256K aligned
views. Pinned regions cannot span more than one 256K view, therefore,
you cannot pin 3-4MB at a time. The most you can pin is 256K starting
at a 256K-aligned offset in the file.

As a side note, using the CcPinXxx interface to map a range of a file
does not ensure that the pages remain resident in memory. While those
pages are pinned, you are guaranteed that the file range will stay
mapped into the system cache virtual address space, but the memory
manager can page out the physical pages as the system’s memory demand
requires.

Thanks,
Molly Brown
Microsoft Corporation

This posting is provided “AS IS” with no warranties and confers no
rights.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Amritanshu
Sent: Wednesday, March 17, 2004 5:21 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] PageFault after pinning the range.

Hello everyone,

I am using the pinning interface provided by cache manager to map a view
of a file.The problem is even after I have mapped an offset and pinned
it I still get a pagefault in the range while accessing it, this results
in a bugcheck PAGE_FAULT_IN_NON_PAGED_AREA.This happens when I am
pinning a range of 3-4 MB.If a smaller range 5-6KB is specified there
is no problem.The valid data length is equals the end of file.

One thing which I had observed when the crash happens is that the BCB
which we obtain contains a list of child BCB ,these child BCB
corresponds to the range,length & the corresponding virtual address(at
offset 0x70 I think),the virtual address for which the pagefault is
caused does not have an associated BCB,instead the Offset for which
pagefault is caused has been mapped to a different address.So is there
an upper limit to the range which can be mapped I am describing the
steps which I am doing to achieve it.
The steps I perform are in a system thread 1.Initiate caching for a file
using CcInitializeCacheMap.
2.MapOffset in the file say from 10244096 ,of length 4096 CcPinRead
with PIN_WAIT .
3.touch the pointer by a memcpy.
4.Unpin the BCB.
5.MapOffset in the file say from 0 ,of length 4096
1000 CcPinRead with
PIN_WAIT .
6.Touch the pointer obtained by a memset
((PCHAR)pBuffer,40961000);//there
are no overruns believe me.
Now,I start getting paging I/O read the read offset > 4096
1000 on the
device object associated a call stack shows that I have caused a
pagefault for a memory location well under (PCHAR)pBuffer+4096*1000
after some time we have a crash.

HELP!

Thank you for reading the mail.
regards,
Johri.


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com