Hi,
I am currently having problems getting CcMapData(…) to work and return any
valid data. I have debugged the code and the CcInitializeCacheMap(…)
appears to have initialized the cache map successfully…
Could anyone tell me if in order for the CcMapData(…) to work, do i need
to implement and initialise the FastIO entry points? or can i use CcMapData
without having fastio support?
Regards,
James…
Valid means TRUE , no ? If caching for the stream has been intialized, then
this routine can be called. This routine will return FALSE if the mapping
request cannot be performed imediately, and the Wait parameter is set to
FALSE. Set Wait to true, and see if you get the expected result.
Dan
----- Original Message -----
From: “James Dunning”
To: “File Systems Developers”
Sent: Thursday, July 11, 2002 3:43 PM
Subject: [ntfsd] Having problems getting CcMapData to work!..
> Hi,
>
> I am currently having problems getting CcMapData(…) to work and return
any
> valid data. I have debugged the code and the CcInitializeCacheMap(…)
> appears to have initialized the cache map successfully…
>
> Could anyone tell me if in order for the CcMapData(…) to work, do i need
> to implement and initialise the FastIO entry points? or can i use
CcMapData
> without having fastio support?
>
> Regards,
> James…
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to %%email.unsub%%
>
But can CcMapData be called without the use of FastIo?
The Wait parameter was actually already set to TRUE, but as i step over the
CcMapData(…) routine Softice seem to behave rather strangely and fails to
step to the next routine, it either exits from the softice debugger and goes
back to the main windows xp desktop, or to the original break point just
above the point where CcMapData(…) is actually called… either way, this
is preventing me from examining the buffer returned from CcMapData!
I am wondering if i have missed something out while setting up the cache
(like not initializing fastio) or whether there is a problem currupting the
memory somewhere, which is causing this bizzare behavour.
Any advice?
Regards,
James
-----Original Message-----
From: Dan Partelly [mailto:xxxxx@rdsor.ro]
Sent: 11 July 2002 14:12
To: File Systems Developers
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
Valid means TRUE , no ? If caching for the stream has been intialized, then
this routine can be called. This routine will return FALSE if the mapping
request cannot be performed imediately, and the Wait parameter is set to
FALSE. Set Wait to true, and see if you get the expected result.
Dan
----- Original Message -----
From: “James Dunning”
To: “File Systems Developers”
Sent: Thursday, July 11, 2002 3:43 PM
Subject: [ntfsd] Having problems getting CcMapData to work!..
> Hi,
>
> I am currently having problems getting CcMapData(…) to work and return
any
> valid data. I have debugged the code and the CcInitializeCacheMap(…)
> appears to have initialized the cache map successfully…
>
> Could anyone tell me if in order for the CcMapData(…) to work, do i need
> to implement and initialise the FastIO entry points? or can i use
CcMapData
> without having fastio support?
>
> Regards,
> James…
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to %%email.unsub%%
>
—
You are currently subscribed to ntfsd as:
xxxxx@generaldynamics.uk.com
To unsubscribe send a blank email to %%email.unsub%%
As far as I know, this routine has nothing to do with fast IO.
However , your debugger behaves strangley.
Practice told me that many Cc functions can raise exceptions so Id sugest to
have a try{} except{}
construct wrapping this call and see if you can catch an exception. If yes,
the status code
can be enlightning.
Dan
“James Dunning” wrote in message
news:xxxxx@ntfsd…
>
> But can CcMapData be called without the use of FastIo?
>
> The Wait parameter was actually already set to TRUE, but as i step over
the
> CcMapData(…) routine Softice seem to behave rather strangely and fails
to
> step to the next routine, it either exits from the softice debugger and
goes
> back to the main windows xp desktop, or to the original break point just
> above the point where CcMapData(…) is actually called… either way,
this
> is preventing me from examining the buffer returned from CcMapData!
>
> I am wondering if i have missed something out while setting up the cache
> (like not initializing fastio) or whether there is a problem currupting
the
> memory somewhere, which is causing this bizzare behavour.
>
> Any advice?
>
> Regards,
> James
>
>
> -----Original Message-----
> From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> Sent: 11 July 2002 14:12
> To: File Systems Developers
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> Valid means TRUE , no ? If caching for the stream has been intialized,
then
> this routine can be called. This routine will return FALSE if the mapping
> request cannot be performed imediately, and the Wait parameter is set to
> FALSE. Set Wait to true, and see if you get the expected result.
>
> Dan
>
> ----- Original Message -----
> From: “James Dunning”
> To: “File Systems Developers”
> Sent: Thursday, July 11, 2002 3:43 PM
> Subject: [ntfsd] Having problems getting CcMapData to work!..
>
>
> > Hi,
> >
> > I am currently having problems getting CcMapData(…) to work and return
> any
> > valid data. I have debugged the code and the CcInitializeCacheMap(…)
> > appears to have initialized the cache map successfully…
> >
> > Could anyone tell me if in order for the CcMapData(…) to work, do i
need
> > to implement and initialise the FastIO entry points? or can i use
> CcMapData
> > without having fastio support?
> >
> > Regards,
> > James…
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@generaldynamics.uk.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
>
Thanks for the reply…
I have just placed a try{} and except{} wrap around the call to
CcMapData(…) and you were right, I am getting an exception! I have managed
to get the exception code and it returning the following status code,
STATUS_IN_PAGE_ERROR 0xC0000006L
which I think means that for some reason, the data is not placed into memory
due to an io error?
Has anyone come across this problem before? and perhaps have an idea how to
get a better understanding why this error is recurring and resolve this
situation?
Regards
James
-----Original Message-----
From: Dan Partelly [mailto:xxxxx@rdsor.ro]
Sent: 11 July 2002 16:33
To: File Systems Developers
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
As far as I know, this routine has nothing to do with fast IO.
However , your debugger behaves strangley.
Practice told me that many Cc functions can raise exceptions so Id sugest to
have a try{} except{}
construct wrapping this call and see if you can catch an exception. If yes,
the status code
can be enlightning.
Dan
“James Dunning” wrote in message
news:xxxxx@ntfsd…
>
> But can CcMapData be called without the use of FastIo?
>
> The Wait parameter was actually already set to TRUE, but as i step over
the
> CcMapData(…) routine Softice seem to behave rather strangely and fails
to
> step to the next routine, it either exits from the softice debugger and
goes
> back to the main windows xp desktop, or to the original break point just
> above the point where CcMapData(…) is actually called… either way,
this
> is preventing me from examining the buffer returned from CcMapData!
>
> I am wondering if i have missed something out while setting up the cache
> (like not initializing fastio) or whether there is a problem currupting
the
> memory somewhere, which is causing this bizzare behavour.
>
> Any advice?
>
> Regards,
> James
>
>
> -----Original Message-----
> From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> Sent: 11 July 2002 14:12
> To: File Systems Developers
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> Valid means TRUE , no ? If caching for the stream has been intialized,
then
> this routine can be called. This routine will return FALSE if the mapping
> request cannot be performed imediately, and the Wait parameter is set to
> FALSE. Set Wait to true, and see if you get the expected result.
>
> Dan
>
> ----- Original Message -----
> From: “James Dunning”
> To: “File Systems Developers”
> Sent: Thursday, July 11, 2002 3:43 PM
> Subject: [ntfsd] Having problems getting CcMapData to work!..
>
>
> > Hi,
> >
> > I am currently having problems getting CcMapData(…) to work and return
> any
> > valid data. I have debugged the code and the CcInitializeCacheMap(…)
> > appears to have initialized the cache map successfully…
> >
> > Could anyone tell me if in order for the CcMapData(…) to work, do i
need
> > to implement and initialise the FastIO entry points? or can i use
> CcMapData
> > without having fastio support?
> >
> > Regards,
> > James…
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@generaldynamics.uk.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
>
—
You are currently subscribed to ntfsd as:
xxxxx@generaldynamics.uk.com
To unsubscribe send a blank email to %%email.unsub%%
STATUS_IN_PAGE_ERROR is returned in the case the page fault handler was not
able to read a swaped out memory page from the secondary storage. This can
be the result of a many causes , such as:
Malfunctioning HW , like bad sectors on your HDD
If you write a file system filter, check the paging IO path and make sure
the logic is error free.
If you write a file system, make sure you handle corectly paging IO logic
for both paging files, and memory mapped files.
For memory mapped files , unless the view is mapped aginst a section backed
by a system pagingfile, the inpage operations will take place directly from
the file backing up the section.
If you have your own filters on the storage stack, or you have your own
scsi miniports in the storage path, make sure
they are corectly written.
However , it should be easy to isolate the cause.
As a general rule , when you work with the cache manager, be aware that
almost all Cc functions can raise exceptions
on error conditions, altough this behaviour is not documented. The status
codes returned are very valuable information
to see what went wrong. Keep this in mind when you code with Cc, and adopt
defensive programming tactics. Employ
SEH to handle this kind of errors. Just as a side note, dont HIDE them
trough SEH(ugly aint it ?), but actualy handle the situation, one way or
another
Reagrds, Dan
----- Original Message -----
From: “James Dunning”
To: “File Systems Developers”
Sent: Friday, July 12, 2002 1:51 PM
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> Thanks for the reply…
>
> I have just placed a try{} and except{} wrap around the call to
> CcMapData(…) and you were right, I am getting an exception! I have
managed
> to get the exception code and it returning the following status code,
> STATUS_IN_PAGE_ERROR 0xC0000006L
>
> which I think means that for some reason, the data is not placed into
memory
> due to an io error?
>
> Has anyone come across this problem before? and perhaps have an idea how
to
> get a better understanding why this error is recurring and resolve this
> situation?
>
> Regards
> James
>
>
>
> -----Original Message-----
> From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> Sent: 11 July 2002 16:33
> To: File Systems Developers
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> As far as I know, this routine has nothing to do with fast IO.
>
> However , your debugger behaves strangley.
> Practice told me that many Cc functions can raise exceptions so Id sugest
to
> have a try{} except{}
> construct wrapping this call and see if you can catch an exception. If
yes,
> the status code
> can be enlightning.
>
> Dan
>
> “James Dunning” wrote in message
> news:xxxxx@ntfsd…
> >
> > But can CcMapData be called without the use of FastIo?
> >
> > The Wait parameter was actually already set to TRUE, but as i step over
> the
> > CcMapData(…) routine Softice seem to behave rather strangely and fails
> to
> > step to the next routine, it either exits from the softice debugger and
> goes
> > back to the main windows xp desktop, or to the original break point just
> > above the point where CcMapData(…) is actually called… either way,
> this
> > is preventing me from examining the buffer returned from CcMapData!
> >
> > I am wondering if i have missed something out while setting up the cache
> > (like not initializing fastio) or whether there is a problem currupting
> the
> > memory somewhere, which is causing this bizzare behavour.
> >
> > Any advice?
> >
> > Regards,
> > James
> >
> >
> > -----Original Message-----
> > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > Sent: 11 July 2002 14:12
> > To: File Systems Developers
> > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> >
> >
> > Valid means TRUE , no ? If caching for the stream has been intialized,
> then
> > this routine can be called. This routine will return FALSE if the
mapping
> > request cannot be performed imediately, and the Wait parameter is set to
> > FALSE. Set Wait to true, and see if you get the expected result.
> >
> > Dan
> >
> > ----- Original Message -----
> > From: “James Dunning”
> > To: “File Systems Developers”
> > Sent: Thursday, July 11, 2002 3:43 PM
> > Subject: [ntfsd] Having problems getting CcMapData to work!..
> >
> >
> > > Hi,
> > >
> > > I am currently having problems getting CcMapData(…) to work and
return
> > any
> > > valid data. I have debugged the code and the
CcInitializeCacheMap(…)
> > > appears to have initialized the cache map successfully…
> > >
> > > Could anyone tell me if in order for the CcMapData(…) to work, do i
> need
> > > to implement and initialise the FastIO entry points? or can i use
> > CcMapData
> > > without having fastio support?
> > >
> > > Regards,
> > > James…
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as:
> > xxxxx@generaldynamics.uk.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
> >
> >
>
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@generaldynamics.uk.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to %%email.unsub%%
>
STATUS_IN_PAGE_ERROR indicates that the read from disk failed for some
reason. This could be due to a hardware problem.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
Hope to see you at the next OSR file systems class in San Jose, CA September
16, 2002!
-----Original Message-----
From: James Dunning [mailto:xxxxx@generaldynamics.uk.com]
Sent: Friday, July 12, 2002 6:52 AM
To: File Systems Developers
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
Thanks for the reply…
I have just placed a try{} and except{} wrap around the call to
CcMapData(…) and you were right, I am getting an exception! I have managed
to get the exception code and it returning the following status code,
STATUS_IN_PAGE_ERROR 0xC0000006L
which I think means that for some reason, the data is not placed into memory
due to an io error?
Has anyone come across this problem before? and perhaps have an idea how to
get a better understanding why this error is recurring and resolve this
situation?
Regards
James
-----Original Message-----
From: Dan Partelly [mailto:xxxxx@rdsor.ro]
Sent: 11 July 2002 16:33
To: File Systems Developers
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
As far as I know, this routine has nothing to do with fast IO.
However , your debugger behaves strangley.
Practice told me that many Cc functions can raise exceptions so Id sugest to
have a try{} except{}
construct wrapping this call and see if you can catch an exception. If yes,
the status code
can be enlightning.
Dan
“James Dunning” wrote in message
news:xxxxx@ntfsd…
>
> But can CcMapData be called without the use of FastIo?
>
> The Wait parameter was actually already set to TRUE, but as i step over
the
> CcMapData(…) routine Softice seem to behave rather strangely and fails
to
> step to the next routine, it either exits from the softice debugger and
goes
> back to the main windows xp desktop, or to the original break point just
> above the point where CcMapData(…) is actually called… either way,
this
> is preventing me from examining the buffer returned from CcMapData!
>
> I am wondering if i have missed something out while setting up the cache
> (like not initializing fastio) or whether there is a problem currupting
the
> memory somewhere, which is causing this bizzare behavour.
>
> Any advice?
>
> Regards,
> James
>
>
> -----Original Message-----
> From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> Sent: 11 July 2002 14:12
> To: File Systems Developers
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> Valid means TRUE , no ? If caching for the stream has been intialized,
then
> this routine can be called. This routine will return FALSE if the mapping
> request cannot be performed imediately, and the Wait parameter is set to
> FALSE. Set Wait to true, and see if you get the expected result.
>
> Dan
>
> ----- Original Message -----
> From: “James Dunning”
> To: “File Systems Developers”
> Sent: Thursday, July 11, 2002 3:43 PM
> Subject: [ntfsd] Having problems getting CcMapData to work!..
>
>
> > Hi,
> >
> > I am currently having problems getting CcMapData(…) to work and return
> any
> > valid data. I have debugged the code and the CcInitializeCacheMap(…)
> > appears to have initialized the cache map successfully…
> >
> > Could anyone tell me if in order for the CcMapData(…) to work, do i
need
> > to implement and initialise the FastIO entry points? or can i use
> CcMapData
> > without having fastio support?
> >
> > Regards,
> > James…
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@generaldynamics.uk.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
>
—
You are currently subscribed to ntfsd as:
xxxxx@generaldynamics.uk.com
To unsubscribe send a blank email to %%email.unsub%%
—
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to %%email.unsub%%
Thanks again for the response!
Unfortunately i am still stumped as to why the CcMapData is throwing a
STATUS_IN_PAGE_ERROR. So I think would be best if i tried to explain what i
am trying to do.
I am currently writing a file system driver for a very simple file system,
and up until now i have been gutting the cdfs file system which came with
the Windows XP IFS kit. I am at a stage where i can now implement the IRP
dispatch routines which are required for my file system, I have decided to
start on the IRP_MJ_FILE_SYSTEM_CONTROL and implement the
IRP_MN_MOUNT_VOLUME request.
Now i would like to use the Cache Manager here so i can cache the volume’s
on disk structures. the primary volume block resides at block 1, which is
only 64 bytes. I have used InitializeVcb to set up a new stream
VirualVolume file object by calling IoCreateStreamFileObject and then called
ccInitializeCacheMap, configuring the cache map filesize to 64 bytes.
Now correct me if i am wrong, now that i have intialized the cache, i should
be able to call CcMapData which should page in the data from the disk?
As far as i am aware the hdd doesnt have any bad sectors, and has been
partitioned with 1 formatted partition with the necessary on disk structures
for my filesystem to pick up and identify. Incidentally I have managed to
read the on disk structures without the aid of the cache manager…
Regards
James
Ps. here’s a few snippets from my code…
// the following code is in the InitializeVcb routine called from
MountVolume…
RealDevice = Vcb->CurrentDevice;
Vcb->VirtualVolumeFile = UnwindFileObject =
IoCreateStreamFileObject( NULL, RealDevice );
StSetFileObject( Vcb->VirtualVolumeFile,
VirtualVolumeFile,
Vcb,
NULL );
Vcb->VirtualVolumeFile->SectionObjectPointer =
&Vcb->SectionObjectPointers;
Vcb->VirtualVolumeFile->ReadAccess = TRUE;
Vcb->VirtualVolumeFile->WriteAccess = TRUE;
Vcb->VirtualVolumeFile->DeleteAccess = TRUE;
FileSizes.AllocationSize.QuadPart =
FileSizes.FileSize.QuadPart = sizeof(DIRECTORY_BLOCK_FORMAT);
FileSizes.ValidDataLength = StMaxLarge;
CcInitializeCacheMap( Vcb->VirtualVolumeFile,
&FileSizes,
TRUE,
&StData.CacheManagerNoOpCallbacks,
Vcb );
// this is in the MountVolume routine
try {
StReadVolumeFile( IrpContext,
Vcb,
0, // Starting Byte
sizeof(DIRECTORY_BLOCK_FORMAT), // byte
count
&VolumeBcb,
(PVOID *)&VolumeBlock );
} except(EXCEPTION_EXECUTE_HANDLER) {
ExceptionCode = GetExceptionCode();
}
// this is basically called from within the StReadVolumeFile routine…
CcMapData( Vcb->VirtualVolumeFile,
&Vbo, // starting byte
ByteCount, // byte count
BooleanFlagOn(IrpContext->Flags, IRP_CONTEXT_FLAG_WAIT),
Bcb, // ref to volume Bcb
Buffer ) // ref to volume Block
-----Original Message-----
From: Dan Partelly [mailto:xxxxx@rdsor.ro]
Sent: 12 July 2002 12:39
To: File Systems Developers
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
STATUS_IN_PAGE_ERROR is returned in the case the page fault handler was not
able to read a swaped out memory page from the secondary storage. This can
be the result of a many causes , such as:
Malfunctioning HW , like bad sectors on your HDD
If you write a file system filter, check the paging IO path and make sure
the logic is error free.
If you write a file system, make sure you handle corectly paging IO logic
for both paging files, and memory mapped files.
For memory mapped files , unless the view is mapped aginst a section backed
by a system pagingfile, the inpage operations will take place directly from
the file backing up the section.
If you have your own filters on the storage stack, or you have your own
scsi miniports in the storage path, make sure
they are corectly written.
However , it should be easy to isolate the cause.
As a general rule , when you work with the cache manager, be aware that
almost all Cc functions can raise exceptions
on error conditions, altough this behaviour is not documented. The status
codes returned are very valuable information
to see what went wrong. Keep this in mind when you code with Cc, and adopt
defensive programming tactics. Employ
SEH to handle this kind of errors. Just as a side note, dont HIDE them
trough SEH(ugly aint it ?), but actualy handle the situation, one way or
another
Reagrds, Dan
----- Original Message -----
From: “James Dunning”
To: “File Systems Developers”
Sent: Friday, July 12, 2002 1:51 PM
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> Thanks for the reply…
>
> I have just placed a try{} and except{} wrap around the call to
> CcMapData(…) and you were right, I am getting an exception! I have
managed
> to get the exception code and it returning the following status code,
> STATUS_IN_PAGE_ERROR 0xC0000006L
>
> which I think means that for some reason, the data is not placed into
memory
> due to an io error?
>
> Has anyone come across this problem before? and perhaps have an idea how
to
> get a better understanding why this error is recurring and resolve this
> situation?
>
> Regards
> James
>
>
>
> -----Original Message-----
> From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> Sent: 11 July 2002 16:33
> To: File Systems Developers
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> As far as I know, this routine has nothing to do with fast IO.
>
> However , your debugger behaves strangley.
> Practice told me that many Cc functions can raise exceptions so Id sugest
to
> have a try{} except{}
> construct wrapping this call and see if you can catch an exception. If
yes,
> the status code
> can be enlightning.
>
> Dan
>
> “James Dunning” wrote in message
> news:xxxxx@ntfsd…
> >
> > But can CcMapData be called without the use of FastIo?
> >
> > The Wait parameter was actually already set to TRUE, but as i step over
> the
> > CcMapData(…) routine Softice seem to behave rather strangely and fails
> to
> > step to the next routine, it either exits from the softice debugger and
> goes
> > back to the main windows xp desktop, or to the original break point just
> > above the point where CcMapData(…) is actually called… either way,
> this
> > is preventing me from examining the buffer returned from CcMapData!
> >
> > I am wondering if i have missed something out while setting up the cache
> > (like not initializing fastio) or whether there is a problem currupting
> the
> > memory somewhere, which is causing this bizzare behavour.
> >
> > Any advice?
> >
> > Regards,
> > James
> >
> >
> > -----Original Message-----
> > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > Sent: 11 July 2002 14:12
> > To: File Systems Developers
> > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> >
> >
> > Valid means TRUE , no ? If caching for the stream has been intialized,
> then
> > this routine can be called. This routine will return FALSE if the
mapping
> > request cannot be performed imediately, and the Wait parameter is set to
> > FALSE. Set Wait to true, and see if you get the expected result.
> >
> > Dan
> >
> > ----- Original Message -----
> > From: “James Dunning”
> > To: “File Systems Developers”
> > Sent: Thursday, July 11, 2002 3:43 PM
> > Subject: [ntfsd] Having problems getting CcMapData to work!..
> >
> >
> > > Hi,
> > >
> > > I am currently having problems getting CcMapData(…) to work and
return
> > any
> > > valid data. I have debugged the code and the
CcInitializeCacheMap(…)
> > > appears to have initialized the cache map successfully…
> > >
> > > Could anyone tell me if in order for the CcMapData(…) to work, do i
> need
> > > to implement and initialise the FastIO entry points? or can i use
> > CcMapData
> > > without having fastio support?
> > >
> > > Regards,
> > > James…
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as:
> > xxxxx@generaldynamics.uk.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
> >
> >
>
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@generaldynamics.uk.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to %%email.unsub%%
>
—
You are currently subscribed to ntfsd as:
xxxxx@generaldynamics.uk.com
To unsubscribe send a blank email to %%email.unsub%%
If your FS driver is not prepared to bring data back from secondary sotrage
(you should properly handle
IRP_MJ_READ , with Irp->Flags IRP_NOCACHE | IRP_PAGING_IO ) your file
system will not be able
to satisfy ANY inpage operations. That means, no cache support and no memory
mapped files support.
Donno if you handle this situation properly so far, for you say you started
with FsControl IRP. But if you build it upon an already made skeleton ,
maybe you have it … Check if you do support paging IO properly in your
file system. This support must be functional and up before you attmept to
use the Cc.
Dan
----- Original Message -----
From: “James Dunning”
To: “File Systems Developers”
Sent: Monday, July 15, 2002 4:45 PM
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> Thanks again for the response!
>
> Unfortunately i am still stumped as to why the CcMapData is throwing a
> STATUS_IN_PAGE_ERROR. So I think would be best if i tried to explain what
i
> am trying to do.
>
> I am currently writing a file system driver for a very simple file system,
> and up until now i have been gutting the cdfs file system which came with
> the Windows XP IFS kit. I am at a stage where i can now implement the IRP
> dispatch routines which are required for my file system, I have decided to
> start on the IRP_MJ_FILE_SYSTEM_CONTROL and implement the
> IRP_MN_MOUNT_VOLUME request.
>
> Now i would like to use the Cache Manager here so i can cache the volume’s
> on disk structures. the primary volume block resides at block 1, which is
> only 64 bytes. I have used InitializeVcb to set up a new stream
> VirualVolume file object by calling IoCreateStreamFileObject and then
called
> ccInitializeCacheMap, configuring the cache map filesize to 64 bytes.
>
> Now correct me if i am wrong, now that i have intialized the cache, i
should
> be able to call CcMapData which should page in the data from the disk?
>
> As far as i am aware the hdd doesnt have any bad sectors, and has been
> partitioned with 1 formatted partition with the necessary on disk
structures
> for my filesystem to pick up and identify. Incidentally I have managed to
> read the on disk structures without the aid of the cache manager…
>
> Regards
> James
>
> Ps. here’s a few snippets from my code…
>
> // the following code is in the InitializeVcb routine called from
> MountVolume…
>
> RealDevice = Vcb->CurrentDevice;
>
> Vcb->VirtualVolumeFile = UnwindFileObject =
> IoCreateStreamFileObject( NULL, RealDevice );
>
> StSetFileObject( Vcb->VirtualVolumeFile,
> VirtualVolumeFile,
> Vcb,
> NULL );
>
> Vcb->VirtualVolumeFile->SectionObjectPointer =
> &Vcb->SectionObjectPointers;
>
> Vcb->VirtualVolumeFile->ReadAccess = TRUE;
> Vcb->VirtualVolumeFile->WriteAccess = TRUE;
> Vcb->VirtualVolumeFile->DeleteAccess = TRUE;
>
> FileSizes.AllocationSize.QuadPart =
> FileSizes.FileSize.QuadPart = sizeof(DIRECTORY_BLOCK_FORMAT);
> FileSizes.ValidDataLength = StMaxLarge;
>
> CcInitializeCacheMap( Vcb->VirtualVolumeFile,
> &FileSizes,
> TRUE,
> &StData.CacheManagerNoOpCallbacks,
> Vcb );
>
> // this is in the MountVolume routine
> try {
> StReadVolumeFile( IrpContext,
> Vcb,
> 0, // Starting Byte
> sizeof(DIRECTORY_BLOCK_FORMAT), // byte
> count
> &VolumeBcb,
> (PVOID *)&VolumeBlock );
>
> } except(EXCEPTION_EXECUTE_HANDLER) {
> ExceptionCode = GetExceptionCode();
> }
>
>
> // this is basically called from within the StReadVolumeFile routine…
>
> CcMapData( Vcb->VirtualVolumeFile,
> &Vbo, // starting byte
> ByteCount, // byte count
> BooleanFlagOn(IrpContext->Flags, IRP_CONTEXT_FLAG_WAIT),
> Bcb, // ref to volume Bcb
> Buffer ) // ref to volume Block
>
>
>
> -----Original Message-----
> From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> Sent: 12 July 2002 12:39
> To: File Systems Developers
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> STATUS_IN_PAGE_ERROR is returned in the case the page fault handler was
not
> able to read a swaped out memory page from the secondary storage. This can
> be the result of a many causes , such as:
>
> 1. Malfunctioning HW , like bad sectors on your HDD
> 2. If you write a file system filter, check the paging IO path and make
sure
> the logic is error free.
> 3. If you write a file system, make sure you handle corectly paging IO
logic
> for both paging files, and memory mapped files.
> For memory mapped files , unless the view is mapped aginst a section
backed
> by a system pagingfile, the inpage operations will take place directly
from
> the file backing up the section.
>
> 4. If you have your own filters on the storage stack, or you have your own
> scsi miniports in the storage path, make sure
> they are corectly written.
>
> However , it should be easy to isolate the cause.
>
> As a general rule , when you work with the cache manager, be aware that
> almost all Cc functions can raise exceptions
> on error conditions, altough this behaviour is not documented. The status
> codes returned are very valuable information
> to see what went wrong. Keep this in mind when you code with Cc, and adopt
> defensive programming tactics. Employ
> SEH to handle this kind of errors. Just as a side note, dont HIDE them
> trough SEH(ugly aint it ?), but actualy handle the situation, one way or
> another
>
> Reagrds, Dan
>
>
>
>
> ----- Original Message -----
> From: “James Dunning”
> To: “File Systems Developers”
> Sent: Friday, July 12, 2002 1:51 PM
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> > Thanks for the reply…
> >
> > I have just placed a try{} and except{} wrap around the call to
> > CcMapData(…) and you were right, I am getting an exception! I have
> managed
> > to get the exception code and it returning the following status code,
> > STATUS_IN_PAGE_ERROR 0xC0000006L
> >
> > which I think means that for some reason, the data is not placed into
> memory
> > due to an io error?
> >
> > Has anyone come across this problem before? and perhaps have an idea how
> to
> > get a better understanding why this error is recurring and resolve this
> > situation?
> >
> > Regards
> > James
> >
> >
> >
> > -----Original Message-----
> > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > Sent: 11 July 2002 16:33
> > To: File Systems Developers
> > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> >
> >
> > As far as I know, this routine has nothing to do with fast IO.
> >
> > However , your debugger behaves strangley.
> > Practice told me that many Cc functions can raise exceptions so Id
sugest
> to
> > have a try{} except{}
> > construct wrapping this call and see if you can catch an exception. If
> yes,
> > the status code
> > can be enlightning.
> >
> > Dan
> >
> > “James Dunning” wrote in message
> > news:xxxxx@ntfsd…
> > >
> > > But can CcMapData be called without the use of FastIo?
> > >
> > > The Wait parameter was actually already set to TRUE, but as i step
over
> > the
> > > CcMapData(…) routine Softice seem to behave rather strangely and
fails
> > to
> > > step to the next routine, it either exits from the softice debugger
and
> > goes
> > > back to the main windows xp desktop, or to the original break point
just
> > > above the point where CcMapData(…) is actually called… either
way,
> > this
> > > is preventing me from examining the buffer returned from CcMapData!
> > >
> > > I am wondering if i have missed something out while setting up the
cache
> > > (like not initializing fastio) or whether there is a problem
currupting
> > the
> > > memory somewhere, which is causing this bizzare behavour.
> > >
> > > Any advice?
> > >
> > > Regards,
> > > James
> > >
> > >
> > > -----Original Message-----
> > > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > > Sent: 11 July 2002 14:12
> > > To: File Systems Developers
> > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > >
> > >
> > > Valid means TRUE , no ? If caching for the stream has been
intialized,
> > then
> > > this routine can be called. This routine will return FALSE if the
> mapping
> > > request cannot be performed imediately, and the Wait parameter is set
to
> > > FALSE. Set Wait to true, and see if you get the expected result.
> > >
> > > Dan
> > >
> > > ----- Original Message -----
> > > From: “James Dunning”
> > > To: “File Systems Developers”
> > > Sent: Thursday, July 11, 2002 3:43 PM
> > > Subject: [ntfsd] Having problems getting CcMapData to work!..
> > >
> > >
> > > > Hi,
> > > >
> > > > I am currently having problems getting CcMapData(…) to work and
> return
> > > any
> > > > valid data. I have debugged the code and the
> CcInitializeCacheMap(…)
> > > > appears to have initialized the cache map successfully…
> > > >
> > > > Could anyone tell me if in order for the CcMapData(…) to work, do
i
> > need
> > > > to implement and initialise the FastIO entry points? or can i use
> > > CcMapData
> > > > without having fastio support?
> > > >
> > > > Regards,
> > > > James…
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > > To unsubscribe send a blank email to %%email.unsub%%
> > > >
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as:
> > > xxxxx@generaldynamics.uk.com
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> > >
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as:
> > xxxxx@generaldynamics.uk.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@generaldynamics.uk.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to %%email.unsub%%
>
Ohhh Right! thats great cheers… I didnt realise that the IRP_MJ_READ and i
guess the IRP_MJ_CLEANUP needed to be implemented within the file system and
with the support for paging IO!
I have quickly placed some breakpoints with in the fsdDispatch routine, and
i have noticed that when CcMapData is called, an IRP_MJ_READ request is
generated, but the routine to handle the IRP_MJ_READ has been gutted… i
guess i have over gutted the filesystem! lol! nevermind, i guess i can add
the routines back as and when i require them! I just assumed that CcMapData
called the device below the fs and did all the work for me!
Thanks again for your help.
James
-----Original Message-----
From: Dan Partelly [mailto:xxxxx@rdsor.ro]
Sent: 15 July 2002 14:55
To: File Systems Developers
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
If your FS driver is not prepared to bring data back from secondary sotrage
(you should properly handle
IRP_MJ_READ , with Irp->Flags IRP_NOCACHE | IRP_PAGING_IO ) your file
system will not be able
to satisfy ANY inpage operations. That means, no cache support and no memory
mapped files support.
Donno if you handle this situation properly so far, for you say you started
with FsControl IRP. But if you build it upon an already made skeleton ,
maybe you have it … Check if you do support paging IO properly in your
file system. This support must be functional and up before you attmept to
use the Cc.
Dan
----- Original Message -----
From: “James Dunning”
To: “File Systems Developers”
Sent: Monday, July 15, 2002 4:45 PM
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> Thanks again for the response!
>
> Unfortunately i am still stumped as to why the CcMapData is throwing a
> STATUS_IN_PAGE_ERROR. So I think would be best if i tried to explain what
i
> am trying to do.
>
> I am currently writing a file system driver for a very simple file system,
> and up until now i have been gutting the cdfs file system which came with
> the Windows XP IFS kit. I am at a stage where i can now implement the IRP
> dispatch routines which are required for my file system, I have decided to
> start on the IRP_MJ_FILE_SYSTEM_CONTROL and implement the
> IRP_MN_MOUNT_VOLUME request.
>
> Now i would like to use the Cache Manager here so i can cache the volume’s
> on disk structures. the primary volume block resides at block 1, which is
> only 64 bytes. I have used InitializeVcb to set up a new stream
> VirualVolume file object by calling IoCreateStreamFileObject and then
called
> ccInitializeCacheMap, configuring the cache map filesize to 64 bytes.
>
> Now correct me if i am wrong, now that i have intialized the cache, i
should
> be able to call CcMapData which should page in the data from the disk?
>
> As far as i am aware the hdd doesnt have any bad sectors, and has been
> partitioned with 1 formatted partition with the necessary on disk
structures
> for my filesystem to pick up and identify. Incidentally I have managed to
> read the on disk structures without the aid of the cache manager…
>
> Regards
> James
>
> Ps. here’s a few snippets from my code…
>
> // the following code is in the InitializeVcb routine called from
> MountVolume…
>
> RealDevice = Vcb->CurrentDevice;
>
> Vcb->VirtualVolumeFile = UnwindFileObject =
> IoCreateStreamFileObject( NULL, RealDevice );
>
> StSetFileObject( Vcb->VirtualVolumeFile,
> VirtualVolumeFile,
> Vcb,
> NULL );
>
> Vcb->VirtualVolumeFile->SectionObjectPointer =
> &Vcb->SectionObjectPointers;
>
> Vcb->VirtualVolumeFile->ReadAccess = TRUE;
> Vcb->VirtualVolumeFile->WriteAccess = TRUE;
> Vcb->VirtualVolumeFile->DeleteAccess = TRUE;
>
> FileSizes.AllocationSize.QuadPart =
> FileSizes.FileSize.QuadPart = sizeof(DIRECTORY_BLOCK_FORMAT);
> FileSizes.ValidDataLength = StMaxLarge;
>
> CcInitializeCacheMap( Vcb->VirtualVolumeFile,
> &FileSizes,
> TRUE,
> &StData.CacheManagerNoOpCallbacks,
> Vcb );
>
> // this is in the MountVolume routine
> try {
> StReadVolumeFile( IrpContext,
> Vcb,
> 0, // Starting Byte
> sizeof(DIRECTORY_BLOCK_FORMAT), // byte
> count
> &VolumeBcb,
> (PVOID *)&VolumeBlock );
>
> } except(EXCEPTION_EXECUTE_HANDLER) {
> ExceptionCode = GetExceptionCode();
> }
>
>
> // this is basically called from within the StReadVolumeFile routine…
>
> CcMapData( Vcb->VirtualVolumeFile,
> &Vbo, // starting byte
> ByteCount, // byte count
> BooleanFlagOn(IrpContext->Flags, IRP_CONTEXT_FLAG_WAIT),
> Bcb, // ref to volume Bcb
> Buffer ) // ref to volume Block
>
>
>
> -----Original Message-----
> From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> Sent: 12 July 2002 12:39
> To: File Systems Developers
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> STATUS_IN_PAGE_ERROR is returned in the case the page fault handler was
not
> able to read a swaped out memory page from the secondary storage. This can
> be the result of a many causes , such as:
>
> 1. Malfunctioning HW , like bad sectors on your HDD
> 2. If you write a file system filter, check the paging IO path and make
sure
> the logic is error free.
> 3. If you write a file system, make sure you handle corectly paging IO
logic
> for both paging files, and memory mapped files.
> For memory mapped files , unless the view is mapped aginst a section
backed
> by a system pagingfile, the inpage operations will take place directly
from
> the file backing up the section.
>
> 4. If you have your own filters on the storage stack, or you have your own
> scsi miniports in the storage path, make sure
> they are corectly written.
>
> However , it should be easy to isolate the cause.
>
> As a general rule , when you work with the cache manager, be aware that
> almost all Cc functions can raise exceptions
> on error conditions, altough this behaviour is not documented. The status
> codes returned are very valuable information
> to see what went wrong. Keep this in mind when you code with Cc, and adopt
> defensive programming tactics. Employ
> SEH to handle this kind of errors. Just as a side note, dont HIDE them
> trough SEH(ugly aint it ?), but actualy handle the situation, one way or
> another
>
> Reagrds, Dan
>
>
>
>
> ----- Original Message -----
> From: “James Dunning”
> To: “File Systems Developers”
> Sent: Friday, July 12, 2002 1:51 PM
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> > Thanks for the reply…
> >
> > I have just placed a try{} and except{} wrap around the call to
> > CcMapData(…) and you were right, I am getting an exception! I have
> managed
> > to get the exception code and it returning the following status code,
> > STATUS_IN_PAGE_ERROR 0xC0000006L
> >
> > which I think means that for some reason, the data is not placed into
> memory
> > due to an io error?
> >
> > Has anyone come across this problem before? and perhaps have an idea how
> to
> > get a better understanding why this error is recurring and resolve this
> > situation?
> >
> > Regards
> > James
> >
> >
> >
> > -----Original Message-----
> > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > Sent: 11 July 2002 16:33
> > To: File Systems Developers
> > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> >
> >
> > As far as I know, this routine has nothing to do with fast IO.
> >
> > However , your debugger behaves strangley.
> > Practice told me that many Cc functions can raise exceptions so Id
sugest
> to
> > have a try{} except{}
> > construct wrapping this call and see if you can catch an exception. If
> yes,
> > the status code
> > can be enlightning.
> >
> > Dan
> >
> > “James Dunning” wrote in message
> > news:xxxxx@ntfsd…
> > >
> > > But can CcMapData be called without the use of FastIo?
> > >
> > > The Wait parameter was actually already set to TRUE, but as i step
over
> > the
> > > CcMapData(…) routine Softice seem to behave rather strangely and
fails
> > to
> > > step to the next routine, it either exits from the softice debugger
and
> > goes
> > > back to the main windows xp desktop, or to the original break point
just
> > > above the point where CcMapData(…) is actually called… either
way,
> > this
> > > is preventing me from examining the buffer returned from CcMapData!
> > >
> > > I am wondering if i have missed something out while setting up the
cache
> > > (like not initializing fastio) or whether there is a problem
currupting
> > the
> > > memory somewhere, which is causing this bizzare behavour.
> > >
> > > Any advice?
> > >
> > > Regards,
> > > James
> > >
> > >
> > > -----Original Message-----
> > > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > > Sent: 11 July 2002 14:12
> > > To: File Systems Developers
> > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > >
> > >
> > > Valid means TRUE , no ? If caching for the stream has been
intialized,
> > then
> > > this routine can be called. This routine will return FALSE if the
> mapping
> > > request cannot be performed imediately, and the Wait parameter is set
to
> > > FALSE. Set Wait to true, and see if you get the expected result.
> > >
> > > Dan
> > >
> > > ----- Original Message -----
> > > From: “James Dunning”
> > > To: “File Systems Developers”
> > > Sent: Thursday, July 11, 2002 3:43 PM
> > > Subject: [ntfsd] Having problems getting CcMapData to work!..
> > >
> > >
> > > > Hi,
> > > >
> > > > I am currently having problems getting CcMapData(…) to work and
> return
> > > any
> > > > valid data. I have debugged the code and the
> CcInitializeCacheMap(…)
> > > > appears to have initialized the cache map successfully…
> > > >
> > > > Could anyone tell me if in order for the CcMapData(…) to work, do
i
> > need
> > > > to implement and initialise the FastIO entry points? or can i use
> > > CcMapData
> > > > without having fastio support?
> > > >
> > > > Regards,
> > > > James…
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > > To unsubscribe send a blank email to %%email.unsub%%
> > > >
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as:
> > > xxxxx@generaldynamics.uk.com
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> > >
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as:
> > xxxxx@generaldynamics.uk.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@generaldynamics.uk.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to %%email.unsub%%
>
—
You are currently subscribed to ntfsd as:
xxxxx@generaldynamics.uk.com
To unsubscribe send a blank email to %%email.unsub%%
>> I just assumed that CcMapData called the device below the fs and did all
the work for me!
Assumption is mother of all f…ups. Besides , how could Cc manager know
which sectors to read from the
storage stack ? This is FS business. Maybe you should read a little more
about file systems, Cc manager and
brothers. Try Rajeev Nagar’s book.
“James Dunning” wrote in message
news:xxxxx@ntfsd…
>
> Ohhh Right! thats great cheers… I didnt realise that the IRP_MJ_READ and
i
> guess the IRP_MJ_CLEANUP needed to be implemented within the file system
and
> with the support for paging IO!
>
> I have quickly placed some breakpoints with in the fsdDispatch routine,
and
> i have noticed that when CcMapData is called, an IRP_MJ_READ request is
> generated, but the routine to handle the IRP_MJ_READ has been gutted… i
> guess i have over gutted the filesystem! lol! nevermind, i guess i can add
> the routines back as and when i require them! I just assumed that
CcMapData
> called the device below the fs and did all the work for me!
>
> Thanks again for your help.
>
> James
>
> -----Original Message-----
> From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> Sent: 15 July 2002 14:55
> To: File Systems Developers
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> If your FS driver is not prepared to bring data back from secondary
sotrage
> (you should properly handle
> IRP_MJ_READ , with Irp->Flags IRP_NOCACHE | IRP_PAGING_IO ) your file
> system will not be able
> to satisfy ANY inpage operations. That means, no cache support and no
memory
> mapped files support.
> Donno if you handle this situation properly so far, for you say you
started
> with FsControl IRP. But if you build it upon an already made skeleton ,
> maybe you have it … Check if you do support paging IO properly in your
> file system. This support must be functional and up before you attmept to
> use the Cc.
>
> Dan
>
>
>
>
>
>
>
> ----- Original Message -----
> From: “James Dunning”
> To: “File Systems Developers”
> Sent: Monday, July 15, 2002 4:45 PM
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> > Thanks again for the response!
> >
> > Unfortunately i am still stumped as to why the CcMapData is throwing a
> > STATUS_IN_PAGE_ERROR. So I think would be best if i tried to explain
what
> i
> > am trying to do.
> >
> > I am currently writing a file system driver for a very simple file
system,
> > and up until now i have been gutting the cdfs file system which came
with
> > the Windows XP IFS kit. I am at a stage where i can now implement the
IRP
> > dispatch routines which are required for my file system, I have decided
to
> > start on the IRP_MJ_FILE_SYSTEM_CONTROL and implement the
> > IRP_MN_MOUNT_VOLUME request.
> >
> > Now i would like to use the Cache Manager here so i can cache the
volume’s
> > on disk structures. the primary volume block resides at block 1, which
is
> > only 64 bytes. I have used InitializeVcb to set up a new stream
> > VirualVolume file object by calling IoCreateStreamFileObject and then
> called
> > ccInitializeCacheMap, configuring the cache map filesize to 64 bytes.
> >
> > Now correct me if i am wrong, now that i have intialized the cache, i
> should
> > be able to call CcMapData which should page in the data from the disk?
> >
> > As far as i am aware the hdd doesnt have any bad sectors, and has been
> > partitioned with 1 formatted partition with the necessary on disk
> structures
> > for my filesystem to pick up and identify. Incidentally I have managed
to
> > read the on disk structures without the aid of the cache manager…
> >
> > Regards
> > James
> >
> > Ps. here’s a few snippets from my code…
> >
> > // the following code is in the InitializeVcb routine called from
> > MountVolume…
> >
> > RealDevice = Vcb->CurrentDevice;
> >
> > Vcb->VirtualVolumeFile = UnwindFileObject =
> > IoCreateStreamFileObject( NULL, RealDevice );
> >
> > StSetFileObject( Vcb->VirtualVolumeFile,
> > VirtualVolumeFile,
> > Vcb,
> > NULL );
> >
> > Vcb->VirtualVolumeFile->SectionObjectPointer =
> > &Vcb->SectionObjectPointers;
> >
> > Vcb->VirtualVolumeFile->ReadAccess = TRUE;
> > Vcb->VirtualVolumeFile->WriteAccess = TRUE;
> > Vcb->VirtualVolumeFile->DeleteAccess = TRUE;
> >
> > FileSizes.AllocationSize.QuadPart =
> > FileSizes.FileSize.QuadPart = sizeof(DIRECTORY_BLOCK_FORMAT);
> > FileSizes.ValidDataLength = StMaxLarge;
> >
> > CcInitializeCacheMap( Vcb->VirtualVolumeFile,
> > &FileSizes,
> > TRUE,
> > &StData.CacheManagerNoOpCallbacks,
> > Vcb );
> >
> > // this is in the MountVolume routine
> > try {
> > StReadVolumeFile( IrpContext,
> > Vcb,
> > 0, // Starting Byte
> > sizeof(DIRECTORY_BLOCK_FORMAT), // byte
> > count
> > &VolumeBcb,
> > (PVOID *)&VolumeBlock );
> >
> > } except(EXCEPTION_EXECUTE_HANDLER) {
> > ExceptionCode = GetExceptionCode();
> > }
> >
> >
> > // this is basically called from within the StReadVolumeFile routine…
> >
> > CcMapData( Vcb->VirtualVolumeFile,
> > &Vbo, // starting byte
> > ByteCount, // byte count
> > BooleanFlagOn(IrpContext->Flags, IRP_CONTEXT_FLAG_WAIT),
> > Bcb, // ref to volume Bcb
> > Buffer ) // ref to volume Block
> >
> >
> >
> > -----Original Message-----
> > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > Sent: 12 July 2002 12:39
> > To: File Systems Developers
> > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> >
> >
> > STATUS_IN_PAGE_ERROR is returned in the case the page fault handler was
> not
> > able to read a swaped out memory page from the secondary storage. This
can
> > be the result of a many causes , such as:
> >
> > 1. Malfunctioning HW , like bad sectors on your HDD
> > 2. If you write a file system filter, check the paging IO path and make
> sure
> > the logic is error free.
> > 3. If you write a file system, make sure you handle corectly paging IO
> logic
> > for both paging files, and memory mapped files.
> > For memory mapped files , unless the view is mapped aginst a section
> backed
> > by a system pagingfile, the inpage operations will take place directly
> from
> > the file backing up the section.
> >
> > 4. If you have your own filters on the storage stack, or you have your
own
> > scsi miniports in the storage path, make sure
> > they are corectly written.
> >
> > However , it should be easy to isolate the cause.
> >
> > As a general rule , when you work with the cache manager, be aware that
> > almost all Cc functions can raise exceptions
> > on error conditions, altough this behaviour is not documented. The
status
> > codes returned are very valuable information
> > to see what went wrong. Keep this in mind when you code with Cc, and
adopt
> > defensive programming tactics. Employ
> > SEH to handle this kind of errors. Just as a side note, dont HIDE them
> > trough SEH(ugly aint it ?), but actualy handle the situation, one way or
> > another
> >
> > Reagrds, Dan
> >
> >
> >
> >
> > ----- Original Message -----
> > From: “James Dunning”
> > To: “File Systems Developers”
> > Sent: Friday, July 12, 2002 1:51 PM
> > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> >
> >
> > > Thanks for the reply…
> > >
> > > I have just placed a try{} and except{} wrap around the call to
> > > CcMapData(…) and you were right, I am getting an exception! I have
> > managed
> > > to get the exception code and it returning the following status code,
> > > STATUS_IN_PAGE_ERROR 0xC0000006L
> > >
> > > which I think means that for some reason, the data is not placed into
> > memory
> > > due to an io error?
> > >
> > > Has anyone come across this problem before? and perhaps have an idea
how
> > to
> > > get a better understanding why this error is recurring and resolve
this
> > > situation?
> > >
> > > Regards
> > > James
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > > Sent: 11 July 2002 16:33
> > > To: File Systems Developers
> > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > >
> > >
> > > As far as I know, this routine has nothing to do with fast IO.
> > >
> > > However , your debugger behaves strangley.
> > > Practice told me that many Cc functions can raise exceptions so Id
> sugest
> > to
> > > have a try{} except{}
> > > construct wrapping this call and see if you can catch an exception.
If
> > yes,
> > > the status code
> > > can be enlightning.
> > >
> > > Dan
> > >
> > > “James Dunning” wrote in
message
> > > news:xxxxx@ntfsd…
> > > >
> > > > But can CcMapData be called without the use of FastIo?
> > > >
> > > > The Wait parameter was actually already set to TRUE, but as i step
> over
> > > the
> > > > CcMapData(…) routine Softice seem to behave rather strangely and
> fails
> > > to
> > > > step to the next routine, it either exits from the softice debugger
> and
> > > goes
> > > > back to the main windows xp desktop, or to the original break point
> just
> > > > above the point where CcMapData(…) is actually called… either
> way,
> > > this
> > > > is preventing me from examining the buffer returned from CcMapData!
> > > >
> > > > I am wondering if i have missed something out while setting up the
> cache
> > > > (like not initializing fastio) or whether there is a problem
> currupting
> > > the
> > > > memory somewhere, which is causing this bizzare behavour.
> > > >
> > > > Any advice?
> > > >
> > > > Regards,
> > > > James
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > > > Sent: 11 July 2002 14:12
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > > >
> > > >
> > > > Valid means TRUE , no ? If caching for the stream has been
> intialized,
> > > then
> > > > this routine can be called. This routine will return FALSE if the
> > mapping
> > > > request cannot be performed imediately, and the Wait parameter is
set
> to
> > > > FALSE. Set Wait to true, and see if you get the expected result.
> > > >
> > > > Dan
> > > >
> > > > ----- Original Message -----
> > > > From: “James Dunning”
> > > > To: “File Systems Developers”
> > > > Sent: Thursday, July 11, 2002 3:43 PM
> > > > Subject: [ntfsd] Having problems getting CcMapData to work!..
> > > >
> > > >
> > > > > Hi,
> > > > >
> > > > > I am currently having problems getting CcMapData(…) to work and
> > return
> > > > any
> > > > > valid data. I have debugged the code and the
> > CcInitializeCacheMap(…)
> > > > > appears to have initialized the cache map successfully…
> > > > >
> > > > > Could anyone tell me if in order for the CcMapData(…) to work,
do
> i
> > > need
> > > > > to implement and initialise the FastIO entry points? or can i use
> > > > CcMapData
> > > > > without having fastio support?
> > > > >
> > > > > Regards,
> > > > > James…
> > > > >
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > > > To unsubscribe send a blank email to %%email.unsub%%
> > > > >
> > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as:
> > > > xxxxx@generaldynamics.uk.com
> > > > To unsubscribe send a blank email to %%email.unsub%%
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as:
> > > xxxxx@generaldynamics.uk.com
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as:
> > xxxxx@generaldynamics.uk.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@generaldynamics.uk.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
>
Hi Dan,
I would just like to quickly make sure that i have understood you
correctly…
In the previous reply, you mentioned that i should properly handle
IRP_MJ_READ, with Irp->Flags IRP_NOCACHE|IRP_PAGING_IO in my file system, in
order to satisfy ANY inpage operations.
Now if i use CcMapData, should the following flags IRP_NOCACHE (true) and
IRP_PAGING_IO (true) be set within the IRP_MJ_READ dispatch routine?
BOOLEAN PagingIo = BooleanFlagOn(Irp->Flags, IRP_PAGING_IO);
BOOLEAN NonCachedIo = BooleanFlagOn(Irp->Flags,IRP_NOCACHE);
The reason why i ask is because im getting confused why NonCacheIo should be
set to true, if CcMapData is one of the IFS cache routines. Surely that
NonCacheIo should be set to FALSE and PagingIo should be set to TRUE?
Regards,
James
-----Original Message-----
From: Dan Partelly [mailto:xxxxx@rdsor.ro]
Sent: 15 July 2002 15:55
To: File Systems Developers
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> I just assumed that CcMapData called the device below the fs and did all
the work for me!
Assumption is mother of all f…ups. Besides , how could Cc manager know
which sectors to read from the
storage stack ? This is FS business. Maybe you should read a little more
about file systems, Cc manager and
brothers. Try Rajeev Nagar’s book.
“James Dunning” wrote in message
news:xxxxx@ntfsd…
>
> Ohhh Right! thats great cheers… I didnt realise that the IRP_MJ_READ and
i
> guess the IRP_MJ_CLEANUP needed to be implemented within the file system
and
> with the support for paging IO!
>
> I have quickly placed some breakpoints with in the fsdDispatch routine,
and
> i have noticed that when CcMapData is called, an IRP_MJ_READ request is
> generated, but the routine to handle the IRP_MJ_READ has been gutted… i
> guess i have over gutted the filesystem! lol! nevermind, i guess i can add
> the routines back as and when i require them! I just assumed that
CcMapData
> called the device below the fs and did all the work for me!
>
> Thanks again for your help.
>
> James
>
> -----Original Message-----
> From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> Sent: 15 July 2002 14:55
> To: File Systems Developers
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> If your FS driver is not prepared to bring data back from secondary
sotrage
> (you should properly handle
> IRP_MJ_READ , with Irp->Flags IRP_NOCACHE | IRP_PAGING_IO ) your file
> system will not be able
> to satisfy ANY inpage operations. That means, no cache support and no
memory
> mapped files support.
> Donno if you handle this situation properly so far, for you say you
started
> with FsControl IRP. But if you build it upon an already made skeleton ,
> maybe you have it … Check if you do support paging IO properly in your
> file system. This support must be functional and up before you attmept to
> use the Cc.
>
> Dan
>
>
>
>
>
>
>
> ----- Original Message -----
> From: “James Dunning”
> To: “File Systems Developers”
> Sent: Monday, July 15, 2002 4:45 PM
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> > Thanks again for the response!
> >
> > Unfortunately i am still stumped as to why the CcMapData is throwing a
> > STATUS_IN_PAGE_ERROR. So I think would be best if i tried to explain
what
> i
> > am trying to do.
> >
> > I am currently writing a file system driver for a very simple file
system,
> > and up until now i have been gutting the cdfs file system which came
with
> > the Windows XP IFS kit. I am at a stage where i can now implement the
IRP
> > dispatch routines which are required for my file system, I have decided
to
> > start on the IRP_MJ_FILE_SYSTEM_CONTROL and implement the
> > IRP_MN_MOUNT_VOLUME request.
> >
> > Now i would like to use the Cache Manager here so i can cache the
volume’s
> > on disk structures. the primary volume block resides at block 1, which
is
> > only 64 bytes. I have used InitializeVcb to set up a new stream
> > VirualVolume file object by calling IoCreateStreamFileObject and then
> called
> > ccInitializeCacheMap, configuring the cache map filesize to 64 bytes.
> >
> > Now correct me if i am wrong, now that i have intialized the cache, i
> should
> > be able to call CcMapData which should page in the data from the disk?
> >
> > As far as i am aware the hdd doesnt have any bad sectors, and has been
> > partitioned with 1 formatted partition with the necessary on disk
> structures
> > for my filesystem to pick up and identify. Incidentally I have managed
to
> > read the on disk structures without the aid of the cache manager…
> >
> > Regards
> > James
> >
> > Ps. here’s a few snippets from my code…
> >
> > // the following code is in the InitializeVcb routine called from
> > MountVolume…
> >
> > RealDevice = Vcb->CurrentDevice;
> >
> > Vcb->VirtualVolumeFile = UnwindFileObject =
> > IoCreateStreamFileObject( NULL, RealDevice );
> >
> > StSetFileObject( Vcb->VirtualVolumeFile,
> > VirtualVolumeFile,
> > Vcb,
> > NULL );
> >
> > Vcb->VirtualVolumeFile->SectionObjectPointer =
> > &Vcb->SectionObjectPointers;
> >
> > Vcb->VirtualVolumeFile->ReadAccess = TRUE;
> > Vcb->VirtualVolumeFile->WriteAccess = TRUE;
> > Vcb->VirtualVolumeFile->DeleteAccess = TRUE;
> >
> > FileSizes.AllocationSize.QuadPart =
> > FileSizes.FileSize.QuadPart = sizeof(DIRECTORY_BLOCK_FORMAT);
> > FileSizes.ValidDataLength = StMaxLarge;
> >
> > CcInitializeCacheMap( Vcb->VirtualVolumeFile,
> > &FileSizes,
> > TRUE,
> > &StData.CacheManagerNoOpCallbacks,
> > Vcb );
> >
> > // this is in the MountVolume routine
> > try {
> > StReadVolumeFile( IrpContext,
> > Vcb,
> > 0, // Starting Byte
> > sizeof(DIRECTORY_BLOCK_FORMAT), // byte
> > count
> > &VolumeBcb,
> > (PVOID *)&VolumeBlock );
> >
> > } except(EXCEPTION_EXECUTE_HANDLER) {
> > ExceptionCode = GetExceptionCode();
> > }
> >
> >
> > // this is basically called from within the StReadVolumeFile routine…
> >
> > CcMapData( Vcb->VirtualVolumeFile,
> > &Vbo, // starting byte
> > ByteCount, // byte count
> > BooleanFlagOn(IrpContext->Flags, IRP_CONTEXT_FLAG_WAIT),
> > Bcb, // ref to volume Bcb
> > Buffer ) // ref to volume Block
> >
> >
> >
> > -----Original Message-----
> > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > Sent: 12 July 2002 12:39
> > To: File Systems Developers
> > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> >
> >
> > STATUS_IN_PAGE_ERROR is returned in the case the page fault handler was
> not
> > able to read a swaped out memory page from the secondary storage. This
can
> > be the result of a many causes , such as:
> >
> > 1. Malfunctioning HW , like bad sectors on your HDD
> > 2. If you write a file system filter, check the paging IO path and make
> sure
> > the logic is error free.
> > 3. If you write a file system, make sure you handle corectly paging IO
> logic
> > for both paging files, and memory mapped files.
> > For memory mapped files , unless the view is mapped aginst a section
> backed
> > by a system pagingfile, the inpage operations will take place directly
> from
> > the file backing up the section.
> >
> > 4. If you have your own filters on the storage stack, or you have your
own
> > scsi miniports in the storage path, make sure
> > they are corectly written.
> >
> > However , it should be easy to isolate the cause.
> >
> > As a general rule , when you work with the cache manager, be aware that
> > almost all Cc functions can raise exceptions
> > on error conditions, altough this behaviour is not documented. The
status
> > codes returned are very valuable information
> > to see what went wrong. Keep this in mind when you code with Cc, and
adopt
> > defensive programming tactics. Employ
> > SEH to handle this kind of errors. Just as a side note, dont HIDE them
> > trough SEH(ugly aint it ?), but actualy handle the situation, one way or
> > another
> >
> > Reagrds, Dan
> >
> >
> >
> >
> > ----- Original Message -----
> > From: “James Dunning”
> > To: “File Systems Developers”
> > Sent: Friday, July 12, 2002 1:51 PM
> > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> >
> >
> > > Thanks for the reply…
> > >
> > > I have just placed a try{} and except{} wrap around the call to
> > > CcMapData(…) and you were right, I am getting an exception! I have
> > managed
> > > to get the exception code and it returning the following status code,
> > > STATUS_IN_PAGE_ERROR 0xC0000006L
> > >
> > > which I think means that for some reason, the data is not placed into
> > memory
> > > due to an io error?
> > >
> > > Has anyone come across this problem before? and perhaps have an idea
how
> > to
> > > get a better understanding why this error is recurring and resolve
this
> > > situation?
> > >
> > > Regards
> > > James
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > > Sent: 11 July 2002 16:33
> > > To: File Systems Developers
> > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > >
> > >
> > > As far as I know, this routine has nothing to do with fast IO.
> > >
> > > However , your debugger behaves strangley.
> > > Practice told me that many Cc functions can raise exceptions so Id
> sugest
> > to
> > > have a try{} except{}
> > > construct wrapping this call and see if you can catch an exception.
If
> > yes,
> > > the status code
> > > can be enlightning.
> > >
> > > Dan
> > >
> > > “James Dunning” wrote in
message
> > > news:xxxxx@ntfsd…
> > > >
> > > > But can CcMapData be called without the use of FastIo?
> > > >
> > > > The Wait parameter was actually already set to TRUE, but as i step
> over
> > > the
> > > > CcMapData(…) routine Softice seem to behave rather strangely and
> fails
> > > to
> > > > step to the next routine, it either exits from the softice debugger
> and
> > > goes
> > > > back to the main windows xp desktop, or to the original break point
> just
> > > > above the point where CcMapData(…) is actually called… either
> way,
> > > this
> > > > is preventing me from examining the buffer returned from CcMapData!
> > > >
> > > > I am wondering if i have missed something out while setting up the
> cache
> > > > (like not initializing fastio) or whether there is a problem
> currupting
> > > the
> > > > memory somewhere, which is causing this bizzare behavour.
> > > >
> > > > Any advice?
> > > >
> > > > Regards,
> > > > James
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > > > Sent: 11 July 2002 14:12
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > > >
> > > >
> > > > Valid means TRUE , no ? If caching for the stream has been
> intialized,
> > > then
> > > > this routine can be called. This routine will return FALSE if the
> > mapping
> > > > request cannot be performed imediately, and the Wait parameter is
set
> to
> > > > FALSE. Set Wait to true, and see if you get the expected result.
> > > >
> > > > Dan
> > > >
> > > > ----- Original Message -----
> > > > From: “James Dunning”
> > > > To: “File Systems Developers”
> > > > Sent: Thursday, July 11, 2002 3:43 PM
> > > > Subject: [ntfsd] Having problems getting CcMapData to work!..
> > > >
> > > >
> > > > > Hi,
> > > > >
> > > > > I am currently having problems getting CcMapData(…) to work and
> > return
> > > > any
> > > > > valid data. I have debugged the code and the
> > CcInitializeCacheMap(…)
> > > > > appears to have initialized the cache map successfully…
> > > > >
> > > > > Could anyone tell me if in order for the CcMapData(…) to work,
do
> i
> > > need
> > > > > to implement and initialise the FastIO entry points? or can i use
> > > > CcMapData
> > > > > without having fastio support?
> > > > >
> > > > > Regards,
> > > > > James…
> > > > >
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > > > To unsubscribe send a blank email to %%email.unsub%%
> > > > >
> > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as:
> > > > xxxxx@generaldynamics.uk.com
> > > > To unsubscribe send a blank email to %%email.unsub%%
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as:
> > > xxxxx@generaldynamics.uk.com
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as:
> > xxxxx@generaldynamics.uk.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@generaldynamics.uk.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
>
—
You are currently subscribed to ntfsd as:
xxxxx@generaldynamics.uk.com
To unsubscribe send a blank email to %%email.unsub%%
The function of the IRP_NOCACHE bit is to indicate that you should NOT
retrieve the data from the cache. If you are satisfying a paging I/O
operation, you can’t retrieve it from the cache - since if it were in the
cache, you wouldn’t be getting the paging I/O request at all.
The valid combinations here would be:
Neither bit set - normal user read/write request
Both bits set - normal paging I/O request
No-cache set, paging I/O clear - non-cached user read/write request
The fourth combination (paging I/O set, no-cache clear) would be invalid.
Regards,
Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com
-----Original Message-----
From: James Dunning [mailto:xxxxx@generaldynamics.uk.com]
Sent: Tuesday, July 16, 2002 7:31 AM
To: File Systems Developers
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
Hi Dan,
I would just like to quickly make sure that i have understood you
correctly…
In the previous reply, you mentioned that i should properly handle
IRP_MJ_READ, with Irp->Flags IRP_NOCACHE|IRP_PAGING_IO in my file system, in
order to satisfy ANY inpage operations.
Now if i use CcMapData, should the following flags IRP_NOCACHE (true) and
IRP_PAGING_IO (true) be set within the IRP_MJ_READ dispatch routine?
BOOLEAN PagingIo = BooleanFlagOn(Irp->Flags, IRP_PAGING_IO);
BOOLEAN NonCachedIo = BooleanFlagOn(Irp->Flags,IRP_NOCACHE);
The reason why i ask is because im getting confused why NonCacheIo should be
set to true, if CcMapData is one of the IFS cache routines. Surely that
NonCacheIo should be set to FALSE and PagingIo should be set to TRUE?
Regards,
James
-----Original Message-----
From: Dan Partelly [mailto:xxxxx@rdsor.ro]
Sent: 15 July 2002 15:55
To: File Systems Developers
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> I just assumed that CcMapData called the device below the fs and did all
the work for me!
Assumption is mother of all f…ups. Besides , how could Cc manager know
which sectors to read from the
storage stack ? This is FS business. Maybe you should read a little more
about file systems, Cc manager and
brothers. Try Rajeev Nagar’s book.
“James Dunning” wrote in message
news:xxxxx@ntfsd…
>
> Ohhh Right! thats great cheers… I didnt realise that the IRP_MJ_READ and
i
> guess the IRP_MJ_CLEANUP needed to be implemented within the file system
and
> with the support for paging IO!
>
> I have quickly placed some breakpoints with in the fsdDispatch routine,
and
> i have noticed that when CcMapData is called, an IRP_MJ_READ request is
> generated, but the routine to handle the IRP_MJ_READ has been gutted… i
> guess i have over gutted the filesystem! lol! nevermind, i guess i can add
> the routines back as and when i require them! I just assumed that
CcMapData
> called the device below the fs and did all the work for me!
>
> Thanks again for your help.
>
> James
>
> -----Original Message-----
> From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> Sent: 15 July 2002 14:55
> To: File Systems Developers
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> If your FS driver is not prepared to bring data back from secondary
sotrage
> (you should properly handle
> IRP_MJ_READ , with Irp->Flags IRP_NOCACHE | IRP_PAGING_IO ) your file
> system will not be able
> to satisfy ANY inpage operations. That means, no cache support and no
memory
> mapped files support.
> Donno if you handle this situation properly so far, for you say you
started
> with FsControl IRP. But if you build it upon an already made skeleton ,
> maybe you have it … Check if you do support paging IO properly in your
> file system. This support must be functional and up before you attmept to
> use the Cc.
>
> Dan
>
>
>
>
>
>
>
> ----- Original Message -----
> From: “James Dunning”
> To: “File Systems Developers”
> Sent: Monday, July 15, 2002 4:45 PM
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> > Thanks again for the response!
> >
> > Unfortunately i am still stumped as to why the CcMapData is throwing a
> > STATUS_IN_PAGE_ERROR. So I think would be best if i tried to explain
what
> i
> > am trying to do.
> >
> > I am currently writing a file system driver for a very simple file
system,
> > and up until now i have been gutting the cdfs file system which came
with
> > the Windows XP IFS kit. I am at a stage where i can now implement the
IRP
> > dispatch routines which are required for my file system, I have decided
to
> > start on the IRP_MJ_FILE_SYSTEM_CONTROL and implement the
> > IRP_MN_MOUNT_VOLUME request.
> >
> > Now i would like to use the Cache Manager here so i can cache the
volume’s
> > on disk structures. the primary volume block resides at block 1, which
is
> > only 64 bytes. I have used InitializeVcb to set up a new stream
> > VirualVolume file object by calling IoCreateStreamFileObject and then
> called
> > ccInitializeCacheMap, configuring the cache map filesize to 64 bytes.
> >
> > Now correct me if i am wrong, now that i have intialized the cache, i
> should
> > be able to call CcMapData which should page in the data from the disk?
> >
> > As far as i am aware the hdd doesnt have any bad sectors, and has been
> > partitioned with 1 formatted partition with the necessary on disk
> structures
> > for my filesystem to pick up and identify. Incidentally I have managed
to
> > read the on disk structures without the aid of the cache manager…
> >
> > Regards
> > James
> >
> > Ps. here’s a few snippets from my code…
> >
> > // the following code is in the InitializeVcb routine called from
> > MountVolume…
> >
> > RealDevice = Vcb->CurrentDevice;
> >
> > Vcb->VirtualVolumeFile = UnwindFileObject =
> > IoCreateStreamFileObject( NULL, RealDevice );
> >
> > StSetFileObject( Vcb->VirtualVolumeFile,
> > VirtualVolumeFile,
> > Vcb,
> > NULL );
> >
> > Vcb->VirtualVolumeFile->SectionObjectPointer =
> > &Vcb->SectionObjectPointers;
> >
> > Vcb->VirtualVolumeFile->ReadAccess = TRUE;
> > Vcb->VirtualVolumeFile->WriteAccess = TRUE;
> > Vcb->VirtualVolumeFile->DeleteAccess = TRUE;
> >
> > FileSizes.AllocationSize.QuadPart =
> > FileSizes.FileSize.QuadPart = sizeof(DIRECTORY_BLOCK_FORMAT);
> > FileSizes.ValidDataLength = StMaxLarge;
> >
> > CcInitializeCacheMap( Vcb->VirtualVolumeFile,
> > &FileSizes,
> > TRUE,
> > &StData.CacheManagerNoOpCallbacks,
> > Vcb );
> >
> > // this is in the MountVolume routine
> > try {
> > StReadVolumeFile( IrpContext,
> > Vcb,
> > 0, // Starting Byte
> > sizeof(DIRECTORY_BLOCK_FORMAT), // byte
> > count
> > &VolumeBcb,
> > (PVOID *)&VolumeBlock );
> >
> > } except(EXCEPTION_EXECUTE_HANDLER) {
> > ExceptionCode = GetExceptionCode();
> > }
> >
> >
> > // this is basically called from within the StReadVolumeFile routine…
> >
> > CcMapData( Vcb->VirtualVolumeFile,
> > &Vbo, // starting byte
> > ByteCount, // byte count
> > BooleanFlagOn(IrpContext->Flags, IRP_CONTEXT_FLAG_WAIT),
> > Bcb, // ref to volume Bcb
> > Buffer ) // ref to volume Block
> >
> >
> >
> > -----Original Message-----
> > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > Sent: 12 July 2002 12:39
> > To: File Systems Developers
> > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> >
> >
> > STATUS_IN_PAGE_ERROR is returned in the case the page fault handler was
> not
> > able to read a swaped out memory page from the secondary storage. This
can
> > be the result of a many causes , such as:
> >
> > 1. Malfunctioning HW , like bad sectors on your HDD
> > 2. If you write a file system filter, check the paging IO path and make
> sure
> > the logic is error free.
> > 3. If you write a file system, make sure you handle corectly paging IO
> logic
> > for both paging files, and memory mapped files.
> > For memory mapped files , unless the view is mapped aginst a section
> backed
> > by a system pagingfile, the inpage operations will take place directly
> from
> > the file backing up the section.
> >
> > 4. If you have your own filters on the storage stack, or you have your
own
> > scsi miniports in the storage path, make sure
> > they are corectly written.
> >
> > However , it should be easy to isolate the cause.
> >
> > As a general rule , when you work with the cache manager, be aware that
> > almost all Cc functions can raise exceptions
> > on error conditions, altough this behaviour is not documented. The
status
> > codes returned are very valuable information
> > to see what went wrong. Keep this in mind when you code with Cc, and
adopt
> > defensive programming tactics. Employ
> > SEH to handle this kind of errors. Just as a side note, dont HIDE them
> > trough SEH(ugly aint it ?), but actualy handle the situation, one way or
> > another
> >
> > Reagrds, Dan
> >
> >
> >
> >
> > ----- Original Message -----
> > From: “James Dunning”
> > To: “File Systems Developers”
> > Sent: Friday, July 12, 2002 1:51 PM
> > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> >
> >
> > > Thanks for the reply…
> > >
> > > I have just placed a try{} and except{} wrap around the call to
> > > CcMapData(…) and you were right, I am getting an exception! I have
> > managed
> > > to get the exception code and it returning the following status code,
> > > STATUS_IN_PAGE_ERROR 0xC0000006L
> > >
> > > which I think means that for some reason, the data is not placed into
> > memory
> > > due to an io error?
> > >
> > > Has anyone come across this problem before? and perhaps have an idea
how
> > to
> > > get a better understanding why this error is recurring and resolve
this
> > > situation?
> > >
> > > Regards
> > > James
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > > Sent: 11 July 2002 16:33
> > > To: File Systems Developers
> > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > >
> > >
> > > As far as I know, this routine has nothing to do with fast IO.
> > >
> > > However , your debugger behaves strangley.
> > > Practice told me that many Cc functions can raise exceptions so Id
> sugest
> > to
> > > have a try{} except{}
> > > construct wrapping this call and see if you can catch an exception.
If
> > yes,
> > > the status code
> > > can be enlightning.
> > >
> > > Dan
> > >
> > > “James Dunning” wrote in
message
> > > news:xxxxx@ntfsd…
> > > >
> > > > But can CcMapData be called without the use of FastIo?
> > > >
> > > > The Wait parameter was actually already set to TRUE, but as i step
> over
> > > the
> > > > CcMapData(…) routine Softice seem to behave rather strangely and
> fails
> > > to
> > > > step to the next routine, it either exits from the softice debugger
> and
> > > goes
> > > > back to the main windows xp desktop, or to the original break point
> just
> > > > above the point where CcMapData(…) is actually called… either
> way,
> > > this
> > > > is preventing me from examining the buffer returned from CcMapData!
> > > >
> > > > I am wondering if i have missed something out while setting up the
> cache
> > > > (like not initializing fastio) or whether there is a problem
> currupting
> > > the
> > > > memory somewhere, which is causing this bizzare behavour.
> > > >
> > > > Any advice?
> > > >
> > > > Regards,
> > > > James
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > > > Sent: 11 July 2002 14:12
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > > >
> > > >
> > > > Valid means TRUE , no ? If caching for the stream has been
> intialized,
> > > then
> > > > this routine can be called. This routine will return FALSE if the
> > mapping
> > > > request cannot be performed imediately, and the Wait parameter is
set
> to
> > > > FALSE. Set Wait to true, and see if you get the expected result.
> > > >
> > > > Dan
> > > >
> > > > ----- Original Message -----
> > > > From: “James Dunning”
> > > > To: “File Systems Developers”
> > > > Sent: Thursday, July 11, 2002 3:43 PM
> > > > Subject: [ntfsd] Having problems getting CcMapData to work!..
> > > >
> > > >
> > > > > Hi,
> > > > >
> > > > > I am currently having problems getting CcMapData(…) to work and
> > return
> > > > any
> > > > > valid data. I have debugged the code and the
> > CcInitializeCacheMap(…)
> > > > > appears to have initialized the cache map successfully…
> > > > >
> > > > > Could anyone tell me if in order for the CcMapData(…) to work,
do
> i
> > > need
> > > > > to implement and initialise the FastIO entry points? or can i use
> > > > CcMapData
> > > > > without having fastio support?
> > > > >
> > > > > Regards,
> > > > > James…
> > > > >
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > > > To unsubscribe send a blank email to %%email.unsub%%
> > > > >
> > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as:
> > > > xxxxx@generaldynamics.uk.com
> > > > To unsubscribe send a blank email to %%email.unsub%%
> > > >
> > > >
> > > >
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as:
> > > xxxxx@generaldynamics.uk.com
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as:
> > xxxxx@generaldynamics.uk.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
> >
> > —
> > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > To unsubscribe send a blank email to %%email.unsub%%
> >
>
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@generaldynamics.uk.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
>
—
You are currently subscribed to ntfsd as:
xxxxx@generaldynamics.uk.com
To unsubscribe send a blank email to %%email.unsub%%
—
You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to %%email.unsub%%
Read Tony Mason’s answer to get a picture of the states which can be defined
through Irp->Flags. When the Cc manager is requested to read data , it will
read a mapped view of a section object , backed up by the file which is
cached. It does this by simply reading memory from the mapped view. If data
is present in cache , then everything is OK. If data is not in the cache,
when the page is touched for reading, it will generate a page fault. Here
the Mm manager will kick in , and will request a inpage operation from the
file system. The IRP for this request will have IRP_NOCACHE | IRP_PAGING_IO
set to indicate the file system 2 important things:
Also be aware that a third flag can kick in , namely
IRP_SYNCHRONOUS_PAGING_IO , which indicates that the request should be
synchronusly completed by the file system. If this flag is not present,
asynchronous operation is assumed.
>The reason why i ask is because im getting confused why NonCacheIo should
be set to true, if CcMapData is one of the IFS cache routines
You must understand that Cc is very tightly coupled with Mm, and all IO
performed by the Cc manager as a result of a cache miss, read ahead, or
write behind, flush or teardown section, mapping & pinning, operations will
come to life as IRP_NO_CACHE | IRP_PAGING_IO. The Cc manager itself needs
the data to satisfy a request.
Dan
----- Original Message -----
From: “James Dunning”
To: “File Systems Developers”
Sent: Tuesday, July 16, 2002 2:30 PM
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> Hi Dan,
>
> I would just like to quickly make sure that i have understood you
> correctly…
>
> In the previous reply, you mentioned that i should properly handle
> IRP_MJ_READ, with Irp->Flags IRP_NOCACHE|IRP_PAGING_IO in my file system,
in
> order to satisfy ANY inpage operations.
>
> Now if i use CcMapData, should the following flags IRP_NOCACHE (true) and
> IRP_PAGING_IO (true) be set within the IRP_MJ_READ dispatch routine?
>
> BOOLEAN PagingIo = BooleanFlagOn(Irp->Flags, IRP_PAGING_IO);
> BOOLEAN NonCachedIo = BooleanFlagOn(Irp->Flags,IRP_NOCACHE);
>
> The reason why i ask is because im getting confused why NonCacheIo should
be
> set to true, if CcMapData is one of the IFS cache routines. Surely that
> NonCacheIo should be set to FALSE and PagingIo should be set to TRUE?
>
> Regards,
> James
>
>
>
> -----Original Message-----
> From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> Sent: 15 July 2002 15:55
> To: File Systems Developers
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> >> I just assumed that CcMapData called the device below the fs and did
all
> the work for me!
>
> Assumption is mother of all f…ups. Besides , how could Cc manager know
> which sectors to read from the
> storage stack ? This is FS business. Maybe you should read a little more
> about file systems, Cc manager and
> brothers. Try Rajeev Nagar’s book.
>
>
>
> “James Dunning” wrote in message
> news:xxxxx@ntfsd…
> >
> > Ohhh Right! thats great cheers… I didnt realise that the IRP_MJ_READ
and
> i
> > guess the IRP_MJ_CLEANUP needed to be implemented within the file system
> and
> > with the support for paging IO!
> >
> > I have quickly placed some breakpoints with in the fsdDispatch routine,
> and
> > i have noticed that when CcMapData is called, an IRP_MJ_READ request is
> > generated, but the routine to handle the IRP_MJ_READ has been gutted… i
> > guess i have over gutted the filesystem! lol! nevermind, i guess i can
add
> > the routines back as and when i require them! I just assumed that
> CcMapData
> > called the device below the fs and did all the work for me!
> >
> > Thanks again for your help.
> >
> > James
> >
> > -----Original Message-----
> > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > Sent: 15 July 2002 14:55
> > To: File Systems Developers
> > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> >
> >
> > If your FS driver is not prepared to bring data back from secondary
> sotrage
> > (you should properly handle
> > IRP_MJ_READ , with Irp->Flags IRP_NOCACHE | IRP_PAGING_IO ) your file
> > system will not be able
> > to satisfy ANY inpage operations. That means, no cache support and no
> memory
> > mapped files support.
> > Donno if you handle this situation properly so far, for you say you
> started
> > with FsControl IRP. But if you build it upon an already made skeleton ,
> > maybe you have it … Check if you do support paging IO properly in
your
> > file system. This support must be functional and up before you attmept
to
> > use the Cc.
> >
> > Dan
> >
> >
> >
> >
> >
> >
> >
> > ----- Original Message -----
> > From: “James Dunning”
> > To: “File Systems Developers”
> > Sent: Monday, July 15, 2002 4:45 PM
> > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> >
> >
> > > Thanks again for the response!
> > >
> > > Unfortunately i am still stumped as to why the CcMapData is throwing a
> > > STATUS_IN_PAGE_ERROR. So I think would be best if i tried to explain
> what
> > i
> > > am trying to do.
> > >
> > > I am currently writing a file system driver for a very simple file
> system,
> > > and up until now i have been gutting the cdfs file system which came
> with
> > > the Windows XP IFS kit. I am at a stage where i can now implement the
> IRP
> > > dispatch routines which are required for my file system, I have
decided
> to
> > > start on the IRP_MJ_FILE_SYSTEM_CONTROL and implement the
> > > IRP_MN_MOUNT_VOLUME request.
> > >
> > > Now i would like to use the Cache Manager here so i can cache the
> volume’s
> > > on disk structures. the primary volume block resides at block 1,
which
> is
> > > only 64 bytes. I have used InitializeVcb to set up a new stream
> > > VirualVolume file object by calling IoCreateStreamFileObject and then
> > called
> > > ccInitializeCacheMap, configuring the cache map filesize to 64 bytes.
> > >
> > > Now correct me if i am wrong, now that i have intialized the cache, i
> > should
> > > be able to call CcMapData which should page in the data from the disk?
> > >
> > > As far as i am aware the hdd doesnt have any bad sectors, and has been
> > > partitioned with 1 formatted partition with the necessary on disk
> > structures
> > > for my filesystem to pick up and identify. Incidentally I have
managed
> to
> > > read the on disk structures without the aid of the cache manager…
> > >
> > > Regards
> > > James
> > >
> > > Ps. here’s a few snippets from my code…
> > >
> > > // the following code is in the InitializeVcb routine called from
> > > MountVolume…
> > >
> > > RealDevice = Vcb->CurrentDevice;
> > >
> > > Vcb->VirtualVolumeFile = UnwindFileObject =
> > > IoCreateStreamFileObject( NULL, RealDevice );
> > >
> > > StSetFileObject( Vcb->VirtualVolumeFile,
> > > VirtualVolumeFile,
> > > Vcb,
> > > NULL );
> > >
> > > Vcb->VirtualVolumeFile->SectionObjectPointer =
> > > &Vcb->SectionObjectPointers;
> > >
> > > Vcb->VirtualVolumeFile->ReadAccess = TRUE;
> > > Vcb->VirtualVolumeFile->WriteAccess = TRUE;
> > > Vcb->VirtualVolumeFile->DeleteAccess = TRUE;
> > >
> > > FileSizes.AllocationSize.QuadPart =
> > > FileSizes.FileSize.QuadPart = sizeof(DIRECTORY_BLOCK_FORMAT);
> > > FileSizes.ValidDataLength = StMaxLarge;
> > >
> > > CcInitializeCacheMap( Vcb->VirtualVolumeFile,
> > > &FileSizes,
> > > TRUE,
> > > &StData.CacheManagerNoOpCallbacks,
> > > Vcb );
> > >
> > > // this is in the MountVolume routine
> > > try {
> > > StReadVolumeFile( IrpContext,
> > > Vcb,
> > > 0, // Starting Byte
> > > sizeof(DIRECTORY_BLOCK_FORMAT), // byte
> > > count
> > > &VolumeBcb,
> > > (PVOID *)&VolumeBlock );
> > >
> > > } except(EXCEPTION_EXECUTE_HANDLER) {
> > > ExceptionCode = GetExceptionCode();
> > > }
> > >
> > >
> > > // this is basically called from within the StReadVolumeFile routine…
> > >
> > > CcMapData( Vcb->VirtualVolumeFile,
> > > &Vbo, // starting byte
> > > ByteCount, // byte count
> > > BooleanFlagOn(IrpContext->Flags, IRP_CONTEXT_FLAG_WAIT),
> > > Bcb, // ref to volume Bcb
> > > Buffer ) // ref to volume Block
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > > Sent: 12 July 2002 12:39
> > > To: File Systems Developers
> > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > >
> > >
> > > STATUS_IN_PAGE_ERROR is returned in the case the page fault handler
was
> > not
> > > able to read a swaped out memory page from the secondary storage. This
> can
> > > be the result of a many causes , such as:
> > >
> > > 1. Malfunctioning HW , like bad sectors on your HDD
> > > 2. If you write a file system filter, check the paging IO path and
make
> > sure
> > > the logic is error free.
> > > 3. If you write a file system, make sure you handle corectly paging IO
> > logic
> > > for both paging files, and memory mapped files.
> > > For memory mapped files , unless the view is mapped aginst a section
> > backed
> > > by a system pagingfile, the inpage operations will take place directly
> > from
> > > the file backing up the section.
> > >
> > > 4. If you have your own filters on the storage stack, or you have your
> own
> > > scsi miniports in the storage path, make sure
> > > they are corectly written.
> > >
> > > However , it should be easy to isolate the cause.
> > >
> > > As a general rule , when you work with the cache manager, be aware
that
> > > almost all Cc functions can raise exceptions
> > > on error conditions, altough this behaviour is not documented. The
> status
> > > codes returned are very valuable information
> > > to see what went wrong. Keep this in mind when you code with Cc, and
> adopt
> > > defensive programming tactics. Employ
> > > SEH to handle this kind of errors. Just as a side note, dont HIDE them
> > > trough SEH(ugly aint it ?), but actualy handle the situation, one way
or
> > > another
> > >
> > > Reagrds, Dan
> > >
> > >
> > >
> > >
> > > ----- Original Message -----
> > > From: “James Dunning”
> > > To: “File Systems Developers”
> > > Sent: Friday, July 12, 2002 1:51 PM
> > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > >
> > >
> > > > Thanks for the reply…
> > > >
> > > > I have just placed a try{} and except{} wrap around the call to
> > > > CcMapData(…) and you were right, I am getting an exception! I have
> > > managed
> > > > to get the exception code and it returning the following status
code,
> > > > STATUS_IN_PAGE_ERROR 0xC0000006L
> > > >
> > > > which I think means that for some reason, the data is not placed
into
> > > memory
> > > > due to an io error?
> > > >
> > > > Has anyone come across this problem before? and perhaps have an idea
> how
> > > to
> > > > get a better understanding why this error is recurring and resolve
> this
> > > > situation?
> > > >
> > > > Regards
> > > > James
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > > > Sent: 11 July 2002 16:33
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > > >
> > > >
> > > > As far as I know, this routine has nothing to do with fast IO.
> > > >
> > > > However , your debugger behaves strangley.
> > > > Practice told me that many Cc functions can raise exceptions so Id
> > sugest
> > > to
> > > > have a try{} except{}
> > > > construct wrapping this call and see if you can catch an exception.
> If
> > > yes,
> > > > the status code
> > > > can be enlightning.
> > > >
> > > > Dan
> > > >
> > > > “James Dunning” wrote in
> message
> > > > news:xxxxx@ntfsd…
> > > > >
> > > > > But can CcMapData be called without the use of FastIo?
> > > > >
> > > > > The Wait parameter was actually already set to TRUE, but as i step
> > over
> > > > the
> > > > > CcMapData(…) routine Softice seem to behave rather strangely and
> > fails
> > > > to
> > > > > step to the next routine, it either exits from the softice
debugger
> > and
> > > > goes
> > > > > back to the main windows xp desktop, or to the original break
point
> > just
> > > > > above the point where CcMapData(…) is actually called… either
> > way,
> > > > this
> > > > > is preventing me from examining the buffer returned from
CcMapData!
> > > > >
> > > > > I am wondering if i have missed something out while setting up the
> > cache
> > > > > (like not initializing fastio) or whether there is a problem
> > currupting
> > > > the
> > > > > memory somewhere, which is causing this bizzare behavour.
> > > > >
> > > > > Any advice?
> > > > >
> > > > > Regards,
> > > > > James
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > > > > Sent: 11 July 2002 14:12
> > > > > To: File Systems Developers
> > > > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > > > >
> > > > >
> > > > > Valid means TRUE , no ? If caching for the stream has been
> > intialized,
> > > > then
> > > > > this routine can be called. This routine will return FALSE if the
> > > mapping
> > > > > request cannot be performed imediately, and the Wait parameter is
> set
> > to
> > > > > FALSE. Set Wait to true, and see if you get the expected result.
> > > > >
> > > > > Dan
> > > > >
> > > > > ----- Original Message -----
> > > > > From: “James Dunning”
> > > > > To: “File Systems Developers”
> > > > > Sent: Thursday, July 11, 2002 3:43 PM
> > > > > Subject: [ntfsd] Having problems getting CcMapData to work!..
> > > > >
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I am currently having problems getting CcMapData(…) to work
and
> > > return
> > > > > any
> > > > > > valid data. I have debugged the code and the
> > > CcInitializeCacheMap(…)
> > > > > > appears to have initialized the cache map successfully…
> > > > > >
> > > > > > Could anyone tell me if in order for the CcMapData(…) to work,
> do
> > i
> > > > need
> > > > > > to implement and initialise the FastIO entry points? or can i
use
> > > > > CcMapData
> > > > > > without having fastio support?
> > > > > >
> > > > > > Regards,
> > > > > > James…
> > > > > >
> > > > > >
> > > > > > —
> > > > > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > > > > To unsubscribe send a blank email to %%email.unsub%%
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as:
> > > > > xxxxx@generaldynamics.uk.com
> > > > > To unsubscribe send a blank email to %%email.unsub%%
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as:
> > > > xxxxx@generaldynamics.uk.com
> > > > To unsubscribe send a blank email to %%email.unsub%%
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > > To unsubscribe send a blank email to %%email.unsub%%
> > > >
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as:
> > > xxxxx@generaldynamics.uk.com
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as:
> > xxxxx@generaldynamics.uk.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
> >
> >
>
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@generaldynamics.uk.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to %%email.unsub%%
>
I would like to thank you and Tony Mason, this has been really helpful. I
am currently adding cache and memory mapped files support to the IRP_MJ_READ
dispatch handler, so this information has been very useful! especially when
it comes to understanding the way the cdfs and fastfat IRP_MJ_READ dispatch
handlers have been implemented.
regards,
James
-----Original Message-----
From: Dan Partelly [mailto:xxxxx@rdsor.ro]
Sent: 16 July 2002 13:51
To: File Systems Developers
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
Read Tony Mason’s answer to get a picture of the states which can be defined
through Irp->Flags. When the Cc manager is requested to read data , it will
read a mapped view of a section object , backed up by the file which is
cached. It does this by simply reading memory from the mapped view. If data
is present in cache , then everything is OK. If data is not in the cache,
when the page is touched for reading, it will generate a page fault. Here
the Mm manager will kick in , and will request a inpage operation from the
file system. The IRP for this request will have IRP_NOCACHE | IRP_PAGING_IO
set to indicate the file system 2 important things:
Also be aware that a third flag can kick in , namely
IRP_SYNCHRONOUS_PAGING_IO , which indicates that the request should be
synchronusly completed by the file system. If this flag is not present,
asynchronous operation is assumed.
>The reason why i ask is because im getting confused why NonCacheIo should
be set to true, if CcMapData is one of the IFS cache routines
You must understand that Cc is very tightly coupled with Mm, and all IO
performed by the Cc manager as a result of a cache miss, read ahead, or
write behind, flush or teardown section, mapping & pinning, operations will
come to life as IRP_NO_CACHE | IRP_PAGING_IO. The Cc manager itself needs
the data to satisfy a request.
Dan
----- Original Message -----
From: “James Dunning”
To: “File Systems Developers”
Sent: Tuesday, July 16, 2002 2:30 PM
Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> Hi Dan,
>
> I would just like to quickly make sure that i have understood you
> correctly…
>
> In the previous reply, you mentioned that i should properly handle
> IRP_MJ_READ, with Irp->Flags IRP_NOCACHE|IRP_PAGING_IO in my file system,
in
> order to satisfy ANY inpage operations.
>
> Now if i use CcMapData, should the following flags IRP_NOCACHE (true) and
> IRP_PAGING_IO (true) be set within the IRP_MJ_READ dispatch routine?
>
> BOOLEAN PagingIo = BooleanFlagOn(Irp->Flags, IRP_PAGING_IO);
> BOOLEAN NonCachedIo = BooleanFlagOn(Irp->Flags,IRP_NOCACHE);
>
> The reason why i ask is because im getting confused why NonCacheIo should
be
> set to true, if CcMapData is one of the IFS cache routines. Surely that
> NonCacheIo should be set to FALSE and PagingIo should be set to TRUE?
>
> Regards,
> James
>
>
>
> -----Original Message-----
> From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> Sent: 15 July 2002 15:55
> To: File Systems Developers
> Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
>
>
> >> I just assumed that CcMapData called the device below the fs and did
all
> the work for me!
>
> Assumption is mother of all f…ups. Besides , how could Cc manager know
> which sectors to read from the
> storage stack ? This is FS business. Maybe you should read a little more
> about file systems, Cc manager and
> brothers. Try Rajeev Nagar’s book.
>
>
>
> “James Dunning” wrote in message
> news:xxxxx@ntfsd…
> >
> > Ohhh Right! thats great cheers… I didnt realise that the IRP_MJ_READ
and
> i
> > guess the IRP_MJ_CLEANUP needed to be implemented within the file system
> and
> > with the support for paging IO!
> >
> > I have quickly placed some breakpoints with in the fsdDispatch routine,
> and
> > i have noticed that when CcMapData is called, an IRP_MJ_READ request is
> > generated, but the routine to handle the IRP_MJ_READ has been gutted… i
> > guess i have over gutted the filesystem! lol! nevermind, i guess i can
add
> > the routines back as and when i require them! I just assumed that
> CcMapData
> > called the device below the fs and did all the work for me!
> >
> > Thanks again for your help.
> >
> > James
> >
> > -----Original Message-----
> > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > Sent: 15 July 2002 14:55
> > To: File Systems Developers
> > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> >
> >
> > If your FS driver is not prepared to bring data back from secondary
> sotrage
> > (you should properly handle
> > IRP_MJ_READ , with Irp->Flags IRP_NOCACHE | IRP_PAGING_IO ) your file
> > system will not be able
> > to satisfy ANY inpage operations. That means, no cache support and no
> memory
> > mapped files support.
> > Donno if you handle this situation properly so far, for you say you
> started
> > with FsControl IRP. But if you build it upon an already made skeleton ,
> > maybe you have it … Check if you do support paging IO properly in
your
> > file system. This support must be functional and up before you attmept
to
> > use the Cc.
> >
> > Dan
> >
> >
> >
> >
> >
> >
> >
> > ----- Original Message -----
> > From: “James Dunning”
> > To: “File Systems Developers”
> > Sent: Monday, July 15, 2002 4:45 PM
> > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> >
> >
> > > Thanks again for the response!
> > >
> > > Unfortunately i am still stumped as to why the CcMapData is throwing a
> > > STATUS_IN_PAGE_ERROR. So I think would be best if i tried to explain
> what
> > i
> > > am trying to do.
> > >
> > > I am currently writing a file system driver for a very simple file
> system,
> > > and up until now i have been gutting the cdfs file system which came
> with
> > > the Windows XP IFS kit. I am at a stage where i can now implement the
> IRP
> > > dispatch routines which are required for my file system, I have
decided
> to
> > > start on the IRP_MJ_FILE_SYSTEM_CONTROL and implement the
> > > IRP_MN_MOUNT_VOLUME request.
> > >
> > > Now i would like to use the Cache Manager here so i can cache the
> volume’s
> > > on disk structures. the primary volume block resides at block 1,
which
> is
> > > only 64 bytes. I have used InitializeVcb to set up a new stream
> > > VirualVolume file object by calling IoCreateStreamFileObject and then
> > called
> > > ccInitializeCacheMap, configuring the cache map filesize to 64 bytes.
> > >
> > > Now correct me if i am wrong, now that i have intialized the cache, i
> > should
> > > be able to call CcMapData which should page in the data from the disk?
> > >
> > > As far as i am aware the hdd doesnt have any bad sectors, and has been
> > > partitioned with 1 formatted partition with the necessary on disk
> > structures
> > > for my filesystem to pick up and identify. Incidentally I have
managed
> to
> > > read the on disk structures without the aid of the cache manager…
> > >
> > > Regards
> > > James
> > >
> > > Ps. here’s a few snippets from my code…
> > >
> > > // the following code is in the InitializeVcb routine called from
> > > MountVolume…
> > >
> > > RealDevice = Vcb->CurrentDevice;
> > >
> > > Vcb->VirtualVolumeFile = UnwindFileObject =
> > > IoCreateStreamFileObject( NULL, RealDevice );
> > >
> > > StSetFileObject( Vcb->VirtualVolumeFile,
> > > VirtualVolumeFile,
> > > Vcb,
> > > NULL );
> > >
> > > Vcb->VirtualVolumeFile->SectionObjectPointer =
> > > &Vcb->SectionObjectPointers;
> > >
> > > Vcb->VirtualVolumeFile->ReadAccess = TRUE;
> > > Vcb->VirtualVolumeFile->WriteAccess = TRUE;
> > > Vcb->VirtualVolumeFile->DeleteAccess = TRUE;
> > >
> > > FileSizes.AllocationSize.QuadPart =
> > > FileSizes.FileSize.QuadPart = sizeof(DIRECTORY_BLOCK_FORMAT);
> > > FileSizes.ValidDataLength = StMaxLarge;
> > >
> > > CcInitializeCacheMap( Vcb->VirtualVolumeFile,
> > > &FileSizes,
> > > TRUE,
> > > &StData.CacheManagerNoOpCallbacks,
> > > Vcb );
> > >
> > > // this is in the MountVolume routine
> > > try {
> > > StReadVolumeFile( IrpContext,
> > > Vcb,
> > > 0, // Starting Byte
> > > sizeof(DIRECTORY_BLOCK_FORMAT), // byte
> > > count
> > > &VolumeBcb,
> > > (PVOID *)&VolumeBlock );
> > >
> > > } except(EXCEPTION_EXECUTE_HANDLER) {
> > > ExceptionCode = GetExceptionCode();
> > > }
> > >
> > >
> > > // this is basically called from within the StReadVolumeFile routine…
> > >
> > > CcMapData( Vcb->VirtualVolumeFile,
> > > &Vbo, // starting byte
> > > ByteCount, // byte count
> > > BooleanFlagOn(IrpContext->Flags, IRP_CONTEXT_FLAG_WAIT),
> > > Bcb, // ref to volume Bcb
> > > Buffer ) // ref to volume Block
> > >
> > >
> > >
> > > -----Original Message-----
> > > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > > Sent: 12 July 2002 12:39
> > > To: File Systems Developers
> > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > >
> > >
> > > STATUS_IN_PAGE_ERROR is returned in the case the page fault handler
was
> > not
> > > able to read a swaped out memory page from the secondary storage. This
> can
> > > be the result of a many causes , such as:
> > >
> > > 1. Malfunctioning HW , like bad sectors on your HDD
> > > 2. If you write a file system filter, check the paging IO path and
make
> > sure
> > > the logic is error free.
> > > 3. If you write a file system, make sure you handle corectly paging IO
> > logic
> > > for both paging files, and memory mapped files.
> > > For memory mapped files , unless the view is mapped aginst a section
> > backed
> > > by a system pagingfile, the inpage operations will take place directly
> > from
> > > the file backing up the section.
> > >
> > > 4. If you have your own filters on the storage stack, or you have your
> own
> > > scsi miniports in the storage path, make sure
> > > they are corectly written.
> > >
> > > However , it should be easy to isolate the cause.
> > >
> > > As a general rule , when you work with the cache manager, be aware
that
> > > almost all Cc functions can raise exceptions
> > > on error conditions, altough this behaviour is not documented. The
> status
> > > codes returned are very valuable information
> > > to see what went wrong. Keep this in mind when you code with Cc, and
> adopt
> > > defensive programming tactics. Employ
> > > SEH to handle this kind of errors. Just as a side note, dont HIDE them
> > > trough SEH(ugly aint it ?), but actualy handle the situation, one way
or
> > > another
> > >
> > > Reagrds, Dan
> > >
> > >
> > >
> > >
> > > ----- Original Message -----
> > > From: “James Dunning”
> > > To: “File Systems Developers”
> > > Sent: Friday, July 12, 2002 1:51 PM
> > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > >
> > >
> > > > Thanks for the reply…
> > > >
> > > > I have just placed a try{} and except{} wrap around the call to
> > > > CcMapData(…) and you were right, I am getting an exception! I have
> > > managed
> > > > to get the exception code and it returning the following status
code,
> > > > STATUS_IN_PAGE_ERROR 0xC0000006L
> > > >
> > > > which I think means that for some reason, the data is not placed
into
> > > memory
> > > > due to an io error?
> > > >
> > > > Has anyone come across this problem before? and perhaps have an idea
> how
> > > to
> > > > get a better understanding why this error is recurring and resolve
> this
> > > > situation?
> > > >
> > > > Regards
> > > > James
> > > >
> > > >
> > > >
> > > > -----Original Message-----
> > > > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > > > Sent: 11 July 2002 16:33
> > > > To: File Systems Developers
> > > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > > >
> > > >
> > > > As far as I know, this routine has nothing to do with fast IO.
> > > >
> > > > However , your debugger behaves strangley.
> > > > Practice told me that many Cc functions can raise exceptions so Id
> > sugest
> > > to
> > > > have a try{} except{}
> > > > construct wrapping this call and see if you can catch an exception.
> If
> > > yes,
> > > > the status code
> > > > can be enlightning.
> > > >
> > > > Dan
> > > >
> > > > “James Dunning” wrote in
> message
> > > > news:xxxxx@ntfsd…
> > > > >
> > > > > But can CcMapData be called without the use of FastIo?
> > > > >
> > > > > The Wait parameter was actually already set to TRUE, but as i step
> > over
> > > > the
> > > > > CcMapData(…) routine Softice seem to behave rather strangely and
> > fails
> > > > to
> > > > > step to the next routine, it either exits from the softice
debugger
> > and
> > > > goes
> > > > > back to the main windows xp desktop, or to the original break
point
> > just
> > > > > above the point where CcMapData(…) is actually called… either
> > way,
> > > > this
> > > > > is preventing me from examining the buffer returned from
CcMapData!
> > > > >
> > > > > I am wondering if i have missed something out while setting up the
> > cache
> > > > > (like not initializing fastio) or whether there is a problem
> > currupting
> > > > the
> > > > > memory somewhere, which is causing this bizzare behavour.
> > > > >
> > > > > Any advice?
> > > > >
> > > > > Regards,
> > > > > James
> > > > >
> > > > >
> > > > > -----Original Message-----
> > > > > From: Dan Partelly [mailto:xxxxx@rdsor.ro]
> > > > > Sent: 11 July 2002 14:12
> > > > > To: File Systems Developers
> > > > > Subject: [ntfsd] Re: Having problems getting CcMapData to work!..
> > > > >
> > > > >
> > > > > Valid means TRUE , no ? If caching for the stream has been
> > intialized,
> > > > then
> > > > > this routine can be called. This routine will return FALSE if the
> > > mapping
> > > > > request cannot be performed imediately, and the Wait parameter is
> set
> > to
> > > > > FALSE. Set Wait to true, and see if you get the expected result.
> > > > >
> > > > > Dan
> > > > >
> > > > > ----- Original Message -----
> > > > > From: “James Dunning”
> > > > > To: “File Systems Developers”
> > > > > Sent: Thursday, July 11, 2002 3:43 PM
> > > > > Subject: [ntfsd] Having problems getting CcMapData to work!..
> > > > >
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > I am currently having problems getting CcMapData(…) to work
and
> > > return
> > > > > any
> > > > > > valid data. I have debugged the code and the
> > > CcInitializeCacheMap(…)
> > > > > > appears to have initialized the cache map successfully…
> > > > > >
> > > > > > Could anyone tell me if in order for the CcMapData(…) to work,
> do
> > i
> > > > need
> > > > > > to implement and initialise the FastIO entry points? or can i
use
> > > > > CcMapData
> > > > > > without having fastio support?
> > > > > >
> > > > > > Regards,
> > > > > > James…
> > > > > >
> > > > > >
> > > > > > —
> > > > > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > > > > To unsubscribe send a blank email to %%email.unsub%%
> > > > > >
> > > > >
> > > > >
> > > > >
> > > > > —
> > > > > You are currently subscribed to ntfsd as:
> > > > > xxxxx@generaldynamics.uk.com
> > > > > To unsubscribe send a blank email to %%email.unsub%%
> > > > >
> > > > >
> > > > >
> > > >
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as:
> > > > xxxxx@generaldynamics.uk.com
> > > > To unsubscribe send a blank email to %%email.unsub%%
> > > >
> > > >
> > > > —
> > > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > > To unsubscribe send a blank email to %%email.unsub%%
> > > >
> > >
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as:
> > > xxxxx@generaldynamics.uk.com
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> > >
> > > —
> > > You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> > > To unsubscribe send a blank email to %%email.unsub%%
> > >
> >
> >
> >
> > —
> > You are currently subscribed to ntfsd as:
> > xxxxx@generaldynamics.uk.com
> > To unsubscribe send a blank email to %%email.unsub%%
> >
> >
> >
>
>
>
> —
> You are currently subscribed to ntfsd as:
> xxxxx@generaldynamics.uk.com
> To unsubscribe send a blank email to %%email.unsub%%
>
>
> —
> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to %%email.unsub%%
>
—
You are currently subscribed to ntfsd as:
xxxxx@generaldynamics.uk.com
To unsubscribe send a blank email to %%email.unsub%%