Stuck with 'Hello World' file system driver (long)

Hello everyone!

I hope for some help or a bit of advice for my “Hello World” file
system driver (either a fix to make it work or a hint how to debug it).
Maybe
it’s something obvious that someone out there spots instantly :slight_smile:

Current status:

The driver creates a drive letter M: and supports a single hard
coded (“write protected”) file named HelloWorld with a size of 5 bytes.
The contents (“YES!!”) can be accessed with normal file access
(for example, with cmd.exe’s “type M:\HelloWorld”). The file can be
copied off the volume with explorer, or just by using copy.exe or
Windows 2000 backup, so it seems that the basic stuff is working fine.

Now I want to support memory mapped files as well, so I’ve added the
four cache manager callbacks, and three fast IO calls [IsFastIoPossible,
read and write], added section object pointers to my hello-world-FCB
(I was not entirely sure what is really needed) and copied some lines
of code over to VfsDispatchRead() in my test driver:

if( CachedIO && !pFile->PrivateCacheMap ) {
CcInitializeCacheMap( pFile,
(PCC_FILE_SIZES)(&pDevExt->…AllocationSize), FALSE,
&pDevExt->CacheMgrCallBacks,

&pDevExt->world_fcb );
}

// if not MDL read:

if( !CcCopyRead( pFile, (PLARGE_INTEGER)&ccb->CurrentByteOffset,
(ULONG)len, FALSE, (PVOID)SystemBuffer,

&pIrp->IoStatus )) {

}

To me this looks like all the examples I have seen so far. I did expect
some recursive
call back to my driver with the CcCopyRead, but if the CcCopyRead gets
executed,
then I just get this error:

Unknown exception - code c00002e8 (!!! second chance !!!)
nt!CcCopyRead+0x5f2:

In ntstatus.h I found STATUS_MULTIPLE_FAULT_VIOLATION with that code.

If I continue with the execution (‘gh’ in windbg) then IoCompleteRequest()
(a few lines of code later in my driver) bugchecks, “multiple IRP
completion”.

The next test I’ve done is to step through CcCopyRead. It has been a
couple
years since I looked at assembler (and back then there were no fancy
registers like eax) but I’ve found the offending command. Unfortunately,
I have no idea what to do now, or how to find the *real* error behind all
this.

Any help to solve this problem would be greatly appreciated!

I’ve added some information below, I hope this helps to track down the
problem
or does ‘ring some bells’… Oh, I don’t have much experience with windbg,
as I rely usually on KdPrint() statements and I got really confused when,
for example, KdPrint((“…bla %p”, pIrp)) shows a different value than the
‘locals’-window of Windbg…

Thanks a lot for reading all this. I’d try anything including dancing
naked
around a fire at midnight if it helps to solve that problem :–)

Cheers,
Michael B.

------- hopefully helpful stuff below ---------
Computer:
DELL 670 [runs the driver] DELL 8100 [runs Windbg]
Windos 2000, SP4 [on both machines]
Windbg 6.4.0004.3

What happens from the user side:
opentst.exe, calls CreateFile() to open M:\HelloWorld, then
CreateFileMapping()
and MapViewOfFile() as usual to get a memory pointer to the ‘mapped’ file.
This succeeds, but when I try to access the data using that memory (char
*)
pointer I trigger the problem within the CcCopyRead() function.

What happens from the driver side:

  • driver gets loaded (DriverEntry())
  • Create() for ''
  • Cleanup() for ''
  • Close() for ''
  • Create() for ‘\HelloWorld’
  • QueryInformation() Type=FileStandardInformation
  • QueryInformation() Type=FileStandardInformation (yes, twice)
  • Read() at offset 0 length 4096

DispatchRead() with more details:

Stats of the file:
AllocationSize: 5
FileSize: 5
ValidDataLength: 5

CcInitializeCacheMap() gets called first, it’s no MDL read, and then
CcCopyRead():

CcCopyRead parameters: CurrentByteOffset=0, Length=5,
SystemBuffer=975D8000 (for example).
The strange part is, the last KdPrint() of “pIrp->IoStatus” (%p) shows a
different value
than what is actually pushed on the stack or displayed in the locals
window, while pFile
(PFILE_OBJECT) is consistent with KdPrint(“%p”) and the ‘locals’ window of
Windbg.

Anyway, when I step through CcCopyRead() the following calls are executed:
nt!CcPinFileData (eax is 1 on return, in case that matters?)
nt!MmCheckCachedPageState (eax is 1 again. Sorry I don’t know what to look
for)

A few assembler commands before the trouble starts:

nt!CcCopyRead+0x559:
804e2355 8b7d18 mov edi,[ebp+0x18]
kd> p
nt!CcCopyRead+0x55c:
804e2358 8bc1 mov eax,ecx
kd> p
nt!CcCopyRead+0x55e:
804e235a c1e902 shr ecx,0x2 // ecx is 1 then, possibly 2
DWORDS for the 5 bytes-to-copy?
kd> p
nt!CcCopyRead+0x561:
804e235d f3a5 rep movsd

*** This is the instruction that causes the problems.
I think this tries to copy some DWORDS over from ESI to EDI which would
be 0xCF040000 to 0x975D8000 for example [the last pointer is the
SystemBuffer that I got in the DispatchRead()]. I can access the
SystemBuffer-data area from WinDbg, but not the CF040000:
Unable to retrieve information, NTSTATUS 0xC0000001

Some general stuff that might be relevant:

The file system driver does not mount over a disk or other device
object, it just exists, but it has to lie to the system and declare
as DISK_FILE_SYSTEM instead of just FILE_SYSTEM, otherwise I run
into problems. Also, this means I don’t have a Vpb pointer (it’s NULL).

I have initialized ERESOURCE’s (main, paging) for each FCB but I don’t
use them (it’s all read-only anyway and no calls to other drivers
down any driver stack, with the file contents being hard-coded).

There is no worker thread, I complete each and every request synchronous.

Storage for FCB and CCB’s are hard coded and stored within the
device extension (I allow two CCB’s for each FCB). This is just to
simplify my debugging. It’s a hello-world-driver only anyway :slight_smile:

I don’t worry/care about sharing flags. I’ve messed with pFile within
the DispatchRead() and modified those values a bit, but this didn’t
change the behaviour of the CcCopyRead problem.


Vogon International GmbH
http://www.vogon-international.com

In your file object, did you set your FsContext structure to contain the
FSRTL_ADVANCED_FCB_HEADER (or at least the common FCB header?) If not,
that can generate a series of problems.

Some of the steps you described are not necessary - you don’t need any
of the Fast I/O routines, since if they aren’t present there are default
actions that are taken anyway. But nothing you described should cause a
problem at this stage.

Your file system allocates the section object pointers structure (you
allocated it, Cc and Mm use it). Memory should be non-paged and
scrubbed (zero filled). The CcCopyRead should trigger (the first time)
a second IRP_MJ_READ but this time with the IRP_PAGING_IO bit set (also
IRP_NOCACHE, but all paging I/O is non-cached). It doesn’t sound like
you even get to the second (reentrant) read operation.

But to support *memory mapped* files you don’t even need to integrate
with the cache. All you need to do is allocate the section object
pointers structure in your file object. With that done, Mm has
everything it needs to memory map. Cache Manager integration is not a
requirement for supporting memory mapped files. Further, since your
files never change, there is no coherency problem (between the memory
mapped copy and the copy on disk).

I hope this helps.

Regards,

Tony

Tony Mason

Consulting Partner

OSR Open Systems Resources, Inc.

http://www.osr.com http:</http:>


From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@vogon.de
Sent: Tuesday, March 01, 2005 7:02 AM
To: ntfsd redirect
Subject: [ntfsd] Stuck with ‘Hello World’ file system driver (long)

Hello everyone!

I hope for some help or a bit of advice for my “Hello World” file
system driver (either a fix to make it work or a hint how to debug it).
Maybe
it’s something obvious that someone out there spots instantly :slight_smile:

Current status:

The driver creates a drive letter M: and supports a single hard
coded (“write protected”) file named HelloWorld with a size of 5 bytes.
The contents (“YES!!”) can be accessed with normal file access
(for example, with cmd.exe’s “type M:\HelloWorld”). The file can be
copied off the volume with explorer, or just by using copy.exe or
Windows 2000 backup, so it seems that the basic stuff is working fine.

Now I want to support memory mapped files as well, so I’ve added the
four cache manager callbacks, and three fast IO calls [IsFastIoPossible,

read and write], added section object pointers to my hello-world-FCB
(I was not entirely sure what is really needed) and copied some lines
of code over to VfsDispatchRead() in my test driver:

if( CachedIO && !pFile->PrivateCacheMap ) {
CcInitializeCacheMap( pFile,
(PCC_FILE_SIZES)(&pDevExt->…AllocationSize), FALSE,
&pDevExt->CacheMgrCallBacks,

&pDevExt->world_fcb );
}

// if not MDL read:

if( !CcCopyRead( pFile, (PLARGE_INTEGER)&ccb->CurrentByteOffset,
(ULONG)len, FALSE, (PVOID)SystemBuffer,

&pIrp->IoStatus )) {

}

To me this looks like all the examples I have seen so far. I did expect
some recursive
call back to my driver with the CcCopyRead, but if the CcCopyRead gets
executed,
then I just get this error:

Unknown exception - code c00002e8 (!!! second chance !!!)
nt!CcCopyRead+0x5f2:

In ntstatus.h I found STATUS_MULTIPLE_FAULT_VIOLATION with that code.

If I continue with the execution (‘gh’ in windbg) then
IoCompleteRequest()
(a few lines of code later in my driver) bugchecks, “multiple IRP
completion”.

The next test I’ve done is to step through CcCopyRead. It has been a
couple
years since I looked at assembler (and back then there were no fancy
registers like eax) but I’ve found the offending command. Unfortunately,

I have no idea what to do now, or how to find the *real* error behind
all this.

Any help to solve this problem would be greatly appreciated!

I’ve added some information below, I hope this helps to track down the
problem
or does ‘ring some bells’… Oh, I don’t have much experience with
windbg,
as I rely usually on KdPrint() statements and I got really confused
when,
for example, KdPrint((“…bla %p”, pIrp)) shows a different value than
the
‘locals’-window of Windbg…

Thanks a lot for reading all this. I’d try anything including dancing
naked
around a fire at midnight if it helps to solve that problem :–)

Cheers,
Michael B.

------- hopefully helpful stuff below ---------
Computer:
DELL 670 [runs the driver] DELL 8100 [runs Windbg]
Windos 2000, SP4 [on both machines]
Windbg 6.4.0004.3

What happens from the user side:
opentst.exe, calls CreateFile() to open M:\HelloWorld, then
CreateFileMapping()
and MapViewOfFile() as usual to get a memory pointer to the ‘mapped’
file.
This succeeds, but when I try to access the data using that memory (char
*)
pointer I trigger the problem within the CcCopyRead() function.

What happens from the driver side:

  • driver gets loaded (DriverEntry())
  • Create() for ''
  • Cleanup() for ''
  • Close() for ''
  • Create() for ‘\HelloWorld’
  • QueryInformation() Type=FileStandardInformation
  • QueryInformation() Type=FileStandardInformation (yes, twice)
  • Read() at offset 0 length 4096

DispatchRead() with more details:

Stats of the file:
AllocationSize: 5
FileSize: 5
ValidDataLength: 5

CcInitializeCacheMap() gets called first, it’s no MDL read, and then
CcCopyRead():

CcCopyRead parameters: CurrentByteOffset=0, Length=5,
SystemBuffer=975D8000 (for example).
The strange part is, the last KdPrint() of “pIrp->IoStatus” (%p) shows a
different value
than what is actually pushed on the stack or displayed in the locals
window, while pFile
(PFILE_OBJECT) is consistent with KdPrint(“%p”) and the ‘locals’ window
of Windbg.

Anyway, when I step through CcCopyRead() the following calls are
executed:
nt!CcPinFileData (eax is 1 on return, in case that matters?)
nt!MmCheckCachedPageState (eax is 1 again. Sorry I don’t know what to
look for)

A few assembler commands before the trouble starts:

nt!CcCopyRead+0x559:
804e2355 8b7d18 mov edi,[ebp+0x18]
kd> p
nt!CcCopyRead+0x55c:
804e2358 8bc1 mov eax,ecx
kd> p
nt!CcCopyRead+0x55e:
804e235a c1e902 shr ecx,0x2 // ecx is 1 then, possibly
2 DWORDS for the 5 bytes-to-copy?
kd> p
nt!CcCopyRead+0x561:
804e235d f3a5 rep movsd

*** This is the instruction that causes the problems.
I think this tries to copy some DWORDS over from ESI to EDI which would
be 0xCF040000 to 0x975D8000 for example [the last pointer is the
SystemBuffer that I got in the DispatchRead()]. I can access the
SystemBuffer-data area from WinDbg, but not the CF040000:
Unable to retrieve information, NTSTATUS 0xC0000001

Some general stuff that might be relevant:

The file system driver does not mount over a disk or other device
object, it just exists, but it has to lie to the system and declare
as DISK_FILE_SYSTEM instead of just FILE_SYSTEM, otherwise I run
into problems. Also, this means I don’t have a Vpb pointer (it’s NULL).

I have initialized ERESOURCE’s (main, paging) for each FCB but I don’t
use them (it’s all read-only anyway and no calls to other drivers
down any driver stack, with the file contents being hard-coded).

There is no worker thread, I complete each and every request
synchronous.

Storage for FCB and CCB’s are hard coded and stored within the
device extension (I allow two CCB’s for each FCB). This is just to
simplify my debugging. It’s a hello-world-driver only anyway :slight_smile:

I don’t worry/care about sharing flags. I’ve messed with pFile within
the DispatchRead() and modified those values a bit, but this didn’t
change the behaviour of the CcCopyRead problem.


Vogon International GmbH
http://www.vogon-international.com — Questions? First check the IFS
FAQ at https://www.osronline.com/article.cfm?id=17 You are currently
subscribed to ntfsd as: unknown lmsubst tag argument: ‘’ To unsubscribe
send a blank email to xxxxx@lists.osr.com

Hi,

Thanks a lot for your reply!

“Tony Mason” wrote on 01.03.2005 13:38:27:
> In your file object, did you set your FsContext structure to contain
> the FSRTL_ADVANCED_FCB_HEADER (or at least the common FCB header?)
> If not, that can generate a series of problems.

FsContext is a pointer to a structure where the first entry is the
common fcb header, so I hope there are no problems there :slight_smile: I never
knew there is an advanced fcb header…

> Some of the steps you described are not necessary ? you don?t need
> any of the Fast I/O routines, since if they aren?t present there are
> default actions that are taken anyway. But nothing you described
> should cause a problem at this stage.

> Your file system allocates the section object pointers structure
> (you allocated it, Cc and Mm use it). Memory should be non-paged
> and scrubbed (zero filled). The CcCopyRead should trigger (the
> first time) a second IRP_MJ_READ but this time with the
> IRP_PAGING_IO bit set (also IRP_NOCACHE, but all paging I/O is non-
> cached). It doesn?t sound like you even get to the second
> (reentrant) read operation.

> But to support memory mapped files you don?t even need to
> integrate with the cache. […]

That did clarify+help a lot! After removing fast i/o and the Cc-calls with
just leaving section object pointers in place, it worked! Except for
one call that I had to add if the last CCB received a cleanup call:
MmForceSectionClosed( fcb->SectionObjectPointer, FALSE );

Is this necessary, or is the requirement of this line a hint that
I might have messed up something somewhere else?

Now this leaves me with the puzzling problem as to why
CcCopyRead() crashes, that’s a bit worrying… But then I probably
won’t need caching.

Btw, thanks to you I have now a working [hopefully, still doing extensive
testing] Hello World file system driver :slight_smile: And it was just 4 years after
I first had the idea/wish to create it.

Cheers,
Michael B.

---------------------------------------------------------------
Vogon International GmbH
http://www.vogon-international.com

I’m not sure why you *had* to add the MmForceSectionClosed. It shouldn’t have crashed if you left this out, although it will cause the data to remain in memory until the memory needs to be reused for something else, which of course keeps an added reference around to your file object.

FsContext is whatever you want. However, once you start interacting with Mm and Cc it expects one of two formats: either the common or the advanced FCB header. The advanced header has additional state available for filter drivers to use.

Integration with Cc is a bit more work - Mm is happy just having the SOP to use, but Cc needs quite a bit more state set up properly. Given the information you’ve provided I can’t suggest why you are having Cc problems. But you really don’t need them in the file system that you just described.

Four years is not so bad - there are people on this list who have been reading at least as long and are still trying to get their file system to work! :wink:

Best of luck with your file system project!

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com


From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@vogon.de
Sent: Tuesday, March 01, 2005 9:54 AM
To: ntfsd redirect
Subject: RE: [ntfsd] Stuck with ‘Hello World’ file system driver (long)

Hi,

Thanks a lot for your reply!

“Tony Mason” wrote on 01.03.2005 13:38:27:
> In your file object, did you set your FsContext structure to contain
> the FSRTL_ADVANCED_FCB_HEADER (or at least the common FCB header?)?
> If not, that can generate a series of problems.

FsContext is a pointer to a structure where the first entry is the
common fcb header, so I hope there are no problems there :slight_smile: I never
knew there is an advanced fcb header…

> Some of the steps you described are not necessary - you don’t need
> any of the Fast I/O routines, since if they aren’t present there are
> default actions that are taken anyway.? But nothing you described
> should cause a problem at this stage.

> Your file system allocates the section object pointers structure
> (you allocated it, Cc and Mm use it).? Memory should be non-paged
> and scrubbed (zero filled).? The CcCopyRead should trigger (the
> first time) a second IRP_MJ_READ but this time with the
> IRP_PAGING_IO bit set (also IRP_NOCACHE, but all paging I/O is non-
> cached).? It doesn’t sound like you even get to the second
> (reentrant) read operation.

> But to support memory mapped files you don’t even need to
> integrate with the cache. […]

That did clarify+help a lot! After removing fast i/o and the Cc-calls with
just leaving section object pointers in place, it worked! Except for
one call that I had to add if the last CCB received a cleanup call:
? ?MmForceSectionClosed( fcb->SectionObjectPointer, FALSE );

Is this necessary, or is the requirement of this line a hint that
I might have messed up something somewhere else?

Now this leaves me with the puzzling problem as to why
CcCopyRead() crashes, that’s a bit worrying… But then I probably
won’t need caching.

Btw, thanks to you I have now a working [hopefully, still doing extensive
testing] Hello World file system driver :slight_smile: And it was just 4 years after
I first had the idea/wish to create it.

Cheers,
Michael B.

---------------------------------------------------------------
Vogon International GmbH
http://www.vogon-international.com — Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17 You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’ To unsubscribe send a blank email to xxxxx@lists.osr.com

Tony Mason,???ã?

???ҳ?޸???

======= 2005-03-02 08:27:25 ???д???=======

I’m not sure why you *had* to add the MmForceSectionClosed. It shouldn’t have crashed if you left this out, although it will cause the data to remain in memory until the memory needs to be reused for something else, which of course keeps an added reference around to your file object.

FsContext is whatever you want. However, once you start interacting with Mm and Cc it expects one of two formats: either the common or the advanced FCB header. The advanced header has additional state available for filter drivers to use.

Integration with Cc is a bit more work - Mm is happy just having the SOP to use, but Cc needs quite a bit more state set up properly. Given the information you’ve provided I can’t suggest why you are having Cc problems. But you really don’t need them in the file system that you just described.

Four years is not so bad - there are people on this list who have been reading at least as long and are still trying to get their file system to work! :wink:

Best of luck with your file system project!

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com


From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@vogon.de
Sent: Tuesday, March 01, 2005 9:54 AM
To: ntfsd redirect
Subject: RE: [ntfsd] Stuck with ‘Hello World’ file system driver (long)

Hi,

Thanks a lot for your reply!

“Tony Mason” wrote on 01.03.2005 13:38:27:
>> In your file object, did you set your FsContext structure to contain
>> the FSRTL_ADVANCED_FCB_HEADER (or at least the common FCB header?)?
>> If not, that can generate a series of problems.
>
>FsContext is a pointer to a structure where the first entry is the
>common fcb header, so I hope there are no problems there :slight_smile: I never
>knew there is an advanced fcb header…
>
>
>> Some of the steps you described are not necessary - you don’t need
>> any of the Fast I/O routines, since if they aren’t present there are
>> default actions that are taken anyway.?But nothing you described
>> should cause a problem at this stage.
>
>> Your file system allocates the section object pointers structure
>> (you allocated it, Cc and Mm use it).?Memory should be non-paged
>> and scrubbed (zero filled).?The CcCopyRead should trigger (the
>> first time) a second IRP_MJ_READ but this time with the
>> IRP_PAGING_IO bit set (also IRP_NOCACHE, but all paging I/O is non-
>> cached).?It doesn’t sound like you even get to the second
>> (reentrant) read operation.
>
>> But to support memory mapped files you don’t even need to
>> integrate with the cache. […]
>
>
>That did clarify+help a lot! After removing fast i/o and the Cc-calls with
>just leaving section object pointers in place, it worked! Except for
>one call that I had to add if the last CCB received a cleanup call:
>??MmForceSectionClosed( fcb->SectionObjectPointer, FALSE );
>
>Is this necessary, or is the requirement of this line a hint that
>I might have messed up something somewhere else?
>
>
>Now this leaves me with the puzzling problem as to why
>CcCopyRead() crashes, that’s a bit worrying… But then I probably
>won’t need caching.
>
>
>Btw, thanks to you I have now a working [hopefully, still doing extensive
>testing] Hello World file system driver :slight_smile: And it was just 4 years after
>I first had the idea/wish to create it.
>
>Cheers,
>Michael B.
>
>
>---------------------------------------------------------------
>Vogon International GmbH
>http://www.vogon-international.com — Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17 You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’ To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>—
>Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
>To unsubscribe send a blank email to xxxxx@lists.osr.com

= = = = = = = = = = = = = = = = = = = =

???
???

???znsoft
???xxxxx@zndev.com
???2005-03-02

Through the modern miracle of Google, I tried to translate your message (http://www.worldlingo.com/en/products_services/worldlingo_translator.html) It said:

“Tony Mason,Hello!
To home page revision mailbox”

Unfortunately, I’m not certain what your message means.

(And now the reverse translation of my message via the service, so I hope this does not translate into some horrible insult):

!!!
?Ҳ??˽???ĵ???ʼ???

:slight_smile:

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of znsoft
Sent: Tuesday, March 01, 2005 8:24 PM
To: ntfsd redirect
Subject: Re: RE: [ntfsd] Stuck with ‘Hello World’ file system driver (long)

Tony Mason,???ã?

???ҳ?޸???

======= 2005-03-02 08:27:25 ???д???=======

I’m not sure why you *had* to add the MmForceSectionClosed. It shouldn’t have crashed if you left this out, although it will cause the data to remain in memory until the memory needs to be reused for something else, which of course keeps an added reference around to your file object.

FsContext is whatever you want. However, once you start interacting with Mm and Cc it expects one of two formats: either the common or the advanced FCB header. The advanced header has additional state available for filter drivers to use.

Integration with Cc is a bit more work - Mm is happy just having the SOP to use, but Cc needs quite a bit more state set up properly. Given the information you’ve provided I can’t suggest why you are having Cc problems. But you really don’t need them in the file system that you just described.

Four years is not so bad - there are people on this list who have been reading at least as long and are still trying to get their file system to work! :wink:

Best of luck with your file system project!

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com


From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@vogon.de
Sent: Tuesday, March 01, 2005 9:54 AM
To: ntfsd redirect
Subject: RE: [ntfsd] Stuck with ‘Hello World’ file system driver (long)

Hi,

Thanks a lot for your reply!

“Tony Mason” wrote on 01.03.2005 13:38:27:
>> In your file object, did you set your FsContext structure to contain
>> the FSRTL_ADVANCED_FCB_HEADER (or at least the common FCB header?)?
>> If not, that can generate a series of problems.
>
>FsContext is a pointer to a structure where the first entry is the
>common fcb header, so I hope there are no problems there :slight_smile: I never
>knew there is an advanced fcb header…
>
>
>> Some of the steps you described are not necessary - you don’t need
>> any of the Fast I/O routines, since if they aren’t present there are
>> default actions that are taken anyway.?But nothing you described
>> should cause a problem at this stage.
>
>> Your file system allocates the section object pointers structure
>> (you allocated it, Cc and Mm use it).?Memory should be non-paged
>> and scrubbed (zero filled).?The CcCopyRead should trigger (the
>> first time) a second IRP_MJ_READ but this time with the
>> IRP_PAGING_IO bit set (also IRP_NOCACHE, but all paging I/O is non-
>> cached).?It doesn’t sound like you even get to the second
>> (reentrant) read operation.
>
>> But to support memory mapped files you don’t even need to
>> integrate with the cache. […]
>
>
>That did clarify+help a lot! After removing fast i/o and the Cc-calls with
>just leaving section object pointers in place, it worked! Except for
>one call that I had to add if the last CCB received a cleanup call:
>??MmForceSectionClosed( fcb->SectionObjectPointer, FALSE );
>
>Is this necessary, or is the requirement of this line a hint that
>I might have messed up something somewhere else?
>
>
>Now this leaves me with the puzzling problem as to why
>CcCopyRead() crashes, that’s a bit worrying… But then I probably
>won’t need caching.
>
>
>Btw, thanks to you I have now a working [hopefully, still doing extensive
>testing] Hello World file system driver :slight_smile: And it was just 4 years after
>I first had the idea/wish to create it.
>
>Cheers,
>Michael B.
>
>
>---------------------------------------------------------------
>Vogon International GmbH
>http://www.vogon-international.com — Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17 You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’ To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>—
>Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
>To unsubscribe send a blank email to xxxxx@lists.osr.com

= = = = = = = = = = = = = = = = = = = =

???
???

???znsoft
???xxxxx@zndev.com
???2005-03-02


Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to $subst(‘List.Name’) as: $subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to $subst(‘Email.UnSub’)

Excellent translation and answer :slight_smile:

Regards,
ZG [@ Sydney]
Windows Driver Developer
[Remove ‘stopspam’ & ‘nospam’ when you send email to me]

“Tony Mason” wrote in message news:xxxxx@ntfsd…
Through the modern miracle of Google, I tried to translate your message
(http://www.worldlingo.com/en/products_services/worldlingo_translator.html)
It said:

“Tony Mason,Hello!
To home page revision mailbox”

Unfortunately, I’m not certain what your message means.

(And now the reverse translation of my message via the service, so I hope
this does not translate into some horrible insult):

??!
???

:slight_smile:

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com]
On Behalf Of znsoft
Sent: Tuesday, March 01, 2005 8:24 PM
To: ntfsd redirect
Subject: Re: RE: [ntfsd] Stuck with ‘Hello World’ file system driver (long)

Tony Mason,??!

???

======= 2005-03-02 08:27:25 ???:=======

>I’m not sure why you had to add the MmForceSectionClosed. It shouldn’t
>have crashed if you left this out, although it will cause the data to
>remain in memory until the memory needs to be reused for something else,
>which of course keeps an added reference around to your file object.
>
>FsContext is whatever you want. However, once you start interacting with
>Mm and Cc it expects one of two formats: either the common or the advanced
>FCB header. The advanced header has additional state available for filter
>drivers to use.
>
>Integration with Cc is a bit more work - Mm is happy just having the SOP to
>use, but Cc needs quite a bit more state set up properly. Given the
>information you’ve provided I can’t suggest why you are having Cc problems.
>But you really don’t need them in the file system that you just described.
>
>Four years is not so bad - there are people on this list who have been
>reading at least as long and are still trying to get their file system to
>work! :wink:
>
>Best of luck with your file system project!
>
>Regards,
>
>Tony
>
>Tony Mason
>Consulting Partner
>OSR Open Systems Resources, Inc.
>http://www.osr.com
>
>
> ________________________________________
>From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com]
>On Behalf Of xxxxx@vogon.de
>Sent: Tuesday, March 01, 2005 9:54 AM
>To: ntfsd redirect
>Subject: RE: [ntfsd] Stuck with ‘Hello World’ file system driver (long)
>
>
>
>Hi,
>
>Thanks a lot for your reply!
>
>
>“Tony Mason” wrote on 01.03.2005 13:38:27:
>> In your file object, did you set your FsContext structure to contain
>> the FSRTL_ADVANCED_FCB_HEADER (or at least the common FCB header?)?
>> If not, that can generate a series of problems.
>
>FsContext is a pointer to a structure where the first entry is the
>common fcb header, so I hope there are no problems there :slight_smile: I never
>knew there is an advanced fcb header…
>
>
>> Some of the steps you described are not necessary - you don’t need
>> any of the Fast I/O routines, since if they aren’t present there are
>> default actions that are taken anyway.?But nothing you described
>> should cause a problem at this stage.
>
>> Your file system allocates the section object pointers structure
>> (you allocated it, Cc and Mm use it).?Memory should be non-paged
>> and scrubbed (zero filled).?The CcCopyRead should trigger (the
>> first time) a second IRP_MJ_READ but this time with the
>> IRP_PAGING_IO bit set (also IRP_NOCACHE, but all paging I/O is non-
>> cached).?It doesn’t sound like you even get to the second
>> (reentrant) read operation.
>
>> But to support memory mapped files you don’t even need to
>> integrate with the cache. […]
>
>
>That did clarify+help a lot! After removing fast i/o and the Cc-calls with
>just leaving section object pointers in place, it worked! Except for
>one call that I had to add if the last CCB received a cleanup call:
>??mForceSectionClosed( fcb->SectionObjectPointer, FALSE );
>
>Is this necessary, or is the requirement of this line a hint that
>I might have messed up something somewhere else?
>
>
>Now this leaves me with the puzzling problem as to why
>CcCopyRead() crashes, that’s a bit worrying… But then I probably
>won’t need caching.
>
>
>Btw, thanks to you I have now a working [hopefully, still doing extensive
>testing] Hello World file system driver :slight_smile: And it was just 4 years after
>I first had the idea/wish to create it.
>
>Cheers,
>Michael B.
>
>
>---------------------------------------------------------------
>Vogon International GmbH
>http://www.vogon-international.com — Questions? First check the IFS FAQ
>at https://www.osronline.com/article.cfm?id=17 You are currently subscribed
>to ntfsd as: unknown lmsubst tag argument: ‘’ To unsubscribe send a blank
>email to xxxxx@lists.osr.com
>
>—
>Questions? First check the IFS FAQ at
>https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
>To unsubscribe send a blank email to xxxxx@lists.osr.com

= = = = = = = = = = = = = = = = = = = =

???
?!

???znsoft
???xxxxx@zndev.com
???2005-03-02


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to $subst(‘List.Name’) as:
$subst(‘Recip.EmailAddr’)
To unsubscribe send a blank email to $subst(‘Email.UnSub’)

The file size query must also be supported for memory-mapped files to work.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: Tony Mason
To: Windows File Systems Devs Interest List
Sent: Tuesday, March 01, 2005 3:38 PM
Subject: RE: [ntfsd] Stuck with ‘Hello World’ file system driver (long)

In your file object, did you set your FsContext structure to contain the FSRTL_ADVANCED_FCB_HEADER (or at least the common FCB header?) If not, that can generate a series of problems.

Some of the steps you described are not necessary - you don’t need any of the Fast I/O routines, since if they aren’t present there are default actions that are taken anyway. But nothing you described should cause a problem at this stage.

Your file system allocates the section object pointers structure (you allocated it, Cc and Mm use it). Memory should be non-paged and scrubbed (zero filled). The CcCopyRead should trigger (the first time) a second IRP_MJ_READ but this time with the IRP_PAGING_IO bit set (also IRP_NOCACHE, but all paging I/O is non-cached). It doesn’t sound like you even get to the second (reentrant) read operation.

But to support *memory mapped* files you don’t even need to integrate with the cache. All you need to do is allocate the section object pointers structure in your file object. With that done, Mm has everything it needs to memory map. Cache Manager integration is not a requirement for supporting memory mapped files. Further, since your files never change, there is no coherency problem (between the memory mapped copy and the copy on disk).

I hope this helps.

Regards,

Tony

Tony Mason

Consulting Partner

OSR Open Systems Resources, Inc.

http://www.osr.com


From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@vogon.de
Sent: Tuesday, March 01, 2005 7:02 AM
To: ntfsd redirect
Subject: [ntfsd] Stuck with ‘Hello World’ file system driver (long)

Hello everyone!

I hope for some help or a bit of advice for my “Hello World” file
system driver (either a fix to make it work or a hint how to debug it). Maybe
it’s something obvious that someone out there spots instantly :slight_smile:

Current status:

The driver creates a drive letter M: and supports a single hard
coded (“write protected”) file named HelloWorld with a size of 5 bytes.
The contents (“YES!!”) can be accessed with normal file access
(for example, with cmd.exe’s “type M:\HelloWorld”). The file can be
copied off the volume with explorer, or just by using copy.exe or
Windows 2000 backup, so it seems that the basic stuff is working fine.

Now I want to support memory mapped files as well, so I’ve added the
four cache manager callbacks, and three fast IO calls [IsFastIoPossible,
read and write], added section object pointers to my hello-world-FCB
(I was not entirely sure what is really needed) and copied some lines
of code over to VfsDispatchRead() in my test driver:

if( CachedIO && !pFile->PrivateCacheMap ) {
CcInitializeCacheMap( pFile, (PCC_FILE_SIZES)(&pDevExt->…AllocationSize), FALSE, &pDevExt->CacheMgrCallBacks,

&pDevExt->world_fcb );
}

// if not MDL read:

if( !CcCopyRead( pFile, (PLARGE_INTEGER)&ccb->CurrentByteOffset, (ULONG)len, FALSE, (PVOID)SystemBuffer,

&pIrp->IoStatus )) {

}

To me this looks like all the examples I have seen so far. I did expect some recursive
call back to my driver with the CcCopyRead, but if the CcCopyRead gets executed,
then I just get this error:

Unknown exception - code c00002e8 (!!! second chance !!!)
nt!CcCopyRead+0x5f2:

In ntstatus.h I found STATUS_MULTIPLE_FAULT_VIOLATION with that code.

If I continue with the execution (‘gh’ in windbg) then IoCompleteRequest()
(a few lines of code later in my driver) bugchecks, “multiple IRP completion”.

The next test I’ve done is to step through CcCopyRead. It has been a couple
years since I looked at assembler (and back then there were no fancy
registers like eax) but I’ve found the offending command. Unfortunately,
I have no idea what to do now, or how to find the *real* error behind all this.

Any help to solve this problem would be greatly appreciated!

I’ve added some information below, I hope this helps to track down the problem
or does ‘ring some bells’… Oh, I don’t have much experience with windbg,
as I rely usually on KdPrint() statements and I got really confused when,
for example, KdPrint((“…bla %p”, pIrp)) shows a different value than the
‘locals’-window of Windbg…

Thanks a lot for reading all this. I’d try anything including dancing naked
around a fire at midnight if it helps to solve that problem :–)

Cheers,
Michael B.

------- hopefully helpful stuff below ---------
Computer:
DELL 670 [runs the driver] DELL 8100 [runs Windbg]
Windos 2000, SP4 [on both machines]
Windbg 6.4.0004.3

What happens from the user side:
opentst.exe, calls CreateFile() to open M:\HelloWorld, then CreateFileMapping()
and MapViewOfFile() as usual to get a memory pointer to the ‘mapped’ file.
This succeeds, but when I try to access the data using that memory (char *)
pointer I trigger the problem within the CcCopyRead() function.

What happens from the driver side:

  • driver gets loaded (DriverEntry())
  • Create() for ''
  • Cleanup() for ''
  • Close() for ''
  • Create() for ‘\HelloWorld’
  • QueryInformation() Type=FileStandardInformation
  • QueryInformation() Type=FileStandardInformation (yes, twice)
  • Read() at offset 0 length 4096

DispatchRead() with more details:

Stats of the file:
AllocationSize: 5
FileSize: 5
ValidDataLength: 5

CcInitializeCacheMap() gets called first, it’s no MDL read, and then CcCopyRead():

CcCopyRead parameters: CurrentByteOffset=0, Length=5, SystemBuffer=975D8000 (for example).
The strange part is, the last KdPrint() of “pIrp->IoStatus” (%p) shows a different value
than what is actually pushed on the stack or displayed in the locals window, while pFile
(PFILE_OBJECT) is consistent with KdPrint(“%p”) and the ‘locals’ window of Windbg.

Anyway, when I step through CcCopyRead() the following calls are executed:
nt!CcPinFileData (eax is 1 on return, in case that matters?)
nt!MmCheckCachedPageState (eax is 1 again. Sorry I don’t know what to look for)

A few assembler commands before the trouble starts:

nt!CcCopyRead+0x559:
804e2355 8b7d18 mov edi,[ebp+0x18]
kd> p
nt!CcCopyRead+0x55c:
804e2358 8bc1 mov eax,ecx
kd> p
nt!CcCopyRead+0x55e:
804e235a c1e902 shr ecx,0x2 // ecx is 1 then, possibly 2 DWORDS for the 5 bytes-to-copy?
kd> p
nt!CcCopyRead+0x561:
804e235d f3a5 rep movsd

*** This is the instruction that causes the problems.
I think this tries to copy some DWORDS over from ESI to EDI which would
be 0xCF040000 to 0x975D8000 for example [the last pointer is the
SystemBuffer that I got in the DispatchRead()]. I can access the
SystemBuffer-data area from WinDbg, but not the CF040000:
Unable to retrieve information, NTSTATUS 0xC0000001

Some general stuff that might be relevant:

The file system driver does not mount over a disk or other device
object, it just exists, but it has to lie to the system and declare
as DISK_FILE_SYSTEM instead of just FILE_SYSTEM, otherwise I run
into problems. Also, this means I don’t have a Vpb pointer (it’s NULL).

I have initialized ERESOURCE’s (main, paging) for each FCB but I don’t
use them (it’s all read-only anyway and no calls to other drivers
down any driver stack, with the file contents being hard-coded).

There is no worker thread, I complete each and every request synchronous.

Storage for FCB and CCB’s are hard coded and stored within the
device extension (I allow two CCB’s for each FCB). This is just to
simplify my debugging. It’s a hello-world-driver only anyway :slight_smile:

I don’t worry/care about sharing flags. I’ve messed with pFile within
the DispatchRead() and modified those values a bit, but this didn’t
change the behaviour of the CcCopyRead problem.


Vogon International GmbH
http://www.vogon-international.com — Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17 You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’ To unsubscribe send a blank email to xxxxx@lists.osr.com


Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com