Problems about memory mapped io and minifilter

NTFSD
1

Dear all,

I’m writing a kernel mode encryption/decryption filter. I first try to implement it by
modifying filespy sample in DDK samples. It works fine when file is small even
using notepad. However, there is something wrong to deal with file over 4096
bytes or so. Although I’m still trying to solving this problem, I start to rewrite
the whole program based on the architecture of minifilter.

I implement it by modifying the swapbuffers sample in DDK. It works well to
deal with big file. However, I find that it can not decrypt data for applications
(like notepad) using memory mapped io.

I have use several tools. I find that IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION
packets will be received before I try to read a file through notepad. If I complete the
request directly with error status, the notepad will not show any words. Therefore,
I believe that it is a critical point. Because my filespy version works well to deal with
small files in notepad, I guess there are three possible solutions:

  1. I may miss some read requests. It may come from the following two reason:
    a) I decide whether or not to encrypt/decrypt a file by name. However, memory
    mapped file may request by handle ID.
    b) I may only hook selective IRPs in simrep, minispy, and swapbuffers example.
    There may be other IRPs.

  2. I found that when I first open a file with notepad. The messages is decrypted.
    However, notepad only shows the orignal data. Maybe I should copy data to other
    place different from original swapbuffers sample.

  3. Maybe I can modify the response of IRP_CREATE to reject the mapped io request
    or force it to go through other path.

Would you please tell me what solution would be right and how to solve it? I know
the solution would be buy OSR data modification kit. However, I am an assistant
professor. It’s a POC experiment to my encryption algorithm.

Thank you very much,

p.s. I find that OSR will hold a seminar about minifilter. Can I obtain the solution in
the seminar?

CSC

2

> deal with big file. However, I find that it can not decrypt data for applications

(like notepad) using memory mapped io.

Just decrypt paging IO only and you’re fine.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

3

csc@cs.ntust.edu.tw wrote:

Dear all,

I’m writing a kernel mode encryption/decryption filter. I first try to implement it by
modifying filespy sample in DDK samples. It works fine when file is small even
using notepad. However, there is something wrong to deal with file over 4096
bytes or so. Although I’m still trying to solving this problem, I start to rewrite
the whole program based on the architecture of minifilter.

I implement it by modifying the swapbuffers sample in DDK. It works well to
deal with big file. However, I find that it can not decrypt data for applications
(like notepad) using memory mapped io.

I have use several tools. I find that IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION
packets will be received before I try to read a file through notepad. If I complete the
request directly with error status, the notepad will not show any words. Therefore,
I believe that it is a critical point.

This request is not critical from your perspective.

Because my filespy version works well to deal with

small files in notepad, I guess there are three possible solutions:

  1. I may miss some read requests. It may come from the following two reason:
    a) I decide whether or not to encrypt/decrypt a file by name. However, memory
    mapped file may request by handle ID.

I assume you are making this determination in the IO pathway itself? Or
are you making this determination in your pre-create call back? If the
former, that is you problem.

As Max said, only process paging and non-cached requests.

b) I may only hook selective IRPs in simrep, minispy, and swapbuffers example.
There may be other IRPs.

You need pre-create, pre-set info, read and write. You’ll need to also
process directory enumeration but this is not effecting your issue.

  1. I found that when I first open a file with notepad. The messages is decrypted.
    However, notepad only shows the orignal data. Maybe I should copy data to other
    place different from original swapbuffers sample.

Reads you can decrypt in place within the post-read handler. Writes you
need encrypt using a second buffer otherwise you will end up with
encrypted data in the system cache.

  1. Maybe I can modify the response of IRP_CREATE to reject the mapped io request
    or force it to go through other path.

Memory mapped is handled the same handlers as normal IO except you only
get paging IO. You are missing something in your logic.

Would you please tell me what solution would be right and how to solve it? I know
the solution would be buy OSR data modification kit. However, I am an assistant
professor. It’s a POC experiment to my encryption algorithm.

Thank you very much,

p.s. I find that OSR will hold a seminar about minifilter. Can I obtain the solution in
the seminar?

The solution to implementing an encryption driver within a filter driver
is very difficult, some even say it is not 100% not feasible. The only
way to understand the solution is to work through the issues.

Pete

CSC

NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

4

Thank Peter and Max.

I’ve try to print out the flags of IRP_MJ_READ request in my pre-read
function.
It is 0x900, which is equal to IRP_READ_OPERATION +
IRP_DEFER_IO_COMPLETION. Assume that it is Paging IO, I rember in
Windows NT File Internals P431:

For paging I/O, the FSD has to trust the VMM to do the right thing.
First, mark the IRP as pending, then invoke the lower-level driver after
setting a completion routine.
Meanwhile, this particular thread can immediately return a STATUS_PENDING
return code.
The completion routine is then responsible for completing the IRP and
unlocking appropriate resources.

I think that’s why the data remain encrypted even I modify
Parameters.Read.MdlAddress
or Parameters.Read.ReadBuffer in post-read callback function. And I see
Acquire for Section
Synchronize.

I think I need to modify data in VMM. Would please tell me whether it works?
If it works, would
you please suggest a book or paper to read.

Thank you very much,

CSC

5

Shi-Cho Cha (Shih-Chao Cha) wrote:

Thank Peter and Max.

I’ve try to print out the flags of IRP_MJ_READ request in my pre-read
function.
It is 0x900, which is equal to IRP_READ_OPERATION +
IRP_DEFER_IO_COMPLETION. Assume that it is Paging IO, I rember in
Windows NT File Internals P431:

This is not paging IO, you need to look for the IRP_NOCACHE flag. This
flag will be set when the IRP_PAGING_IO or IRP_SYNCHRONOUS_PAGING_IO
flags are also set. The NOCACHE flag will tell you that the IO is
destined for the disk and you need to process it.

For paging I/O, the FSD has to trust the VMM to do the right thing.
First, mark the IRP as pending, then invoke the lower-level driver after
setting a completion routine.
Meanwhile, this particular thread can immediately return a
STATUS_PENDING return code.
The completion routine is then responsible for completing the IRP and
unlocking appropriate resources.

I think that’s why the data remain encrypted even I modify
Parameters.Read.MdlAddress
or Parameters.Read.ReadBuffer in post-read callback function. And I see
Acquire for Section
Synchronize.

Not sure why you are making this connection. The post IO handler won’t
be called until the IO is completed; pending the request is NOT
completing it.

I think I need to modify data in VMM. Would please tell me whether it
works? If it works, would
you please suggest a book or paper to read.

There are no books or papers other than what you find in this list or at
OSROnline.

Pete

Thank you very much,

CSC

NTFSD is sponsored by OSR

For our schedule of debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

6

Maybe you can just flush and clean the cache

7

Before you even suggest something like that, read the archives flushing the
cache is not something that works in many circumstances.

–
Don Burn (MVP, Windows DDK)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

“Hack diy” wrote in message news:xxxxx@ntfsd…
> Maybe you can just flush and clean the cache
>
>
>
> Information from ESET NOD32 Antivirus, version of virus
> signature database 3920 (20090309)
>
> The message was checked by ESET NOD32 Antivirus.
>
> http://www.eset.com
>
>

Information from ESET NOD32 Antivirus, version of virus signature database 3920 (20090309)

The message was checked by ESET NOD32 Antivirus.

http://www.eset.com

8

----- Original Message -----
From: “Peter Scott”
To: “Windows File Systems Devs Interest List”
Sent: Monday, March 09, 2009 11:43 PM
Subject: Re: [ntfsd] Problems about memory mapped io and minifilter

>
> This is not paging IO, you need to look for the IRP_NOCACHE flag. This
> flag will be set when the IRP_PAGING_IO or IRP_SYNCHRONOUS_PAGING_IO flags
> are also set. The NOCACHE flag will tell you that the IO is destined for
> the disk and you need to process it.
>

Thanks,

Howver, when I trace all request with minispy, I found that there is only on
read
request to my target file (e.q., test.xyz). The other related IRP received
are IRP_CREATE
IRP_QUERY_INFORMATION (I checked the
QueryFileInformation.FileInformationClass.
It is FileAllInformation.) Then, IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION
is received. Then, the notepad shows non-decrypted data.

I think the system obtain data of the file directly from somewhere. In this
case, I think there
may be two possible solutions:

1) Can I manipulate the IRP_MJ_READ (with IRP_DEFER_IO_COMPLETION) request
to
force system to send a normal IRP_MJ_READ request (eq. IRP_PAGING_IO or
IRP_SYNCHRONOUS_PAGING_IO )?

2) Can I do something when I receive
IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION
to force system to send a normal IRP_MJ_READ request (eq. IRP_PAGING_IO or
IRP_SYNCHRONOUS_PAGING_IO )?

Thank you very much

===========================================
Shi-Cho Cha, Ph.D.
Assistant Professor, Dept. of Information Management, National Taiwna
University of Science and Technology
Adjunct Assistant Professor, Dept. of BA, National Taiwan University
http://140.118.9.108

9

> I think the system obtain data of the file directly from somewhere. In

this case, I think there
may be two possible solutions:

No. Pete gave you the answer. If you are seeing it in notepad it is either
cached (in which case you missed the pagefault which brought it in), or it
was pagefaulted in (in which case you missed the pagefault which brought it
in). Don’t forget that the prefecther will probably have been at work
before you started looking.

10

Shi-Cho Cha (Shih-Chao Cha) wrote:

----- Original Message ----- From: “Peter Scott”
> To: “Windows File Systems Devs Interest List”
> Sent: Monday, March 09, 2009 11:43 PM
> Subject: Re: [ntfsd] Problems about memory mapped io and minifilter
>
>
>>
>> This is not paging IO, you need to look for the IRP_NOCACHE flag. This
>> flag will be set when the IRP_PAGING_IO or IRP_SYNCHRONOUS_PAGING_IO
>> flags are also set. The NOCACHE flag will tell you that the IO is
>> destined for the disk and you need to process it.
>>
>
> Thanks,
>
> Howver, when I trace all request with minispy, I found that there is
> only on read
> request to my target file (e.q., test.xyz). The other related IRP
> received are IRP_CREATE
> IRP_QUERY_INFORMATION (I checked the
> QueryFileInformation.FileInformationClass.
> It is FileAllInformation.) Then, IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION
> is received. Then, the notepad shows non-decrypted data.
>
> I think the system obtain data of the file directly from somewhere. In
> this case, I think there

No, this is not the case. As Rod says, you are missing the in-page
operation somewhere. How are you making the determination that it is
your file that you are interested in during the read operation? If you
are performing name based lookups or using the file object itself that
you tracked during open then your design is broken, you need to also
consider things like stream file objects.

> may be two possible solutions:
>
> 1) Can I manipulate the IRP_MJ_READ (with IRP_DEFER_IO_COMPLETION)
> request to
> force system to send a normal IRP_MJ_READ request (eq. IRP_PAGING_IO or
> IRP_SYNCHRONOUS_PAGING_IO )?
>

Again, you are simply missing the in-page operation probably because of
how you are making the determination that a given read is a read you are
interested in.

> 2) Can I do something when I receive
> IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION
> to force system to send a normal IRP_MJ_READ request (eq. IRP_PAGING_IO or
> IRP_SYNCHRONOUS_PAGING_IO )?
>

This won’t help … you are missing the in-page read.

Pete

> Thank you very much
>
> ===========================================
> Shi-Cho Cha, Ph.D.
> Assistant Professor, Dept. of Information Management, National Taiwna
> University of Science and Technology
> Adjunct Assistant Professor, Dept. of BA, National Taiwan University
> http://140.118.9.108
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars
> (including our new fs mini-filter seminar) visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

–
Kernel Drivers
Windows File System and Device Driver Consulting
www.KernelDrivers.com
866.263.9295

11

Thanks all,

I roughly understand the point. I guess I miss paged IO IRP_MJ_READ
requests of files. Then, the DEFER_IO_COMPLETION requests are
sent. It would come from file name handling problem. I’m trying to use
method in CTX sample to solve the problem.

Thank you very much,

CSC

----- Original Message -----
From: “Peter Scott”
To: “Windows File Systems Devs Interest List”
Sent: Tuesday, March 10, 2009 8:52 PM
Subject: Re: [ntfsd] Problems about memory mapped io and minifilter

> Shi-Cho Cha (Shih-Chao Cha) wrote:
>> ----- Original Message ----- From: “Peter Scott”
>>
>> To: “Windows File Systems Devs Interest List”
>> Sent: Monday, March 09, 2009 11:43 PM
>> Subject: Re: [ntfsd] Problems about memory mapped io and minifilter
>>
>>
>>>
>>> This is not paging IO, you need to look for the IRP_NOCACHE flag. This
>>> flag will be set when the IRP_PAGING_IO or IRP_SYNCHRONOUS_PAGING_IO
>>> flags are also set. The NOCACHE flag will tell you that the IO is
>>> destined for the disk and you need to process it.
>>>
>>
>> Thanks,
>>
>> Howver, when I trace all request with minispy, I found that there is only
>> on read
>> request to my target file (e.q., test.xyz). The other related IRP
>> received are IRP_CREATE
>> IRP_QUERY_INFORMATION (I checked the
>> QueryFileInformation.FileInformationClass.
>> It is FileAllInformation.) Then,
>> IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION
>> is received. Then, the notepad shows non-decrypted data.
>>
>> I think the system obtain data of the file directly from somewhere. In
>> this case, I think there
>
> No, this is not the case. As Rod says, you are missing the in-page
> operation somewhere. How are you making the determination that it is your
> file that you are interested in during the read operation? If you are
> performing name based lookups or using the file object itself that you
> tracked during open then your design is broken, you need to also consider
> things like stream file objects.
>
>> may be two possible solutions:
>>
>> 1) Can I manipulate the IRP_MJ_READ (with IRP_DEFER_IO_COMPLETION)
>> request to
>> force system to send a normal IRP_MJ_READ request (eq. IRP_PAGING_IO or
>> IRP_SYNCHRONOUS_PAGING_IO )?
>>
>
> Again, you are simply missing the in-page operation probably because of
> how you are making the determination that a given read is a read you are
> interested in.
>
>> 2) Can I do something when I receive
>> IRP_MJ_ACQUIRE_FOR_SECTION_SYNCHRONIZATION
>> to force system to send a normal IRP_MJ_READ request (eq. IRP_PAGING_IO
>> or
>> IRP_SYNCHRONOUS_PAGING_IO )?
>>
>
> This won’t help … you are missing the in-page read.
>
> Pete
>
>> Thank you very much
>>
>> ===========================================
>> Shi-Cho Cha, Ph.D.
>> Assistant Professor, Dept. of Information Management, National Taiwna
>> University of Science and Technology
>> Adjunct Assistant Professor, Dept. of BA, National Taiwan University
>> http://140.118.9.108
>>
>> —
>> NTFSD is sponsored by OSR
>>
>> For our schedule of debugging and file system seminars
>> (including our new fs mini-filter seminar) visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>
> –
> Kernel Drivers
> Windows File System and Device Driver Consulting
> www.KernelDrivers.com
> 866.263.9295
>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars
> (including our new fs mini-filter seminar) visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

12

Thanks all,
I finally implement a “simple” encryption/decryption filter. I find that
the
major problem is “the filter I wish to implement is not really a simple
filter”.
Encrypting/decrypting selective files is much harder than encrypting/
decrypting every files in a disk volume.

One interesting thing is: implementing minifilter in vista is much easier
than
XP because file context are supported after vista.

Thank you very much,

CSC

13

> I find that the

major problem is “the filter I wish to implement is not really a simple
filter”.

My belief is that no filter is really simple. The world in which it needs
to live is too complicated. Filter manager gives the impression of making
things easy, but all it does is make things easier, and allows you to get
something to 80% done very quickly. Both of these are important, but there
is nothing that filter manager can do to make the environment in which the
filter operates less complicated.

FWIW its trivially easy to build your own file context support. You
de-dedupe using the fileId, then it’s just refcounting and lookup. Bear in
mind also the file contexts may not be supported on all FSDs that support
streams.

Rod

“Shi-Cho Cha (Shih-Chao Cha)” wrote in message
news:xxxxx@ntfsd…
> Thanks all,
> I finally implement a “simple” encryption/decryption filter. I find that
> the
> major problem is “the filter I wish to implement is not really a simple
> filter”.
> Encrypting/decrypting selective files is much harder than encrypting/
> decrypting every files in a disk volume.
>
> One interesting thing is: implementing minifilter in vista is much easier
> than
> XP because file context are supported after vista.
>
> Thank you very much,
>
> CSC
>