Disadvantages of MMF

NTDEV
1

Hi,
I’d like to know what could be the possible disadvantages of using Memory Mapped Files considering all possible factors.

TIA.
Punit.

2

Using memory mapped files in what context? That’s like asking which color ink is better, red or blue, without telling us what you are writing on…

d

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Monday, February 09, 2009 10:32 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Disadvantages of MMF

Hi,
I’d like to know what could be the possible disadvantages of using Memory Mapped Files considering all possible factors.

TIA.
Punit.

NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

3

Dear Doron,

Thanks for your reply.

I’m developing a secure delete functionality. The data is overwritten in several passes. Is there a possibility that the data I’ve currently written to in an MMF is not written to the actual disk?
What should I do in order to ensure such a thing?

Punit.

4

> I’m developing a secure delete functionality. The data is overwritten in several passes. Is there a

possibility that the data I’ve currently written to in an MMF is not written to the actual disk?
What should I do in order to ensure such a thing?

FlushViewOfFile can help.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

5

> FlushViewOfFile can help.

Well, I would say in this context the only approach that makes sense is WriteFile() on the file that got opened with FILE_FLAG_NO_BUFFERING…

Anton Bassov

6

> Well, I would say in this context the only approach that makes sense is WriteFile() on the file that got

opened with FILE_FLAG_NO_BUFFERING…

Yes, for such a task, I personally would do this exactly instead of relying on MMF.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

7

Hi,

Thank you for all those replies.
Well, if I close the MMF and open it again, will it be assured that all the contents would be reverted back to the disk? Because, I tried comparing MMF and regular file I/O with the same method, same set of similar instructions and on the same data. I found that MMF turned out quite faster than regular file I/O. I’ve also used FILE_FLAG_WRITE_THROUGH flag in CreateFile.

Thanks in advance,
Punit.

8

Hello Everyone,

I’m still waiting for a reply to the above question. Can anyone please post their comment on it.

Thanks,
Punit.

9

xxxxx@gmail.com wrote:

Thank you for all those replies.
Well, if I close the MMF and open it again, will it be assured that all the contents would be reverted back to the disk? Because, I tried comparing MMF and regular file I/O with the same method, same set of similar instructions and on the same data. I found that MMF turned out quite faster than regular file I/O. I’ve also used FILE_FLAG_WRITE_THROUGH flag in CreateFile.

I’m not sure the question is meaningful, and I’m not sure what the first
part has to do with the second. If process A writes to a page in a file
mapping, and process B later reads that page using a mapping, then
process B will see the new data. It’s really irrelevant whether the
mapping was closed and reopened, or even whether the page was ever
flushed to disk or not.

–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

10

Defined “reverted” in this case.

If you open a file, write to it (using ANY sort of I/O, memory mapped or otherwise) and close the file, the contents of that file are changed. When you open the file again, the contents of the file should be the same as the contents that you have written before you closed it… otherwise, the operating system is severely broken, don’t you think??

What are you really trying to find out?

Peter
OSR

11

> If you open a file, write to it (using ANY sort of I/O, memory mapped or otherwise) and

close the file, the contents of that file are changed. When you open the file again, the
contents of the file should be the same as the contents that you have written before you closed it…

However, unless you explicitly flush the file mapping, there is no guarantee that changes that you made
have been actually committed to the disk at this moment. IIRC, Mapped Page Writer will not flush data until the number of dirty pages in the system reaches a certain threshold, so that it may take quite a while.

I think this is what the poster asks about - he just failed to formulate his question properly. In context of his task this part is quite important - otherwise, it may well happen that, instead of overwriting file data multiple times he will do it just once, i.e. all his multiple overwrites will take place mainly in RAM. This is why I believe WriteFile() on the file that got opened with FILE_FLAG_NO_BUFFERING is the most reasonable approach here. Flushing file mapping should work as well, but, depending of the amounts of data involved, performance may be much worse because of the synch nature of MM operations…

Anton Bassov

12

The data on disk will be coherent “rather quickly” (in a matter of seconds, usually)… but the data may remain cached for a very long time (hours, days, weeks even) in absence of memory pressure.

Too much mind reading going on here as of late… OP asks one question, but really wants the answer to another, unstated, question. Makes it difficult to help, you know… and then we (broadly, the folks who answer questions) wind-up talking to each other.

Peter
OSR

13

Dear Peter, Tim, et. al,

I know that if I write something to a file (using any I/O mechanism) and then close the file, theoretically, it should get flushed to the disk. But, I don’t know what Windows does on the inside. Whether there is some kind of optimization in writing to disk and which may introduce some delay.

I already mentioned that I’m writing an application for secure deletion of files. So, there is no “reading” done from the MMF. My application overwrites the contents several times (several passes of overwriting). So, for every pass, it is necessary that the contents (just overwritten with) are being written to the disk before the next pass starts.

Now, for creating MMF we need to open the file (using OpenFile() or CreateFile()), then call CreateFileMapping() to create the file mapping and finally MapViewOfFile() to work with the data.
So, in CreateFile(), I used the flag FILE_FLAG_WRITE_THROUGH to make the write operation flush to disk without delay. And then, called CreateFileMapping() & MapViewOfFile(). Moreover, after every pass I’m closing the file mapping and the file and opening them again (just in case).

So, will this work?

I guess now my requirement and question is clear.

Punit.

14

> So, in CreateFile(), I used the flag FILE_FLAG_WRITE_THROUGH to make the write

operation flush to disk without delay. And then, called CreateFileMapping() & MapViewOfFile().
Moreover, after every pass I’m closing the file mapping and the file and opening them again (just in case).

Do you actually read the replies that you get???

FILE_FLAG_WRITE_THROUGH means that write operation on a file is not considered complete until data gets actually flushed to the disk - it does not mean that it will force flushing file mapping upon closing the handle. In context of your task you should explicitly flush file mapping to the disk before attempting the next overwrite if you are just desperate to use file mapping instead of WriteFile() ( I have earlier explained to you why WriteFile() is better for your task if you use it without intermediate buffering, but you don’t seem to read the replies anyway). However, for the reasons better known to yourself you close and reopen file handle instead…

Anton Bassov

15

Hi Anton,

This is the description written for FILE_FLAG_WRITE_THROUGH in MSDN:
"Write operations will not go through any intermediate cache, they will go directly to disk.

If FILE_FLAG_NO_BUFFERING is not also specified, so that system caching is in effect, then the data is written to the system cache, but is flushed to disk without delay.

If FILE_FLAG_NO_BUFFERING is also specified, so that system caching is not in effect, then the data is immediately flushed to disk without going through the system cache. The operating system also requests a write-through the hard disk cache to persistent media. However, not all hardware supports this write-through capability."

And since there was no note saying “this is not applicable to MMF”, I thought this might work. And like you said in your post, depending on the amount of data, it could just kill the performance.

Anyways, so you propose to go for CreateFile() with FILE_FLAG_NO_BUFFERING.

Thank you all for your replies and suggestions. Sorry for the trouble.

Regards,
Punit.

16

Yes. And write to the file using CONVENTIONAL, not memory mapped, I/O. Forget the use of memory mapped files in this application. They don’t make any sense for what you want to do.

Peter
OSR

17

> And since there was no note saying “this is not applicable to MMF”, I thought this might work.

Forget MMF if you’re doing security-grade disk wipe. Just forget it.

Instead, open the file in noncached mode and write it.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com