Understanding poor performance of memory mapped files in system cache

You could provoke the same behavior locally as well (without using SMB).

It was fixed in win8/2012.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Friday, November 22, 2013 8:37 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Understanding poor performance of memory mapped files in system cache

I don’t think the cache manager will cache the entire file, anyway.

The elementary way of making 2008 (R2 or not) Cc/Mm to go thrashing:

  • create many multi-GB files in the directory (TBs of data)
  • open each of them (there can be 100s of them) and read several bytes from the header
  • this must be done from the SMB client

Voila the thrashing.

A lot of MM policies are implemented outside the kernel, in the sysmain/superfetch service. Things like what priorities to assign to pages, what and when to prefetch, when to perform page combining (aka memory deduplication) etc.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Friday, November 22, 2013 8:32 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] Understanding poor performance of memory mapped files in system cache

However, Windows kernel is full of policy decisions

Same is Linux - MM, scheduling quantums and so on.

Daemons in UNIX were NOT created to separate policy and mechanism (and UNIXen violate this good design rule as often as Windows).

Daemons in UNIX were created EXACTLY with the same purpose as Windows services - to run independently on what user(s) is logged on.

Let’s stop counting misdesigns and kludges in these OSes. All of them have plenty. Nevertheless, all of them can do work and are trustworthy (if properly handled).


Maxim S. Shatskih
Microsoft MVP on File System And Storage xxxxx@storagecraft.com http://www.storagecraft.com


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Max,
Is it not possible to thrash cache by opening sequentially many multi-KB files on a system with small amount of memory ?

Pavel L,
Will you be able to explain the fix ? Although I am yet to encounter a 500gb VM, but I have a client that runs domino on a 32gb iron.

Thanks,
Arvind

Which VM software are you running on?

On Fri, Nov 8, 2013 at 7:59 PM, wrote:
> I have a process that is memory mapping a 160GB file on a system with 500GB of memory.
Kind regards, Dejan.