Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Home NTDEV

Before Posting...

Please check out the Community Guidelines in the Announcements and Administration Category.

More Info on Driver Writing and Debugging


The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.


Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/


Cache manager shared cache map

Carlo_CianfaraniCarlo_Cianfarani Member Posts: 240

Hi,
reading Windows Internals 6th edition, I'd ask for a clarification about the following:

I'm aware that for each file being cached, Cache manager calls a VMM's exported function to create a Section object for it. Therefore, at any given time, there may be in the system multiple section objects for the same file object as well as multiple mapped views of the file stored in different cache slots.
 

For each file being cached, Cache manager maintains shared cache map instances linked in a list (see Fig 11-7 in the book). Each shared cache map instance stores a pointer to a (multilevel) VACB index array.
 

My question is: why can there be multiple shared cache map instances for the same cached file ? Just because can't one instance of the three-level VACB index array be enough to index all the desired views of the cached file ?
 

Thank you.

Comments

  • Carlo_CianfaraniCarlo_Cianfarani Member Posts: 240

    Thinking about it, I believe there exist only one instance of shared cache map for a given file being cached. Maybe shared cache map instances (for different files being cached) are linked together just for efficient handling.

    What do you think about ?

  • Scott_Noone_(OSR)Scott_Noone_(OSR) Administrator Posts: 3,679

    @Carlo_Cianfarani said:
    Hi,
    reading Windows Internals 6th edition, I'd ask for a clarification about the following:

    I'm aware that for each file being cached, Cache manager calls a VMM's exported function to create a Section object for it. Therefore, at any given time, there may be in the system multiple section objects for the same file object

    There can only be up to two section objects per-stream*: one data section object and one image section object. Once a section is created (data or image) subsequent attempts to memory map reference the same section.

    -scott

    *At the file system layer the File Object can be thought of as an open instance of a Stream, which is an in-memory representation of a "stream" of file, directory, or volume data (+ its attributes)

    -scott
    OSR

  • Scott_Noone_(OSR)Scott_Noone_(OSR) Administrator Posts: 3,679

    @Carlo_Cianfarani said:
    Thinking about it, I believe there exist only one instance of shared cache map for a given file being cached. Maybe shared cache map instances (for different files being cached) are linked together just for efficient handling.

    What do you think about ?

    Correct.

    -scott
    OSR

  • Carlo_CianfaraniCarlo_Cianfarani Member Posts: 240

    There can only be up to two section objects per-stream*: one data section object and one image section object. Once a section is created (data or image) subsequent attempts to memory map reference the same section.

    Ok, so even though a given file is memory mapped in multiple processes, there will be at most two section objects for it in the system.
     

    As far as I understand, if the given file is also opened in a process for I/O (read, write), the cache manager will uses the same data section object the Memory manager (MM) created for it. Therefore all (data) memory mapped views of the file (in each process's user address space and in system cache) will reference the same data section object.

  • Carlo_CianfaraniCarlo_Cianfarani Member Posts: 240
    Just to check my understanding: cache manager for each file being cached refers to the same section object regardless of how many views of the file are currently mapped in system cache space.

    Correct ?
  • Tim_RobertsTim_Roberts Member - All Emails Posts: 14,836

    I'm curious to know what possible difference this answer would have in your life. What would you do differently?

    Tim Roberts, [email protected]
    Software Wizard Emeritus

  • Carlo_CianfaraniCarlo_Cianfarani Member Posts: 240
    Nothing. I just wanted to better understand some aspects of Windows implementation.
  • Mark_RoddyMark_Roddy Member - All Emails Posts: 4,754

    There is a guarantee of read coherency for any view of a file. How stunningly awful that would be if the cache manager had to mediate N separate mappings of a file in memory.

  • Carlo_CianfaraniCarlo_Cianfarani Member Posts: 240

    There is a guarantee of read coherency for any view of a file. How stunningly awful that would be if the cache manager had to mediate N separate mappings of a file in memory.

    Therefore when the cache manager maps a new view of a given file (256 KiB in size each) it refers to its unique data section object and maps the view in a system cache slot assigning a free VACB for it, committing 256 KiB of memory for it and creating the Prototype PTEs (PPTEs) needed to map it in the system cache slot's VA range.

  • Scott_Noone_(OSR)Scott_Noone_(OSR) Administrator Posts: 3,679

    @Carlo_Cianfarani said:

    There is a guarantee of read coherency for any view of a file. How stunningly awful that would be if the cache manager had to mediate N separate mappings of a file in memory.

    Therefore when the cache manager maps a new view of a given file (256 KiB in size each) it refers to its unique data section object and maps the view in a system cache slot assigning a free VACB for it, committing 256 KiB of memory for it and creating the Prototype PTEs (PPTEs) needed to map it in the system cache slot's VA range.

    The Cache Manager's section is not unique, it's the Data Section. Whether an application maps that section into user mode or the Cache Manager maps the section into kernel mode (or both) it's the same section. Cached I/O is then just a memcpy into/out of a kernel mapping to the section that the Cache Manager happens to maintain.

    -scott
    OSR

  • Carlo_CianfaraniCarlo_Cianfarani Member Posts: 240

    The Cache Manager's section is not unique, it's the Data Section. Whether an application maps that section into user mode or the Cache Manager maps the section into kernel mode (or both) it's the same section.

    Maybe I misinterpreted the sentence at the Dissecting Windows Section Objects:

    Note that no matter how many sections have been created for the file object, there's always only one segment structure per type of mapping (binary, executable) for all of them. The same applies to Control Area structures, there's only one Control Area per type of mapping regardless of the number of created sections

    It seems to me there could be multiple section objects for a given file object even though there is always one segment structure and one Control Area structure per type of mapping (data/binary, executable).

    Am I wrong ?

  • Scott_Noone_(OSR)Scott_Noone_(OSR) Administrator Posts: 3,679

    Yes, fair point and the article is accurate.

    The point I was trying to stress is that there is only one copy of the data in memory for each mapping type (data or image). Internally these copies are tracked/maintained by internal Mm structures called segments and are accessed by external consumers via section objects.

    -scott
    OSR

  • Carlo_CianfaraniCarlo_Cianfarani Member Posts: 240
    I believe it is just a matter of names: a data section object is actually a data Control Area structure + data Segment structure (same for image section object). For a given file Stream there are at most two of them in the system (one for each type): **the** data section object and/or **the** image section object.
Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. Sign in or register to get started.

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Kernel Debugging 13-17 May 2024 Live, Online
Developing Minifilters 1-5 Apr 2024 Live, Online
Internals & Software Drivers 11-15 Mar 2024 Live, Online
Writing WDF Drivers 20-24 May 2024 Live, Online