Windows System Software -- Consulting, Training, Development -- Unique Expertise, Guaranteed Results

Home NTDEV

Before Posting...

Please check out the Community Guidelines in the Announcements and Administration Category.

More Info on Driver Writing and Debugging


The free OSR Learning Library has more than 50 articles on a wide variety of topics about writing and debugging device drivers and Minifilters. From introductory level to advanced. All the articles have been recently reviewed and updated, and are written using the clear and definitive style you've come to expect from OSR over the years.


Check out The OSR Learning Library at: https://www.osr.com/osr-learning-library/


Implementing a growable region of memory.

EmjayenEmjayen Member Posts: 11
edited October 2023 in NTDEV

In usermode I would normally implement this by reserving the desired maximum size of address-space, and incrementally committing as necessary to grow. ie.,:

Base = VirtualAlloc(..., 0x200000 /* Max size */, MEM_RESERVE)

VirtualAlloc(Base + CurrentSize, 0x1000 /* Grow size */, MEM_COMMIT);  CurrentSize += 0x1000;

I'm struggling however to understand the correct way to accomplish this in KM. My thought is that it should be fundamentally similar; grab a big chunk of system address-space, then to grow: allocate some physical pages for backing and map them (similar to how one would do it with AWE).

The technique I'm currently using entails:

  • Allocating system address-space via MmAllocateMappingAddress
  • Then to grow using MmAllocatePagesForMdlEx followed by MmMapLockedPagesWithReservedMapping

This does work, however:

  • These APIs feel inappropriate to be using for this (based on interface and the docs) and more like what one would be using to e.g, allocate a long-lived pool of DMA buffers in an actual device driver. Despite trawling the reference docs, I couldn't find anything else however.
  • While researching, I came across an old thread here on ntdev where-in the poster was trying to do the same thing, however he noticed (as I myself did), that the documentation for MmMapLockedPagesWithReservedMapping specifies that the base address must be the base allocation (as returned by MmAllocateMappingAddress) which thwarted his attempt (he reports it would bugcheck trying to pass an offsetted address). The problem is mine works, which suggests I'm doing something wrong -- driver verifier doesn't report anything, and nothing is blowing up.
  • The biggest issue: from what I understand from the docs, each allocation made via MmAllocatePagesForMdlEx must be freed by providing the exact same MDL to MmFreePagesFromMdl, which in this case would mean having to hold onto a potentially very large number of them. Is it maybe possible to, at teardown time, simply build an MDL describing the mapped region (since that PFN information previously in the MDLs would exist in the PTE) and free that?

notes:

  • The lifetime of this region matches that of the driver itself/is global.
  • Only grows; never shrinks.
  • Being virtually contiguous is a requirement; physical contiguity is not required.
  • Private to the driver; does not need to be shared.
  • Must be backed by nonpaged memory (due to IRQL)
  • NDIS filter driver.

Thanks,

Matt.

Comments

  • Mark_RoddyMark_Roddy Member - All Emails Posts: 4,753

    The remarks section for the MmMapLockedPagesWithReservedMapping doc here: https://learn.microsoft.com/en-us/windows-hardware/drivers/ddi/wdm/nf-wdm-mmmaplockedpageswithreservedmapping#remarks seems to directly address your situation. Note the requirement to us IoAllocateMdl and MmProbeAndLockPages.

  • EmjayenEmjayen Member Posts: 11

    It does also mention (in the parameter description) that one can use MmAllocatePagesForMdlEx, and infact this is what Peter recommends in the cited thread.

  • EmjayenEmjayen Member Posts: 11

    Actually this technique does not work; I assumed since mapping at an offset address worked that it would allow arbitrary mappings. This does not appear to be the case, and there is indeed a 1:1 mapping between the reserved region and a supplied MDL.

    This also appears to be the case for MmProbeAndLockPages which was another approach I was considering -- allocate pagable memory and incrementally lock.

  • EmjayenEmjayen Member Posts: 11
    edited October 2023

    After some conferring with with the NT5 source, Windows Internals and some reverse-engineering I managed to discover a way to accomplish this.

    There's a [what I thought was] an undocumented flag for MmAllocateMappingAddressEx which enables the size of the region described by the MDL to be used instead of the allocation size when interrogating the PTEs to determine if they're valid (existing translation/mapping).

    Later I discovered it's just not documented on MSDN -- it's actually right there in the header called MM_MAPPING_ADDRESS_DIVISIBLE

    Some trivia after looking into this: in NT5, the size and tag supplied by the user is encoded within the page table itself using 2 prefixed PTE entries, one for the size, one for the tag.

    In recent kernels it appears to be stored externally in a regular pool-allocated linked-list.

Sign In or Register to comment.

Howdy, Stranger!

It looks like you're new here. Sign in or register to get started.

Upcoming OSR Seminars
OSR has suspended in-person seminars due to the Covid-19 outbreak. But, don't miss your training! Attend via the internet instead!
Kernel Debugging 13-17 May 2024 Live, Online
Developing Minifilters 1-5 Apr 2024 Live, Online
Internals & Software Drivers 11-15 Mar 2024 Live, Online
Writing WDF Drivers 20-24 May 2024 Live, Online