Scanning kernel memory

First of all: I want to do this for myself as a means of learning something (I can’t just read about things and not try to do some primitive work to get a taste of the action).
But even so I want to do it right and know if something is wrong with my idea before starting.

I want my driver to be able to scan kernel memory for various kernel structures (by various I mean some simple ones for which I can come up with a invariant signature) and when I find one print it’s contents and it’s address (and hopefully I’'ll be able to check if the output is correct with WinDbg’s dt command).

I think DRIVER_OBJECT structs aren’t that hard to detect as they have some pointers to UNICODE_STRINGs that I can check against (again, it doesn’t have to be perfect, so a few false positives / negatives aren’t a concert as I don’t do anything with the information I get).

The main problem for me is how should I implement the scanning mechanism. First of all I should exclude all user mode memory to speed things up a bit. Is it a good idea to take advantage of the user-kernel memory split (also, check if PAE is active before doing this on x76) and iterate through the physical addresses corresponding to the kernel and use MmMapIoSpace() to map them to virtual memory, scan the mapped section, decide if it has what I’m looking for, unmap it and so on? This also looks like it can take a lot of time / fail. Also, I haven’t figured out how can I dynamically detect the end of the physical memory 9I want to avoid hard coding things), but I suppose I can find that information from an Intel / AMD manual and / or from a Windows Internals book if I decide to go on this path.

Or should I get the Virtual Addresses corresponding to the kernel and check if they are valid (I have a working virtual to physical translating mechanism and I could use that), then scan them?
What are the pitfalls of this approach? Is there a better one that I failed to think of?

If I’m going to learn something I don’t want to start with a bad approach on this so all I’m asking for is a few hints about how to start and where to look (and maybe for some papers on the subject that someone should read before doing things like this).

A driver is not allowed to touch arbitrary addresses. Doing that crashes he OS immediately.

I know that, I should have explained my (probably bad) idea more in depth:
I want to traverse the paging structures looking for valid entries. Map those physical addresses, quickly look at their content, then unmap.
Assuming x86 Non-PAE, I’ll get the page directory base from cr3 (I want to do this in DriverEntry so I’ll be in the context of the system). I know that PD will have 1024 entries of 4 bytes, each pointing to a Page Table. If a PDE is valid I go ahead and parse the PT. Again, PT will have 1024 entries of 4 bytes, each pointing to a physical address.
If a PTE is valid and has the U/S bit = 0 (meaning that User mode can’t touch it) I use MmMapIoSpace and read it’s contents.

The first problem I see with this is that I’m not sure it won’t crash the system on some cases (hey, if I’m not crashing my test machine it means I’m not learning) and second the contents may change from the time I check if it is valid an map it so I can still end up with wrong data or even crash the system.

It feels like there should be a better solution for this, but right now I’m spinning in circles trying to figure out things.

xxxxx@gmail.com wrote:

I want to traverse the paging structures looking for valid entries. Map those physical addresses, quickly look at their content, then unmap.

If they are in the page tables, then they’re already mapped into
memory. That’s the point.

Assuming x86 Non-PAE, I’ll get the page directory base from cr3 (I want to do this in DriverEntry so I’ll be in the context of the system).

If you’re looking at kernel memory, the context is irrelevant. Kernel
memory is global to all processes.

If a PTE is valid and has the U/S bit = 0 (meaning that User mode can’t touch it) I use MmMapIoSpace and read it’s contents.

If the PTE is valid, you don’t have to map it. You can just make a
pointer and read it.

The first problem I see with this is that I’m not sure it won’t crash the system on some cases (hey, if I’m not crashing my test machine it means I’m not learning) and second the contents may change from the time I check if it is valid an map it so I can still end up with wrong data or even crash the system.

More importantly, if you’re working in physical space, you aren’t
guaranteed that a structure will be contiguous. If a structure happens
to be split between two different physical pages, those pages might be
miles apart in virtual space.

It feels like there should be a better solution for this, but right now I’m spinning in circles trying to figure out things.

It’s not just that there is “a better solution”, it’s that there are no
WORSE solutions. If you want to read system data structures, you go
find the root of those data structures in the system symbols and follow
the linked lists. You don’t start scanning through gigabytes of memory
looking for signs. The false positives will kill you. Most data
structures don’t have rock-solid signatures. The signature for most
kernel objects is one byte.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

> If the PTE is valid, you don’t have to map it. You can just make a
pointer and read it.

But if I start from the first paging structure and I start to iterate
through all the entries when I reach a valid PTE it’s simpler to just map
the Physical Address than to figure out what VA I should look at.

But…

If a structure happens
to be split between two different physical pages, those pages might be
miles apart in virtual space.

So I’ll end up failing.

But isn’t there a chance that a rootkit will remove some structures from
the linked lists or provide other structures when I ask nicely for them?

On Fri, Oct 17, 2014 at 7:52 PM, Tim Roberts wrote:

> xxxxx@gmail.com wrote:
> > I want to traverse the paging structures looking for valid entries. Map
> those physical addresses, quickly look at their content, then unmap.
>
> If they are in the page tables, then they’re already mapped into
> memory. That’s the point.
>
>
> > Assuming x86 Non-PAE, I’ll get the page directory base from cr3 (I want
> to do this in DriverEntry so I’ll be in the context of the system).
>
> If you’re looking at kernel memory, the context is irrelevant. Kernel
> memory is global to all processes.
>
>
> > If a PTE is valid and has the U/S bit = 0 (meaning that User mode can’t
> touch it) I use MmMapIoSpace and read it’s contents.
>
> If the PTE is valid, you don’t have to map it. You can just make a
> pointer and read it.
>
>
> > The first problem I see with this is that I’m not sure it won’t crash
> the system on some cases (hey, if I’m not crashing my test machine it means
> I’m not learning) and second the contents may change from the time I check
> if it is valid an map it so I can still end up with wrong data or even
> crash the system.
>
> More importantly, if you’re working in physical space, you aren’t
> guaranteed that a structure will be contiguous. If a structure happens
> to be split between two different physical pages, those pages might be
> miles apart in virtual space.
>
>
> > It feels like there should be a better solution for this, but right now
> I’m spinning in circles trying to figure out things.
>
> It’s not just that there is “a better solution”, it’s that there are no
> WORSE solutions. If you want to read system data structures, you go
> find the root of those data structures in the system symbols and follow
> the linked lists. You don’t start scanning through gigabytes of memory
> looking for signs. The false positives will kill you. Most data
> structures don’t have rock-solid signatures. The signature for most
> kernel objects is one byte.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

(sorry for double reply)

In fact, knowing that a VA is in essence constructed from indexes in the
paging tables I can build it back quite easy.

On Fri, Oct 17, 2014 at 9:02 PM, Cristi A wrote:

> > If the PTE is valid, you don’t have to map it. You can just make a
> pointer and read it.
>
> But if I start from the first paging structure and I start to iterate
> through all the entries when I reach a valid PTE it’s simpler to just map
> the Physical Address than to figure out what VA I should look at.
>
> But…
> > If a structure happens
> to be split between two different physical pages, those pages might be
> miles apart in virtual space.
>
> So I’ll end up failing.
>
> But isn’t there a chance that a rootkit will remove some structures from
> the linked lists or provide other structures when I ask nicely for them?
>
> On Fri, Oct 17, 2014 at 7:52 PM, Tim Roberts wrote:
>
>> xxxxx@gmail.com wrote:
>> > I want to traverse the paging structures looking for valid entries. Map
>> those physical addresses, quickly look at their content, then unmap.
>>
>> If they are in the page tables, then they’re already mapped into
>> memory. That’s the point.
>>
>>
>> > Assuming x86 Non-PAE, I’ll get the page directory base from cr3 (I want
>> to do this in DriverEntry so I’ll be in the context of the system).
>>
>> If you’re looking at kernel memory, the context is irrelevant. Kernel
>> memory is global to all processes.
>>
>>
>> > If a PTE is valid and has the U/S bit = 0 (meaning that User mode can’t
>> touch it) I use MmMapIoSpace and read it’s contents.
>>
>> If the PTE is valid, you don’t have to map it. You can just make a
>> pointer and read it.
>>
>>
>> > The first problem I see with this is that I’m not sure it won’t crash
>> the system on some cases (hey, if I’m not crashing my test machine it means
>> I’m not learning) and second the contents may change from the time I check
>> if it is valid an map it so I can still end up with wrong data or even
>> crash the system.
>>
>> More importantly, if you’re working in physical space, you aren’t
>> guaranteed that a structure will be contiguous. If a structure happens
>> to be split between two different physical pages, those pages might be
>> miles apart in virtual space.
>>
>>
>> > It feels like there should be a better solution for this, but right now
>> I’m spinning in circles trying to figure out things.
>>
>> It’s not just that there is “a better solution”, it’s that there are no
>> WORSE solutions. If you want to read system data structures, you go
>> find the root of those data structures in the system symbols and follow
>> the linked lists. You don’t start scanning through gigabytes of memory
>> looking for signs. The false positives will kill you. Most data
>> structures don’t have rock-solid signatures. The signature for most
>> kernel objects is one byte.
>>
>> –
>> Tim Roberts, xxxxx@probo.com
>> Providenza & Boekelheide, Inc.
>>
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>>
>> OSR is HIRING!! See http://www.osr.com/careers
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>
>

>?If the PTE is valid, you don’t have to map it.? You can just make a pointer and read it.

That is well known to be unsafe. Look at the MmIsAddressValid API. If the PTE is valid than the hardware won’t probably trigger a page fault on the next access. Probably, because on a multiprocessor machine, a thread running on a different processor might unmap the virtual address (VA) after MmIsAddressValid returns and before the VA is accessed. Of course this probability is very low, but as far as it is not null, the scheme is broken because you will then map a physical page that has been freed and maybe assigned to someone else and therefore the page’s content may be irrelevant.

Even on a single processor machine, it is not safe because your thread could be preempted and the scheduler may select another thread for execution and this new thread could possibly unmap the VA you are working on.

To achieve your goal in an effective and therefore deterministic way, your thread must hold certains system defined locks that are reserved.

So Mr. Grig is right.

Cristi A wrote:

> If the PTE is valid, you don’t have to map it. You can just make
apointer and read it.

But if I start from the first paging structure and I start to iterate
through all the entries when I reach a valid PTE it’s simpler to just
map the Physical Address than to figure out what VA I should look at.

No, it’s not, but I see you posted that in a followup.

But…
> If a structure happens
to be split between two different physical pages, those pages might be
miles apart in virtual space.

So I’ll end up failing.

But isn’t there a chance that a rootkit will remove some structures
from the linked lists or provide other structures when I ask nicely
for them?

Sure. And there is absolutely nothing you can do about that. If you
have been infected by a kernel root kit, you are screwed. You cannot
recover, even by poking through memory. You cannot assume anything.
The machine is junk. Format and start over.


Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.

I was thinking that security softwarares tries to not use APIs and other
system routines that can get hooked in order to give a report about the
state in which a machine is and that’s why I assumed it is possible to look
for some structures in this way. But I also know that it is impossible to
implement a perfect anti-rootkit solution so I accept this scenario as one
in which AV software will have little to nothing to do.

Thanks for the replies. I know this things are obvious, but sometimes I
need a bit of trial and error to really get it.

On Fri, Oct 17, 2014 at 10:07 PM, Tim Roberts wrote:

> Cristi A wrote:
>
> > If the PTE is valid, you don’t have to map it. You can just make a
> pointer and read it.
>
> But if I start from the first paging structure and I start to iterate
> through all the entries when I reach a valid PTE it’s simpler to just map
> the Physical Address than to figure out what VA I should look at.
>
>
> No, it’s not, but I see you posted that in a followup.
>
>
> But…
> > If a structure happens
> to be split between two different physical pages, those pages might be
> miles apart in virtual space.
>
> So I’ll end up failing.
>
> But isn’t there a chance that a rootkit will remove some structures from
> the linked lists or provide other structures when I ask nicely for them?
>
>
> Sure. And there is absolutely nothing you can do about that. If you have
> been infected by a kernel root kit, you are screwed. You cannot recover,
> even by poking through memory. You cannot assume anything. The machine is
> junk. Format and start over.
>
> –
> Tim Roberts, xxxxx@probo.com
> Providenza & Boekelheide, Inc.
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

You want to figure this stuff out? You want to see the tables, and examine how memory management works?

Great!

Use WinDbg, which provides you with all SORTS of built-in support for doing exactly this.

Give up writing your own code to do it. It’s annoying, it’ll be bug laden, and you’ll spend at least as much time puzzling over stupid shit in your own code as you will figuring out the mm structures.

And if you DO try something clever (like claiming your very own PTE and walking it around physical memory) don’t forget to INVD after each PTE change. Don’t ask me how I know.

Peter
OSR
@OSRDrivers

Lock these pages to a MDL by an API call, map the MDL, look at the data, unmap.


Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

wrote in message news:xxxxx@ntdev…
>I know that, I should have explained my (probably bad) idea more in depth:
> I want to traverse the paging structures looking for valid entries. Map those physical addresses, quickly look at their content, then unmap.
> Assuming x86 Non-PAE, I’ll get the page directory base from cr3 (I want to do this in DriverEntry so I’ll be in the context of the system). I know that PD will have 1024 entries of 4 bytes, each pointing to a Page Table. If a PDE is valid I go ahead and parse the PT. Again, PT will have 1024 entries of 4 bytes, each pointing to a physical address.
> If a PTE is valid and has the U/S bit = 0 (meaning that User mode can’t touch it) I use MmMapIoSpace and read it’s contents.
>
> The first problem I see with this is that I’m not sure it won’t crash the system on some cases (hey, if I’m not crashing my test machine it means I’m not learning) and second the contents may change from the time I check if it is valid an map it so I can still end up with wrong data or even crash the system.
>
> It feels like there should be a better solution for this, but right now I’m spinning in circles trying to figure out things.
>