some problems about monitor memory.

NTDEV
1

hello, everyone.
I meet some problems about monitor the memory owned by a specific
process.
1.when the specific process runing, it asks for how many memorys?
2.when the specific process exits , it generates memory leaks?
3.i want to know all operations(read/write, allocate/free) on a block of
memory owned by the specific process.

for problem 1, i hook all the memory allocate/free
functions(heapalloc,virtualalloc,…).

for problem 2, when i hook all memory allocate/free functions, i use some
variables store how many times the functions called, when the process
exits, i check whether the allocate counts is matching the free counts, if
not maybe has memory leak.

for problem 3, i use the VirtualProtect function modify the protection of
the block of memory that i want to monitor to NO ACCESS. then I catch the
EXCEPTION_ACCESS_VIOLATION excption, if the adress occurs in the range of
this block of memory, i do some records,and restore the old protection of
this block of memory,then do the normal operations and set single step
flag, when i handle the EXCEPTION_SINGLE_STEP, I modify the protection of
the block of memory to NO ACCESS again. but this method has very low
performance,when it have some operatsons on the block of memory, it
conitues EXCEPTION_ACCESS_VIOLATION and excutes by single step. the usage
of CPU is 100%. but i don’t know the other methed that can know the
read/write status of memory.

it’s all my problems , Can somebody help me?
If you know where i was wrong , pls points out.if you have any other good
methods, pls tell me also,thanks! thanks! thanks!!!

2

This is an interesting subject because a couple of my collegues were looking
at something similar (and I don’t think they really came to any conclusion).

Anyway, would it be possible to explain a little bit more what you’re
actually trying to achieve. I understand the need to know how many calls to
memory allocations, etc. But I don’t really understand what you’re doing
trying to track every single access to a specific section of memory (or many
sections of memory?)

Depending on exactly what you’re trying to do, it may be better to just
record that a particular page has been read/written to.

Naturally, if you take a single-step exception for every memory access to a
particular page, and call the OS during the exception (to change the
protection), the processor will be very busy doing this, and not get much
time to do other things. Taking an exception is not an easy operation in the
first place (CPU has to sync up and finish all pending operations, then jump
to a completely different place through a table entry, then save some state,
and call your function, then return through a complicated route, and
probably do some house-keeping too). It’s not unlikely that every
modification of the page table also flushes the page table (write’s to CR3
will flush the page table, amongst other things). The consequence of all
this is that you’re really making it hard for the processor to actually get
any reasonable work done. It’s probably about two to three orders of
magnitude slower to do this than a single memory read. And I’m not including
your code to do whatever accounting you do in your processor of the memory
access.

Sorry for not providing a simple answer, but just asking more questions…

–
Mats

-----Original Message-----
From: gameplugin [mailto:xxxxx@yahoo.com.cn]
Sent: Friday, December 19, 2003 3:13 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] some problems about monitor memory.

hello, everyone.
I meet some problems about monitor the memory owned by a specific
process.
1.when the specific process runing, it asks for how many memorys?
2.when the specific process exits , it generates memory leaks?
3.i want to know all operations(read/write, allocate/free)
on a block of
memory owned by the specific process.

for problem 1, i hook all the memory allocate/free
functions(heapalloc,virtualalloc,…).

for problem 2, when i hook all memory allocate/free
functions, i use some
variables store how many times the functions called, when the process
exits, i check whether the allocate counts is matching the
free counts, if
not maybe has memory leak.

for problem 3, i use the VirtualProtect function modify the
protection of
the block of memory that i want to monitor to NO ACCESS. then
I catch the
EXCEPTION_ACCESS_VIOLATION excption, if the adress occurs in
the range of
this block of memory, i do some records,and restore the old
protection of
this block of memory,then do the normal operations and set single step
flag, when i handle the EXCEPTION_SINGLE_STEP, I modify the
protection of
the block of memory to NO ACCESS again. but this method has very low
performance,when it have some operatsons on the block of memory, it
conitues EXCEPTION_ACCESS_VIOLATION and excutes by single
step. the usage
of CPU is 100%. but i don’t know the other methed that can know the
read/write status of memory.

it’s all my problems , Can somebody help me?
If you know where i was wrong , pls points out.if you have
any other good
methods, pls tell me also,thanks! thanks! thanks!!!

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@3dlabs.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

3

You may be happy (or sad) to learn that NT already implements a good
deal of this for you. The user-mode heaps have excellent debug support,
although you must use a user-mode debugger that understands how to
interact with the heaps in debug mode. Combined, they can report on the
number and length of each heap block allocated, the thread stack capture
when the block was allocated (if enabled), the block tag (if enabled),
and a few other features. Tools like DH.EXE, although crude, can be
used to detect heap leaks. (You dump the heap at process start up, then
at process exit. There are tools for finding the difference (i.e.
leaks).)

A lot of the tools for debugging the heap are unfortunately not
documented very well, and many have very terse interfaces. However, all
of the technology is fairly mature – it’s been used for a very long
time internally at Microsoft, and has been a real boon. The heaps are
automatically put in debug mode if you start the process under a
debugger. Querying the heaps requires using Microsoft’s user-mode
debuggers (NTSD, CDB). Find and download the most recent version of the
WinDbg debugger (3.x – not to be confused with that horrid excuse for a
debugger, WinDbg 1.x). WinDbg is (these days) a set of fairly powerful,
mature user- and kernel-mode debuggers. And, they’re free.

For #3, you can do what you describe, or you can use data breakpoints in
WinDbg or VS .Net. Data breakpoints do exactly what you describe,
except I believe they are used to monitor access to a specific location
in memory (usually with machine-word granularity (i.e. 32-bit or
64-bit)). This obviously won’t cover the entire block you want to
cover, so you may still want to do it your way. And yes, doing what you
are describing (beating up on the VM protections) will be very, very,
very slow. The exception handling system is built for robustness and
widely-scoped reporting, not speed.

The debugger can be downloaded at
http://www.microsoft.com/whdc/ddk/debugging/ . Don’t let the “ddk” in
the URL mislead you – it’s a full user-mode debugger as well. And it
actually has documentation these days – look for the !heap command,
which will give you an obscene amount of information about the process
heaps.

– arlie

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of gameplugin
Sent: Friday, December 19, 2003 10:13 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] some problems about monitor memory.

hello, everyone.
I meet some problems about monitor the memory owned by a specific
process.
1.when the specific process runing, it asks for how many memorys?
2.when the specific process exits , it generates memory leaks?
3.i want to know all operations(read/write, allocate/free) on a block
of memory owned by the specific process.

for problem 1, i hook all the memory allocate/free
functions(heapalloc,virtualalloc,…).

for problem 2, when i hook all memory allocate/free functions, i use
some variables store how many times the functions called, when the
process exits, i check whether the allocate counts is matching the free
counts, if not maybe has memory leak.

for problem 3, i use the VirtualProtect function modify the protection
of the block of memory that i want to monitor to NO ACCESS. then I catch
the EXCEPTION_ACCESS_VIOLATION excption, if the adress occurs in the
range of this block of memory, i do some records,and restore the old
protection of this block of memory,then do the normal operations and set
single step flag, when i handle the EXCEPTION_SINGLE_STEP, I modify the
protection of the block of memory to NO ACCESS again. but this method
has very low performance,when it have some operatsons on the block of
memory, it conitues EXCEPTION_ACCESS_VIOLATION and excutes by single
step. the usage of CPU is 100%. but i don’t know the other methed that
can know the read/write status of memory.

it’s all my problems , Can somebody help me?
If you know where i was wrong , pls points out.if you have any other
good methods, pls tell me also,thanks! thanks! thanks!!!

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@sublinear.org To
unsubscribe send a blank email to xxxxx@lists.osr.com

4

Our BoundsChecker for Apps does that and more.

Alberto.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of
xxxxx@3Dlabs.com
Sent: Friday, December 19, 2003 10:34 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] RE: some problems about monitor memory.

This is an interesting subject because a couple of my collegues were looking
at something similar (and I don’t think they really came to any conclusion).

Anyway, would it be possible to explain a little bit more what you’re
actually trying to achieve. I understand the need to know how many calls to
memory allocations, etc. But I don’t really understand what you’re doing
trying to track every single access to a specific section of memory (or many
sections of memory?)

Depending on exactly what you’re trying to do, it may be better to just
record that a particular page has been read/written to.

Naturally, if you take a single-step exception for every memory access to a
particular page, and call the OS during the exception (to change the
protection), the processor will be very busy doing this, and not get much
time to do other things. Taking an exception is not an easy operation in the
first place (CPU has to sync up and finish all pending operations, then jump
to a completely different place through a table entry, then save some state,
and call your function, then return through a complicated route, and
probably do some house-keeping too). It’s not unlikely that every
modification of the page table also flushes the page table (write’s to CR3
will flush the page table, amongst other things). The consequence of all
this is that you’re really making it hard for the processor to actually get
any reasonable work done. It’s probably about two to three orders of
magnitude slower to do this than a single memory read. And I’m not including
your code to do whatever accounting you do in your processor of the memory
access.

Sorry for not providing a simple answer, but just asking more questions…

–
Mats

-----Original Message-----
From: gameplugin [mailto:xxxxx@yahoo.com.cn]
Sent: Friday, December 19, 2003 3:13 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] some problems about monitor memory.

hello, everyone.
I meet some problems about monitor the memory owned by a specific
process.
1.when the specific process runing, it asks for how many memorys?
2.when the specific process exits , it generates memory leaks?
3.i want to know all operations(read/write, allocate/free)
on a block of
memory owned by the specific process.

for problem 1, i hook all the memory allocate/free
functions(heapalloc,virtualalloc,…).

for problem 2, when i hook all memory allocate/free
functions, i use some
variables store how many times the functions called, when the process
exits, i check whether the allocate counts is matching the
free counts, if
not maybe has memory leak.

for problem 3, i use the VirtualProtect function modify the
protection of
the block of memory that i want to monitor to NO ACCESS. then
I catch the
EXCEPTION_ACCESS_VIOLATION excption, if the adress occurs in
the range of
this block of memory, i do some records,and restore the old
protection of
this block of memory,then do the normal operations and set single step
flag, when i handle the EXCEPTION_SINGLE_STEP, I modify the
protection of
the block of memory to NO ACCESS again. but this method has very low
performance,when it have some operatsons on the block of memory, it
conitues EXCEPTION_ACCESS_VIOLATION and excutes by single
step. the usage
of CPU is 100%. but i don’t know the other methed that can know the
read/write status of memory.

it’s all my problems , Can somebody help me?
If you know where i was wrong , pls points out.if you have
any other good
methods, pls tell me also,thanks! thanks! thanks!!!

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@3dlabs.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.

5

> For #3, you can do what you describe, or you can use data

breakpoints in
WinDbg or VS .Net. Data breakpoints do exactly what you describe,
except I believe they are used to monitor access to a
specific location
in memory (usually with machine-word granularity (i.e. 32-bit or
64-bit)). This obviously won’t cover the entire block you want to
cover, so you may still want to do it your way. And yes,
doing what you
are describing (beating up on the VM protections) will be very, very,
very slow. The exception handling system is built for robustness and
widely-scoped reporting, not speed.

Data breakpoints are however limited in number, for x86 it’s 4 data
breakpoints as maximum. Once you’ve set 4 different locations, you can’t set
any more.

–
Mats

6

mats:
thanks your reply, thanks!
the reason that I use single step is when i handle the
EXCEPTION_ACCESS_VIOLATION , after I check the Virtual memory address that
occurs the exception and determine whether to do my monitor work, I must
restore the original Protection of this memory so that the specific
process can works normally. but an other problem occurs, if I don’t use
the single step, I haven’t the chance to set the memory’s protection to NO
ACCESS, and I can not know the other access to this memory.
I want to use a timer, but it’s not avalible.

the work I want to do is: monitor a specific block memory belongs to a
process, and I want to know all operations on this block
memory(read/write,allocation/free).

7

Arlie Davis, thanks you too
currentlly, I use the debug method to do my program.
only monitor the heap is not enough, because almost all the memory used by
a process falls into one of these categories:
Executable code in a loaded module
Read-only data in a loaded module (including resources)
Writable memory in a module (for example, the .data section)
Win32® heaps (including the default heap)
Suballocated heaps (for example, from the Visual C++® runtime library)
VirtualAlloced memory
Memory-mapped files
Thread stacks
Environment
System data structures (including the Thread Informa­tion Block and page
tables)
more details pls see the msdn’s under the hood November 1999

so I want to hook all the memory allocate/free functions, but this method
is not avalible for a debug version program, because the debug version
program’s new or malloc function implemented by crt function
_heap_alloc_dbg,ect… but it’s release version will implemented by
heapalloc.

8

Moreira:
I admire company compuware. and I also read the John Robbins’s book <>,
it gives me help.
and can you give me some hints? thanks!

9

If you have any specific question, I’ll be glad to answer, that is, if I
can. Or you can send an email to our xxxxx@compuware.com.

Alberto.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of gameplugin
Sent: Friday, December 19, 2003 10:39 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] RE: some problems about monitor memory.

Moreira:
I admire company compuware. and I also read the John Robbins’s book
<>,
it gives me help.
and can you give me some hints? thanks!

—
Questions? First check the Kernel Driver FAQ at
http://www.osronline.com/article.cfm?id=256

You are currently subscribed to ntdev as: xxxxx@compuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

The contents of this e-mail are intended for the named addressee only. It
contains information that may be confidential. Unless you are the named
addressee or an authorized designee, you may not copy or use it, or disclose
it to anyone else. If you received it in error please notify us immediately
and then destroy it.