Allocation of reparsed file name buffers - memory fragmentation considerations

Hi,

I am testing a filter, which would redirect literally all file
opens (this is just a test, nothing specific in mind). And I am worried about the possibility of memory fragmentation
after some number of file name string buffers get allocated and
deallocated, which will happen, and is only a matter of time.

Now, aside form allocating, say 10K, buffers when the driver loads,
and using those, is there any mechanism which would allow me to
replace the file names, and still not get memory fragmentations, at
some point?
10K string will get eaten within minutes, if not seconds, and even
if I allocate only 256 BYTES (which I would often require), that is
not a small amount of memory.

The second idea was to allocate 4K aligned buffers only, hoping
that would simply lead to less fragmentation, but a lookaside list
would be the best approach. I just don’t see any present in the API.

Does IoReplaceFileObjectName use lookaside lists internally, by any chance?

Kind regards, Dejan.

I have no idea why the styling appeared :wink:

Don’t worry too much about that as the memory manager behind the scenes is not so oblivious to this and does its best to keep things in check.
You could use lookaside lists for one, but in general there are allocations happening all the time in the kernel.
I would worry more about performance rather than fragmentation in your case.

Not so. We have seen memory fragmentation in the past, and since we
need to support legacy OSes (XP and 7), we need to consider the memory
fragmentation. It happens, and it is far from trivial.

I already know I need to worry about it, I just do not know the best
mitigation :wink:

On 1/19/19, Gabriel_Bercea wrote:
> OSR https://community.osr.com/
> Gabriel_Bercea commented on Allocation of reparsed file name buffers -
> memory fragmentation considerations
>
> Don’t worry too much about that as the memory manager behind the scenes is
> not so oblivious to this and does its best to keep things in check.
>
> You could use lookaside lists for one, but in general there are allocations
> happening all the time in the kernel.
>
> I would worry more about performance rather than fragmentation in your
> case.
>
> –
> Reply to this email directly or follow the link below to check it out:
> https://community.osr.com/discussion/comment/292218#Comment_292218
>
> Check it out:
> https://community.osr.com/discussion/comment/292218#Comment_292218
>

I have no idea why the styling appeared

I have a one word answer: Markdown.

Like Mr. Bercea, I am having trouble believing allocating and reallocating file name strings is going to cause you fragmentation problems. But if you insist this is an issue… how about writing your own allocator? At startup, allocate a big chunk of pool… and the handle the string allocation yourself from within the chunk?? I don’t see this being a great idea, but…

Peter

That’s the only thing I have at the moment, but however big that
allocation is, it will run out at some point. Reallocating a new
batch, when the old is near completion is the best I can do
afterwards.

This is under the presumption that there are a LOT of redirections (sa
I mentioned in the tune of 10K a minute possibly).

I know there were fragmentation issues when we allocated Flt contexts
(esseentially file name buffers with a few flags) without lookaside
lists. It was a metter of time only before any machine would run out
of memory, due to fragmentation, even though there is technically
enough free memory.

On 1/19/19, Peter_Viscarola_(OSR)
wrote:
> OSR https://community.osr.com/
> Peter_Viscarola_(OSR) commented on Allocation of reparsed file name buffers
> - memory fragmentation considerations
>
>> I have no idea why the styling appeared
>
> I have a one word answer: Markdown.
>
> Like Mr. Bercea, I am having trouble believing allocating and reallocating
> file name strings is going to cause you fragmentation problems. But if you
> insist this is an issue… how about writing your own allocator? At
> startup, allocate a big chunk of pool… and the handle the string
> allocation yourself from within the chunk?? I don’t see this being a great
> idea, but…
>
> Peter
>
> –
> Reply to this email directly or follow the link below to check it out:
> https://community.osr.com/discussion/comment/292226#Comment_292226
>
> Check it out:
> https://community.osr.com/discussion/comment/292226#Comment_292226
>

But if you insist this is an issue… how about writing your own allocator?

And that’s fun as well. I’ve written one just for “fun” for a kernel library I have which has all sorts of utility functions and functionalities. It’s not trivial, but also not the most difficult task you’ll ever have.

Again though, you have to realize that according to what you are saying:

I am testing a filter, which would redirect literally all file opens
That’s not a lot of opens. Especially if you actually free the memory after the open :smile:
All of those file opens have name allocates associated with them already. You are implying that allocating another Unicode string for each one during the Create path, fragment your memory. Hard to believe, unless you forget to free the memory. Can you test your driver with verifier and unload it to check for leaks ?
There is something more fragmenting than file system activity and that is registry actvity. There are by far more reg api/sec going on. Just open procmon and allow for all the reg activity to flood in and all FS activity and compare. I did one of these experiments for you.
Procmon running for 30 seconds on idle system: 364 filesystem events, 6909 reg events.
Procmon running for 30 seconds on system doing random activity: 20k filesystem Creates/Opens, 84k RegOpens/Creates
You can try that yourself.

And for each one (reg open/create) imagine you need to create a context of your own and then maybe you want to store the name as well. And perhaps you want to create a reg virtualization filter so you would need to allocate the “target key name” as well. Those filters would be impossible to write, but somehow they exist. The only issue that can come up from such an active callback is performance.

I would advise using verifier and make sure you’re not leaking memory.

Gabriel.

You are hunting in the wrong area :slight_smile:

When I allocate contexts, they DO fragment the memory, without any
leaks. Leaks would NOT fragment the memory, as they remain in a used
space.
We had this confirmed by Microsoft kernel guys, it is not a random
thought, leak or something of sorts.
We mitigated that one easily by allocating contexts from a lookaside
list. Simple, and easily doable.

A file name buffer, when used internally by us only, is also allocated
from a lookaside list, thus the fragmentation is not an issue.

The REPARSE buffers are another thing. We can only allocate them - we
do not free them, and we cannot make them appear in any of our
lookasides.

Now, you are considering 30 minutes of activity only - how about 30
weeks? This needs to run on a server, and also on machines that do not
restart at all for a month at least.

So, any thoughts on how to minimize the fragmentation?

On 1/21/19, Gabriel_Bercea wrote:
> OSR https://community.osr.com/
> Gabriel_Bercea commented on Allocation of reparsed file name buffers -
> memory fragmentation considerations
>
>> But if you insist this is an issue… how about writing your own
>> allocator?
>
> And that’s fun as well. I’ve written one just for “fun” for a kernel library
> I have which has all sorts of utility functions and functionalities. It’s
> not trivial, but also not the most difficult task you’ll ever have.
>
> Again though, you have to realize that according to what you are saying:
>
>> I am testing a filter, which would redirect literally all file opens
>
> That’s not a lot of opens. Especially if you actually free the memory after
> the open
>
> All of those file opens have name allocates associated with them already.
> You are implying that allocating another Unicode string for each one during
> the Create path, fragment your memory. Hard to believe, unless you forget to
> free the memory. Can you test your driver with verifier and unload it to
> check for leaks ?
>
> There is something more fragmenting than file system activity and that is
> registry actvity. There are by far more reg api/sec going on. Just open
> procmon and allow for all the reg activity to flood in and all FS activity
> and compare. I did one of these experiments for you.
>
> Procmon running for 30 seconds on idle system: 364 filesystem events, 6909
> reg events.
>
> Procmon running for 30 seconds on system doing random activity: 20k
> filesystem Creates/Opens, 84k RegOpens/Creates
>
> You can try that yourself.
>
> And for each one (reg open/create) imagine you need to create a context of
> your own and then maybe you want to store the name as well. And perhaps you
> want to create a reg virtualization filter so you would need to allocate the
> “target key name” as well. Those filters would be impossible to write, but
> somehow they exist. The only issue that can come up from such an active
> callback is performance.
>
> I would advise using verifier and make sure you’re not leaking memory.
>
> Gabriel.
>
> –
> Reply to this email directly or follow the link below to check it out:
> https://community.osr.com/discussion/comment/292242#Comment_292242
>
> Check it out:
> https://community.osr.com/discussion/comment/292242#Comment_292242
>

The REPARSE buffers are another thing. We can only allocate them - we do not free them, and we cannot make them appear in any of our lookasides.
Are you sure about that ?
First of all if you don’t free these buffers that you say are very high on the activity for months on end I assure you the fragmentation is not your worry but rather the memory capacity.
Secondly, of course you can make everything fit in a lookaside.
As @“Peter_Viscarola_(OSR)” suggested one way would be to write your own allocator.

The allocator could behind the scene, depending on the size you want to allocate would allocate from an internal pool of lookaside lists and free accordingly.

Very cruel skeleton idea comes to mind:

PVOID MyCustomAlloc(ULONG Size)
{
PVOID mem = ExAllocateFromLookaside(Get_Right_Lookaside(Size + sig_size));
//write a signature so you know where to free from
WriteSignature(mem);
return mem+sig_size;
}

VOID MyCustomFree(mem)
{
PVOID mem_header = mem-sig_size;
ExFreeToLookaside(mem_header, Get_RIghtLookaside(*((PULONG)mem_header));
}

You now initialize a number of looksides in driverEntry varying from the low end size of a memory allocation to a high end size of the memory allocations this pool of lookasides could serve as fragmentation protection.
But then again, fragmentation protection is useless in your case as you say you don’t free the buffers, so what’s the point of using lookasides anyway.

Can you explain why reparse buffes cannot be freed ? Also, let’s assume the fragmentation never occurs, if you never free them how do you cope with limited memory ?

If you step through IoReplaceFileObjectName you’ll see it does not use a lookaside list. However, it is smart enough to use the existing buffer if the new name is <= the max buffer length of the old name. It also appears to overallocate the buffer if it’s below certain thresholds (though I’m not exactly sure why the numbers there were chosen).

It would be very difficult for the OS to change the way name buffers are allocated at this point and maintain backwards compatibility.

Do you know what a reparse file name buffer is?

There are no leaks… please don’t dillute the topic into a wrong direction.

Just so we are clear - those buffers cannot be freed by us! They are
freed by the I/O manager, so we cannot put them back into a lookaside.

Unless IoReplaceFileObjectName uses lookasides itself, fragmentation
will occur :frowning:

On 1/23/19, Dejan Maksimovic <dejan.maksimovic> wrote:
> Do you know what a reparse file name buffer is?
>
> There are no leaks… please don’t dillute the topic into a wrong
> direction.
>
> –
>
>
> Kind regards, Dejan.
></dejan.maksimovic>