AllocateCommonBuffer fails after reboot on machines with RAM > 128GB

Our driver uses AllocateCommonBuffer() in HwInitialize() function.
When the driver is loaded for the first time, the driver installation is successful and allocates and AllocateCommonBuffer() is also successful.
But, after reboot the AllocateCommonBuffer() function fails to allocate memory.
This is observed on machines with RAM > 138 GB
Used msconfig->boot options to deduce. If the RAM is <= 138GB then the AllocateCommonBuffer() is successful and for RAM > 138GB it fails.

The memory address obtained from AllocateCommonBuffer() are as follows,
First Installation
First allocation : PA:0x7d011000, VA: FFFF8C007F1AC000
Last allocation : PA:0x485e1000, VA: FFFF8C0087AC4000
After Reboot,
First allocation : PA:0x7cfe7000, VA: FFFFCC01D3040000
Last allocation : PA: ** 0x1a5000** , VA: FFFFCC01DEC6B000 (Before failure)

We are allocating chunks of 20K buffers
After reboot, there is huge address difference between the first and last allocation.

Any suggestions to debug and check where are we doing wrong ?

How much memory in total are you trying to allocate, and why are you doing
it in ‘chunks’ instead of one allocation?

Mark Roddy

@Mark_Roddy said:
How much memory in total are you trying to allocate, and why are you doing
it in ‘chunks’ instead of one allocation?

In total the whole driver allocated 80MB and in the failing case it fails around after 55MB
These are allocation for each individual entities/queues.

Mark Roddy

Well it likely has nothing at all to do with your problem, but I would just
make one 55MB allocation and divide that up on my own.
Mark Roddy

1 Like

he means 80MB of course

The advantage of multiple allocations would be if you are planning to free some of them sooner than others. Then some other component might be able to make use of that memory once you are finished with it. If they all have the same lifespan in your driver, then there is no value in making many small allocations - only the system overhead of keeping track of them individually

you should expect that the virtual addresses that your allocations are assigned will very dramatically each time your driver runs. That’s a feature by design in Windows to make it harder to exploit certain kinds of bugs

Thank you @Mark_Roddy , we will consider implementing as suggested. It’s an old driver and I’m not quite sure about the design of allocating smaller chunks. I am only guessing it to support on machines where contiguous allocations for higher memory fails. But, I will try your suggestion if it works in wider testing appears to be the only visible way out at the moment.

Thank you @MBond2, there are few allocation which are freed but majority of them are freed in ScsiStopAdapter. So, I think leaving the few rest of the memory can be allocated in single block.

But, just curious why would AllocateCommonBuffer() fails when there is more than sufficient memory is available ?