James,
You may have put to many multipliers into the factoring of how many
management structures you need.
The NDIS_PACKET or NET_BUFFER is perfectly capable of keeping track of the
NDIS_BUFFER or MDLs you create (as copies or partial MDLs) - one private set
for each indication.
The only ‘resource’ you have that is shared between packets and thus needs
to be reference counted is the ‘page’ object in your ring buffer.
For those, you have (apparently) a fixed number of 256. I would just
allocate a slab of 256 simple structures that contain the control fields you
need (refCnt, PMDL, etc.)
You do not need to control ‘sharing’ of the partial MDLs because they are
only created and ‘owned’ by a single packet.
The packet (net-buffer, whatever) needs only a single ‘reference’ (pointer)
back to the ‘page’ that contains it’s packet data. Actualy, it may need two
because you mention a single packet might span two pages. The general case
may be it needs ((MAX_PACKET_SIZE / PAGE_SIZE) + 1) page references. Since
you get to allocate the NDIS_PACKETs yourself, you can always size them in
the ProtocolReserved area as large as you want. Anything *after*
ProtocolReserved[MIN_PROTOCOL_RESERVED] is actually the packet creators to
play with and can be thought of as (in your case) an arbitrary sized
MiniportReserved2 area.
Alternatively you could ‘chain’ the page structures together and manage the
reference counting with a ‘cascade’ so that when you process the returned
packet you release a page reference for every page in the chain that the
packet describes a portion of. You have the MDLs which point into the pages
and you have a reference to the ‘first’ page effected by the packet. More
‘work’ but less ‘memory’.
But I really don’t see the calculus that supports having fan-up of the
object count you outlined below. That could well be because I don’t
understand the guts of your driver but it seems to me you need to manage the
sharing of 256 objects and that?s about it.
Good Luck,
Dave
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of James Harper
Sent: Friday, January 23, 2009 6:57 AM
To: Windows System Software Devs Interest List
Subject: RE: [ntdev] multiple MDL’s referring to the same memory…
James,
Tacking stuff to the end of an MDL seems a bit dicey. How many of
these
buffers are we talking about? 100? 1000?
The ring can hold 256 pages, needing one MDL each. Each page could hold
3 packets, needing another 3 MDLs, and on top of that, when I refill the
ring I may not have received all the MDL’s back from NDIS (via
ReturnPacket), so potentially a thousand under extreme load.
Allocate your own data structure that contains a Reference Count,
pointer
to
the NDIS_BUFFER, etc. and get it all working reliably. Use a lookaside
list
if you are worried about alloc/free cycle time. Let the system level
the
lookaside depth based on usage. You can’t possibly be talking about
so
many of these things that doing so would be a problem (for memory
consumption). You are already committed to allocating an MDL for
them.
It is using nonpaged pool so I am supposed to be frugal with memory
usage, but high throughput equates to high memory usage. If memory runs
low then throughput will suffer, but that’s just the way it is.
That is probably the direction I’m looking towards. The NDIS_PACKET
structure contains 3 pointers that I can use however I want, so I just
need one of these to point to my structure with the list of original
MDLs and the reference count. The pointer is NULL when I don’t use
partial MDLs.
I was hoping I could do it per MDL rather than per Packet. Worst case, I
will get a 60K TCP packet (15 pages) from Linux that gets broken down
into 40 NDIS packets, each with two or three MDLs (one MDL for the
header, one for the data, and one more if the data spans two pages), so
I could return the physical memory when all the MDL’s are freed. This
way I can only return all the memory at once, and only when all the NDIS
packets I created are returned.
In my unsolicited opinion, trying to be so clever is inviting
complexity
to
overwhelm usefulness. The MDL is a system datastructure that is
does
not
have ‘extensibility’ designed in. Or at least that is true of the
more
opaque NDIS_BUFFER. What happens when the next Verifier starts
checking
MDLs for validity by looking at the allocated size in the heap or some
such? Boom. Your driver breaks.
Well… only when using the verifier
If you determine (from poolmon) that your little datastructure is
eating
away terribly at NP pool, you can look to optimize it. Slab
allocating a
‘block’ of smaller objects and sub-allocating is a well understood
solution
to dealing with heap header size swamping the size of a dynamically
allocated datastructure. But again, why build it until you measure
you
need it?
Your unsolicited opinion and ideas are greatly appreciated. Thanks
James
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer