Use NDIS_RW_LOCK in Non-NDIS Drivers

It looks like the NDIS_RW_LOCK Reader/Writer lock is fairly useful. As far
as I can tell it has no NDIS-specific dependencies.

Can this lock be safely used in non-NDIS drivers such as WFP or WSK?

A lock with similar functionality can be built form scratch, but I would
like to use system-provided functions if possible.

Thomas F. Divine
http://www.pcausa.com

The NDIS team has no objections.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Thomas F. Divine
Sent: Tuesday, August 16, 2011 7:15 PM
To: Windows System Software Devs Interest List
Subject: [ntdev] Use NDIS_RW_LOCK in Non-NDIS Drivers

It looks like the NDIS_RW_LOCK Reader/Writer lock is fairly useful. As far as I can tell it has no NDIS-specific dependencies.

Can this lock be safely used in non-NDIS drivers such as WFP or WSK?

A lock with similar functionality can be built form scratch, but I would like to use system-provided functions if possible.

Thomas F. Divine
http://www.pcausa.com


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

Another excellent piece of work by the NDIS team that has general
applicability.

Another is the Network Module Registrar and Network Programming Interface
(NPI). A decent solution for allowing two drivers to safely swap vtables.

Thomas F. Divine
http://www.pcausa.com


From: “Jeffrey Tippet”
Sent: Tuesday, August 16, 2011 11:12 PM
To: “Windows System Software Devs Interest List”
Subject: RE: [ntdev] Use NDIS_RW_LOCK in Non-NDIS Drivers

> The NDIS team has no objections.
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Thomas F. Divine
> Sent: Tuesday, August 16, 2011 7:15 PM
> To: Windows System Software Devs Interest List
> Subject: [ntdev] Use NDIS_RW_LOCK in Non-NDIS Drivers
>
> It looks like the NDIS_RW_LOCK Reader/Writer lock is fairly useful. As far
> as I can tell it has no NDIS-specific dependencies.
>
> Can this lock be safely used in non-NDIS drivers such as WFP or WSK?
>
> A lock with similar functionality can be built form scratch, but I would
> like to use system-provided functions if possible.
>
> Thomas F. Divine
> http://www.pcausa.com
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
>
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

> It looks like the NDIS_RW_LOCK Reader/Writer lock is fairly useful.

Funny enough, but they seem to have deprecated it - they claim it is obsolete for NDIS 6.2 drivers

http://msdn.microsoft.com/en-us/library/ff567277(v=vs.85).aspx

Anton Bassov

They have NDIS_RW_LOCK_EX for NDIS 6.2.

Thomas

On Wed, Aug 17, 2011 at 12:16 AM, wrote:

>
> > It looks like the NDIS_RW_LOCK Reader/Writer lock is fairly useful.
>
> Funny enough, but they seem to have deprecated it - they claim it is
> obsolete for NDIS 6.2 drivers
>
> http://msdn.microsoft.com/en-us/library/ff567277(v=vs.85).aspx
>
> Anton Bassov
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>


Thomas F. Divine
http://www.pcausa.com

The deprecated version uses caller-allocated storage. However, the storage needs to contain per-processor information. So with Win7’s move to >64 processors, the caller-allocated storage can be insufficient to hold NDIS’s per-processor data. Therefore, the _EX version uses callee-allocated storage (so we can allocate more memory if the maximum number of processors ever increases).

Other than that, there is little difference between the two APIs. It’s ok to use this “deprecated” API on newer OSes – just be aware that if you use the deprecated version on Win7+, your perf will become problematic when running on a system with more than 64 processors, since then NDIS needs to fall back to a slow compatibility path. (And if you don’t care deeply about perf, then why are you using a fancy rw lock when a tiny little spinlock would be just fine? Spinlocks are quite efficient in CPU cycles if the lock is rarely contended, and they are maximally efficient in terms of storage space. You only need to break out the heavy-duty RW-lock if you know that reader contention is going to be high.)

From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Thomas Divine
Sent: Tuesday, August 16, 2011 9:21 PM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Use NDIS_RW_LOCK in Non-NDIS Drivers

They have NDIS_RW_LOCK_EX for NDIS 6.2.

Thomas
On Wed, Aug 17, 2011 at 12:16 AM, > wrote:

> It looks like the NDIS_RW_LOCK Reader/Writer lock is fairly useful.
Funny enough, but they seem to have deprecated it - they claim it is obsolete for NDIS 6.2 drivers

http://msdn.microsoft.com/en-us/library/ff567277(v=vs.85).aspx

Anton Bassov


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer


Thomas F. Divine
http://www.pcausa.com
— NTDEV is sponsored by OSR For our schedule of WDF, WDM, debugging and other seminars visit: http://www.osr.com/seminars To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

you can use Executive Resources to realize read/write lock, which can run at IRQL = PASSIVE_LEVEL or APC_LEVEL.

By using an executive resource, a driver can implement a read/write lock. Executive resources are designed for use with data structures that require exclusive access for writing but that can be read by several threads concurrently. Executive resources are not maintained in the system?s dispatcher database, so they usually are faster and more efficient than kernel dispatcher objects. A thread can acquire an executive resource for exclusive (write) access or for shared (read) access. Code that runs at IRQL=PASSIVE_LEVEL or APC_LEVEL can use executive resources.
An executive resource is a structure of type ERESOURCE, which must be allocated in nonpaged memory (for example, the device extension of the device object or nonpaged pool). An ERESOURCE structure must be naturally aligned; that is, the structure must be aligned on a 4-byte boundary on a 32-bit system and on an 8-byte boundary on a 64-bit system. The ERESOURCE structure itself is opaque to driver writers.

for more information, please refer to MS WDK doc.

thanks.

Your comment is certainly valid, however in the NDIS environment a lot of
work (well, as little as possible…) is done at DISPATCH_LEVEL.

For example, a lookup table with a large number of records may be
occasionally added to or updated from user-mode as PASSIVE_LEVEL (e.g.,
10Hz rate), but must be examined on per-packet basis as packets are received
at DISPATCH_LEVEL (150KHz rate). In this case the NDIS_RW_LOCK seemed to
provide a good synchronization method. I would think a somewhat better
choice than spinlock.

In what I have done over the years I really haven’t found configurations
where shared data was guaranteed to be accessed only IRQL < DISPATCH+LEVEL.
So, choices are limited.

Thanks for your comment!!!

Thomas F. Divine
http://www.rawether.net


From:
Sent: Thursday, August 18, 2011 9:37 PM
To: “Windows System Software Devs Interest List”
Subject: RE:[ntdev] Use NDIS_RW_LOCK in Non-NDIS Drivers

> you can use Executive Resources to realize read/write lock, which can run
> at IRQL = PASSIVE_LEVEL or APC_LEVEL.
>
> By using an executive resource, a driver can implement a read/write lock.
> Executive resources are designed for use with data structures that require
> exclusive access for writing but that can be read by several threads
> concurrently. Executive resources are not maintained in the system?s
> dispatcher database, so they usually are faster and more efficient than
> kernel dispatcher objects. A thread can acquire an executive resource for
> exclusive (write) access or for shared (read) access. Code that runs at
> IRQL=PASSIVE_LEVEL or APC_LEVEL can use executive resources.
> An executive resource is a structure of type ERESOURCE, which must be
> allocated in nonpaged memory (for example, the device extension of the
> device object or nonpaged pool). An ERESOURCE structure must be naturally
> aligned; that is, the structure must be aligned on a 4-byte boundary on a
> 32-bit system and on an 8-byte boundary on a 64-bit system. The ERESOURCE
> structure itself is opaque to driver writers.
>
> for more information, please refer to MS WDK doc.
>
> thanks.
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

> you can use Executive Resources to realize read/write lock, which can run at IRQL = PASSIVE_LEVEL

or APC_LEVEL.

Please note that we are speaking about NDIS_RW_LOC K that can be acquired at DPC level and, judging from the description provided by Jeffrey, seems to be just a heavy-weight version of Linux RW spinlock.
ERESOURCE is significantly heavier construct that is built on top of kernel dispatcher objects…

Anton Bassov

Using “Queued Spin Locks.”
KeAcquireInStackQueuedSpinLock()
KeReleaseInStackQueuedSpinLock ()

It has much better performance than ordinary spin locks.
Perhaps, it is a good choice for you.

thanks.

That is not always true. In stack queued spinlocks have their own perf profile. When under high contention , queued spin locks have better perf. For licks not under high contention , the regular spin lick can be better. As always, measure first , don’t guess and make changes blindly

d

debt from my phone

-----Original Message-----
From: xxxxx@hotmail.com
Sent: Thursday, August 18, 2011 8:47 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Use NDIS_RW_LOCK in Non-NDIS Drivers

Using “Queued Spin Locks.”
KeAcquireInStackQueuedSpinLock()
KeReleaseInStackQueuedSpinLock ()

It has much better performance than ordinary spin locks.
Perhaps, it is a good choice for you.

thanks.


NTDEV is sponsored by OSR

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer

The Network Module Registrar, which was mentioned recently, provides the
closest kernel-mode analog to a DLL. See:

http://msdn.microsoft.com/en-us/library/ff556950(v=VS.85).aspx

“Although the NMR was developed for use by network modules, the design of
the NMR architecture is sufficiently generic so that it can also be used by
software modules in other technology areas.”

Good luck!

Thomas F. Divine
http://www.pcausa.com


From: “Doron Holan”
Sent: Thursday, August 18, 2011 11:56 PM
To: “Windows System Software Devs Interest List”
Subject: RE: RE:[ntdev] Use NDIS_RW_LOCK in Non-NDIS Drivers

> That is not always true. In stack queued spinlocks have their own perf
> profile. When under high contention , queued spin locks have better perf.
> For licks not under high contention , the regular spin lick can be better.
> As always, measure first , don’t guess and make changes blindly
>
> d
>
> debt from my phone
>
> -----Original Message-----
> From: xxxxx@hotmail.com
> Sent: Thursday, August 18, 2011 8:47 PM
> To: Windows System Software Devs Interest List
> Subject: RE:[ntdev] Use NDIS_RW_LOCK in Non-NDIS Drivers
>
>
> Using “Queued Spin Locks.”
> KeAcquireInStackQueuedSpinLock()
> KeReleaseInStackQueuedSpinLock ()
>
> It has much better performance than ordinary spin locks.
> Perhaps, it is a good choice for you.
>
>
> thanks.
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer

>Executive resources are not maintained in the system?s dispatcher database, so they usually are faster and more

efficient than kernel dispatcher objects.

100% wrong, since ERESOURCE is a very complex structure which has 2 KEVENTs in it.


Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

I have to say I wish I had know about this NDIS implementation having gone
to the trouble, and having created trouble in a few places along the way
to roll my own spinlock r/w lock, which got a bit tricker when hot-add CPU
and processor group support was added.

It would have been nice if this was added to the Ke* family.

t.

On Thu, 18 Aug 2011, Thomas F. Divine wrote:

Your comment is certainly valid, however in the NDIS environment a lot of
work (well, as little as possible…) is done at DISPATCH_LEVEL.

For example, a lookup table with a large number of records may be
occasionally added to or updated from user-mode as PASSIVE_LEVEL (e.g., 10Hz
rate), but must be examined on per-packet basis as packets are received at
DISPATCH_LEVEL (150KHz rate). In this case the NDIS_RW_LOCK seemed to provide
a good synchronization method. I would think a somewhat better choice than
spinlock.

In what I have done over the years I really haven’t found configurations
where shared data was guaranteed to be accessed only IRQL < DISPATCH+LEVEL.
So, choices are limited.

Thanks for your comment!!!

Thomas F. Divine
http://www.rawether.net


From:
> Sent: Thursday, August 18, 2011 9:37 PM
> To: “Windows System Software Devs Interest List”
> Subject: RE:[ntdev] Use NDIS_RW_LOCK in Non-NDIS Drivers
>
>> you can use Executive Resources to realize read/write lock, which can run
>> at IRQL = PASSIVE_LEVEL or APC_LEVEL.
>>
>> By using an executive resource, a driver can implement a read/write lock.
>> Executive resources are designed for use with data structures that require
>> exclusive access for writing but that can be read by several threads
>> concurrently. Executive resources are not maintained in the system?s
>> dispatcher database, so they usually are faster and more efficient than
>> kernel dispatcher objects. A thread can acquire an executive resource for
>> exclusive (write) access or for shared (read) access. Code that runs at
>> IRQL=PASSIVE_LEVEL or APC_LEVEL can use executive resources.
>> An executive resource is a structure of type ERESOURCE, which must be
>> allocated in nonpaged memory (for example, the device extension of the
>> device object or nonpaged pool). An ERESOURCE structure must be naturally
>> aligned; that is, the structure must be aligned on a 4-byte boundary on a
>> 32-bit system and on an 8-byte boundary on a 64-bit system. The ERESOURCE
>> structure itself is opaque to driver writers.
>>
>> for more information, please refer to MS WDK doc.
>>
>> thanks.
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>