Hello,
i am new to this board and this is my first post here. Currently i am developing a kernel mode AoIP (Audio over IP) driver which sends and receives UDP packets through the WSK subsystem. The driver talks via ethernet to an interface box (IP Box) which belongs to a digital mixing console.
The system works as follows:
On driver startup, a callback routine OnReceive() is registered to the WSK subsystem using the WSK_CLIENT_DATAGRAM_DISPATCH structure to get notified about arrived packets. After associating the driver with an IP box, the box starts sending audio data each 2.5 ms (= 400 pkt/s) to the driver. When the driver gets notified about the packet, a workerthread is caused by KeSetEvent() to send his own audio data towards the box. In this way, the box controls the data flow and both audio engines (PC and mixing console) are in sync. For debugging purposes the driver records the time between the notifications. It should stay exactly at 2.5 ms. Using wireshark i verified that the IP box sends each packet exactly at the 2.5ms interval.
Basically that works fine. Audio is transmitted in both directions without noticable distortions. But now to my problem: In the last days i noticed that the interval varies in dependence on running user-mode applications. It seems that the WSK subsystem delivers the packets in bundles to the driver in intervals of 10-20 ms, not at the expected interval of 2.5 ms.
When i start the driver configuration app (a user mode application, built by co-workers) and activate a specific view in this app, some additional DDLs are loaded by the app. At this point, the driver timing changes. The call back routine is now called at an 2.5ms interval. For some reason the WSK subsystem delivers the receive notification now faster to the driver. After closing the app the interval falls back into the 10-20ms scheme. The system load is very low (< 5% at all time).
With ProcessExplorer i could identify the DLLs that may cause this behaviour CSCAPI.dll, CSCDLL.dll, cscui.dll. I think that a third-party component which is used in the app/view causes loading these DLLs.
Questions:
Q1: Why does the WSK deliver the packets in bundles and how can i force a better “granularity in time” for the callback routine? This would improve the stability of the transmission and should allow smaller buffers/delays.
Q2: How can it be, that a user mode component takes influence on a callback routine interval inside the kernel?
Greetings from germany
Marco
>subsystem delivers the receive notification now faster to the driver. After closing the app the interval
falls back into the 10-20ms scheme. The system load is very low (< 5% at all time).
Windows is not a realtime OS, so, such delays in TCP packet receive is normal.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
Thanks for your reply but please read my post completley.
I know Windows is no real time system. Unfortunatly many customers do not accept that fact. It is part my job to provide a solution that works as good as possible within the known limitations.
I choose UDP for transmission to prevent delays as they may occur with TCP. In the situation i described in my first post, windows is definitively able to send and receive packets in a stable interval of 2.5 ms (+/-1 ms) . I can see that and i can meassure that on wire and within applications.
My point of interesst is: why does that change if i load/unload third-party dlls (MS) in user mode? Not only in the moment of load/unload when the system load raises for short time.
There must be any relation, any way for user mode components to take influence on the network sub system to change the timing behaviour permanently. And this is what i am looking for.
Marco
Do you receive->send->re-start receive? If so, the send process might take
longer occasionally. Having multiple concurrent receives in-flight might
change the behavior somewhat.
In Windows kernel and user threads are both scheduled the same way. You are
using receivefrom in kernel-mode instead of user-mode to reduce the
kernel<->user transition latency - which is a good thing. But, that doesn’t
mean that kernel thread timing isn’t influenced by other thread operations.
Perhaps when the behavior is apparently improved when using a particular
view of some app you are using, the cause is indirectly raising the priority
of the threads involved.
If you are running WireShark on the machine that is having the timing
behavior test without it.
Good luck!
Thomas F. Divine
http://www.pcausa.com
From:
Sent: Wednesday, April 18, 2012 6:17 AM
To: “Windows System Software Devs Interest List”
Subject: [ntdev] WSK OnReceiveEvent Interval varies
> Hello,
>
> i am new to this board and this is my first post here. Currently i am
> developing a kernel mode AoIP (Audio over IP) driver which sends and
> receives UDP packets through the WSK subsystem. The driver talks via
> ethernet to an interface box (IP Box) which belongs to a digital mixing
> console.
>
> The system works as follows:
> On driver startup, a callback routine OnReceive() is registered to the WSK
> subsystem using the WSK_CLIENT_DATAGRAM_DISPATCH structure to get notified
> about arrived packets. After associating the driver with an IP box, the
> box starts sending audio data each 2.5 ms (= 400 pkt/s) to the driver.
> When the driver gets notified about the packet, a workerthread is caused
> by KeSetEvent() to send his own audio data towards the box. In this way,
> the box controls the data flow and both audio engines (PC and mixing
> console) are in sync. For debugging purposes the driver records the time
> between the notifications. It should stay exactly at 2.5 ms. Using
> wireshark i verified that the IP box sends each packet exactly at the
> 2.5ms interval.
>
> Basically that works fine. Audio is transmitted in both directions without
> noticable distortions. But now to my problem: In the last days i noticed
> that the interval varies in dependence on running user-mode applications.
> It seems that the WSK subsystem delivers the packets in bundles to the
> driver in intervals of 10-20 ms, not at the expected interval of 2.5 ms.
>
> When i start the driver configuration app (a user mode application, built
> by co-workers) and activate a specific view in this app, some additional
> DDLs are loaded by the app. At this point, the driver timing changes. The
> call back routine is now called at an 2.5ms interval. For some reason the
> WSK subsystem delivers the receive notification now faster to the driver.
> After closing the app the interval falls back into the 10-20ms scheme. The
> system load is very low (< 5% at all time).
>
> With ProcessExplorer i could identify the DLLs that may cause this
> behaviour CSCAPI.dll, CSCDLL.dll, cscui.dll. I think that a third-party
> component which is used in the app/view causes loading these DLLs.
>
> Questions:
> Q1: Why does the WSK deliver the packets in bundles and how can i force a
> better “granularity in time” for the callback routine? This would improve
> the stability of the transmission and should allow smaller buffers/delays.
>
> Q2: How can it be, that a user mode component takes influence on a
> callback routine interval inside the kernel?
>
> Greetings from germany
>
> Marco
>
>
> —
> NTDEV is sponsored by OSR
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
> My point of interesst is: why does that change if i load/unload third-party dlls (MS) in user mode?
Probably any nonzero load on the system will cause the same effect.
Just 2ms is probably not achievable in Windows at all.
Your last chance is to call timeBeginPeriod and switch the hardware timer to 1ms period instead of 10 (or was it 15?) ms.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
Also, raise the priority of your thread to realtime.
–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com
wrote in message news:xxxxx@ntdev…
> Thanks for your reply but please read my post completley.
>
> I know Windows is no real time system. Unfortunatly many customers do not accept that fact. It is part my job to provide a solution that works as good as possible within the known limitations.
>
> I choose UDP for transmission to prevent delays as they may occur with TCP. In the situation i described in my first post, windows is definitively able to send and receive packets in a stable interval of 2.5 ms (+/-1 ms) . I can see that and i can meassure that on wire and within applications.
>
> My point of interesst is: why does that change if i load/unload third-party dlls (MS) in user mode? Not only in the moment of load/unload when the system load raises for short time.
>
> There must be any relation, any way for user mode components to take influence on the network sub system to change the timing behaviour permanently. And this is what i am looking for.
>
> Marco
>
Thanks for your answers.
Wireshark was running on a second PC. I used a managed switch to get copies of the packets over a monitor port.
The receive routine is a callback which gets called from the WSK. After processing an incoming packet it sets only a event and returns. The worker thread waits for the event and starts sending the neccessary number of packets (most time the number of packets is 1). There is no need to restart the receive process because the callback stays active until the socket is closed. There is also no transition between user and kernel mode because the data stays in the driver as long there is no audio application that performs record/playback. The sender thread and all IRP priorities are already set to the highest possible values.
After reading lot of papers i found a function ExSetTimerResolution() which allows to set the hardware timer resolution. I set it to 500ms and the behaviour is acceptable.
Let me show the timings i got: (| = packet arrives, _ = pause (n*2.5ms) )
Best archived as described in the situation on the first post:
… |||||_| … (constant packet flow)
before showing the special view:
… ||||_____||||||||____ … (packets gets “bundled”)
now after setting the timer:
… ||__|||||__||_ … (nearly constant packet flow)
i hope you understand the graphics 
regards
marco
Finally i got it. There is a user space component that calls timeBeginPeriod(1) which has effect even in the kernel.
On driver initialisation i do a call to ExSetTimerResolution(5000) and now it works as expected.
xxxxx@mcx-home.de wrote:
Finally i got it. There is a user space component that calls timeBeginPeriod(1) which has effect even in the kernel.
On driver initialisation i do a call to ExSetTimerResolution(5000) and now it works as expected.
You can’t do that! If the timer resolution has been set to 1, then
there is some user-mode component that needs the resolution to be 1.
DirectShow sets the resolution to 1 so that video frames can be
presented with proper timing.
For you to override that is simply arrogance. You are going to break
multimedia streaming just for the sake of your own convenience.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
On 19-Apr-2012 19:27, Tim Roberts wrote:
xxxxx@mcx-home.de wrote:
> Finally i got it. There is a user space component that calls timeBeginPeriod(1) which has effect even in the kernel.
>
> On driver initialisation i do a call to ExSetTimerResolution(5000) and now it works as expected.
You can’t do that! If the timer resolution has been set to 1, then
there is some user-mode component that needs the resolution to be 1.
DirectShow sets the resolution to 1 so that video frames can be
presented with proper timing.
For you to override that is simply arrogance. You are going to break
multimedia streaming just for the sake of your own convenience.
Tim,
5000 is 0.5 ms, so the other apps that want 1.0 ms won’t suffer.
Yes, NT6 allows resolutions less than 1 ms!
Regards,
– pa
Use this capability with extreme care. Jacking up the timer frequency has ruinous consequences for battery life.
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Pavel A
Sent: Thursday, April 19, 2012 11:35 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] WSK OnReceiveEvent Interval varies
On 19-Apr-2012 19:27, Tim Roberts wrote:
xxxxx@mcx-home.de wrote:
> Finally i got it. There is a user space component that calls timeBeginPeriod(1) which has effect even in the kernel.
>
> On driver initialisation i do a call to ExSetTimerResolution(5000) and now it works as expected.
You can’t do that! If the timer resolution has been set to 1, then
there is some user-mode component that needs the resolution to be 1.
DirectShow sets the resolution to 1 so that video frames can be
presented with proper timing.
For you to override that is simply arrogance. You are going to break
multimedia streaming just for the sake of your own convenience.
Tim,
5000 is 0.5 ms, so the other apps that want 1.0 ms won’t suffer.
Yes, NT6 allows resolutions less than 1 ms!
Regards,
– pa
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
On 19-Apr-2012 21:42, Skywing wrote:
Use this capability with extreme care. Jacking up the timer frequency has ruinous consequences for battery life.
Which battery?? There still are machines that have no battery at all, or
where the whole PC part is not the main consumer (like in a car PC).
/* apropos - when Win8 enters the awesome “connected standby” state and
all normal apps are turned off - will timer resolution set by them stay
effective? */
Regards,
– pa
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Pavel A
Sent: Thursday, April 19, 2012 11:35 AM
To: Windows System Software Devs Interest List
Subject: Re:[ntdev] WSK OnReceiveEvent Interval varies
On 19-Apr-2012 19:27, Tim Roberts wrote:
> xxxxx@mcx-home.de wrote:
>> Finally i got it. There is a user space component that calls timeBeginPeriod(1) which has effect even in the kernel.
>>
>> On driver initialisation i do a call to ExSetTimerResolution(5000) and now it works as expected.
>
> You can’t do that! If the timer resolution has been set to 1, then
> there is some user-mode component that needs the resolution to be 1.
> DirectShow sets the resolution to 1 so that video frames can be
> presented with proper timing.
>
> For you to override that is simply arrogance. You are going to break
> multimedia streaming just for the sake of your own convenience.
Tim,
5000 is 0.5 ms, so the other apps that want 1.0 ms won’t suffer.
Yes, NT6 allows resolutions less than 1 ms!
Regards,
– pa
>Which battery?? There still are machines that have no battery at all, or where the whole PC part is not the main consumer (like in a car PC).
Even owners of AC-powered systems benefit when their PC uses less power. Everybody likes a lower electric bill (or in the case of a car PC, a lower petrol bill).
/* apropos - when Win8 enters the awesome “connected standby” state and all normal apps are turned off - will timer resolution set by them stay effective? */
Yes, but I don’t think it matters much. With a dynamic tick, the kernel only has to fire the timer interrupt if some timer is actually queued to expire in the near future. While in Connected Standby, few usermode timers are allowed to fire. Furthermore, the OS itself is careful to not schedule any noisy timers. So while timers may be scheduled with ultra-fine granularity, it doesn’t matter because there aren’t many timers *to* schedule.
Of course, one bad driver can still ruin it for everyone, but we’re paying a LOT of attention to timer behavior while qualifying drivers for ARM. Even merely using NDIS’s MiniportCheckForHangEx in a NIC driver for an AOAC-class machine will probably get me annoyed with you. (We added a small note to its documentation already, but a more strongly-worded version is in the works.)
Thanks for your thoughts.
Our customers want to run the solution 24/7 for broadcasting. The main goal is a stable audio transmission between automation systems and digital mixing consoles.
On this background is the use power saving features or showing videos on systems that perform such time critical operations extremly contraproductive …
Regards
marco