Hi,
I have been looking and most of the articles have found have been for WDM, how may I go about memory sharing with a user application in WDK? I am looking at actually using memory rather than a virtual file which the mirror driver uses. I’m open to suggestions of a better method too, I looked into inverted calls but I am worried because the driver may give a lot of small pieces of information that several inverted calls may not be enough.
What is the intended use here? Shared memory has its place but you
really have to be doing extremely high speed communications to make it
worthwhile. You can on a typical current system do well over a million
inverted calls a second if you push it. So how many hundreds of
megabytes per second do you think you are going to be doing?
Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
“xxxxx@hotmail.com” wrote in
message news:xxxxx@ntdev:
> Hi,
>
> I have been looking and most of the articles have found have been for WDM, how may I go about memory sharing with a user application in WDK? I am looking at actually using memory rather than a virtual file which the mirror driver uses. I’m open to suggestions of a better method too, I looked into inverted calls but I am worried because the driver may give a lot of small pieces of information that several inverted calls may not be enough.
I won’t be doing that much. The code has already been writing, it is just adding extra code to record the movements of a mouse using the callback already developed. If a invert call can handle that then that is the method I will look at. How can I be sure that I am capturing all the data and not one is missed without waiting inside the driver for the service to issue the next inverted call. How many do you think I will need to be sure I am capturing all?
What is the intended use here? Shared memory has its place but you really have to be doing extremely high speed communications to make it worthwhile. You can on a typical current system do well over a million inverted calls a second if you push it. So how many hundreds of megabytes per second do you think you are going to be doing? Don Burn (MVP, Windows DKD) Windows Filesystem and Driver Consulting Website: http://www.windrvr.com Blog: http://msmvps.com/blogs/WinDrvr “xxxxx@hotmail.com” wrote in message news:xxxxx@ntdev: > Hi, > > I have been looking and most of the articles have found have been for WDM, how may I go about memory sharing with a user application in WDK? I am looking at actually using memory rather than a virtual file which the mirror driver uses. I’m open to suggestions of a better method too, I looked into inverted calls but I am worried because the driver may give a lot of small pieces of information that several inverted calls may not be enough.
You should provide the a number of requests so they can be filled. On
the question of losing one, you can either provide a way to queue data
in the driver if no request is available to fill, or else have a flag to
indicate you did loose data then tune your service to have more
requests.
Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
“xxxxx@hotmail.com” wrote in
message news:xxxxx@ntdev:
> I won’t be doing that much. The code has already been writing, it is just adding extra code to record the movements of a mouse using the callback already developed. If a invert call can handle that then that is the method I will look at. How can I be sure that I am capturing all the data and not one is missed without waiting inside the driver for the service to issue the next inverted call. How many do you think I will need to be sure I am capturing all?
>
> —
>
> What is the intended use here? Shared memory has its place but you really have to be doing extremely high speed communications to make it worthwhile. You can on a typical current system do well over a million inverted calls a second if you push it. So how many hundreds of megabytes per second do you think you are going to be doing? Don Burn (MVP, Windows DKD) Windows Filesystem and Driver Consulting Website: http://www.windrvr.com Blog: http://msmvps.com/blogs/WinDrvr “xxxxx@hotmail.com” wrote in message news:xxxxx@ntdev: > Hi, > > I have been looking and most of the articles have found have been for WDM, how may I go about memory sharing with a user application in WDK? I am looking at actually using memory rather than a virtual file which the mirror driver uses. I’m open to suggestions of a better method too, I looked into inverted calls but I am worried because the driver may give a lot of small pieces of information that several inverted calls may not be enough.
That is OK. So: Please forgive me if I am and correct me.
The UMDF sends an IO request and waits for it to be completed.
The driver picks this up in EvtIoInCallerContext(); and puts the request into a wait mode.
EvtServiceCallback is called when new data is to be sent.
(How do I associate this function to filling and completing the request?)
The driver returns the data to the service.
You should provide the a number of requests so they can be filled. On the question of losing one, you can either provide a way to queue data in the driver if no request is available to fill, or else have a flag to indicate you did loose data then tune your service to have more requests. Don Burn (MVP, Windows DKD) Windows Filesystem and Driver Consulting Website: http://www.windrvr.com Blog: http://msmvps.com/blogs/WinDrvr “xxxxx@hotmail.com” wrote in message news:xxxxx@ntdev: > I won’t be doing that much. The code has already been writing, it is just adding extra code to record the movements of a mouse using the callback already developed. If a invert call can handle that then that is the method I will look at. How can I be sure that I am capturing all the data and not one is missed without waiting inside the driver for the service to issue the next inverted call. How many do you think I will need to be sure I am capturing all? > > — > > What is the intended use here? Shared memory has its place but you really have to be doing extremely high speed communications to make it worthwhile. You can on a typical current system do well over a million inverted calls a second if you push it. So how many hundreds of megabytes per second do you think you are going to be doing? Don Burn (MVP, Windows DKD) Windows Filesystem and Driver Consulting Website: http://www.windrvr.com Blog: http://msmvps.com/blogs/WinDrvr “xxxxx@hotmail.com” wrote in message news:xxxxx@ntdev: > Hi, > > I have been looking and most of the articles have found have been for WDM, how may I go about memory sharing with a user application in WDK? I am looking at actually using memory rather than a virtual file which the mirror driver uses. I’m open to suggestions of a better method too, I looked into inverted calls but I am worried because the driver may give a lot of small pieces of information that several inverted calls may not be enough.
You will likely queue the data up; think about how you’re going to be getting the data … it’s spooling in at a rate X, at that point you either read it at rate Y or queue it. If you’re doing this from a shared memory segment then the “reader” thread is at PASV, subject to being swapped out as soon as a higher priority thread/ APC/ DPC/ ISR comes along … and you’ll stay swapped out for micro or milliseconds, tenths of seconds or days depending on how the higher priority thread was written. Meanwhile, your data continues to spool in at rate X …
If you queue it, then where would you rather gather the data … at PASV, where your thread will be go away everytime SilverLight decides to run (thanks, MS, for dynamically increasing thread priority *just because*!) or similar, or at SYSTEM, where only APC’s, DPC’s and ISR’s can trump you …
Also, how do you “know” when something new has come in? Well, you could poll a location/ pointer, but see above – your polling will get swapped out everytime the multimedia player is run. It’s also very inefficient – you’re tying up a core just reading a location endlessly, mindlessly. Wouldn’t it make more sense to get somehow notified that some new data is available?
So, here’s what I would suggest: queue up your data in one of N buffers of M size using a thread, DPC, etc. and have this data accumulate. In usermode create a thread who simply posts an IOCTL (overlapped) and passes in a buffer of M size … and waits. When the thread gathers the data you want it copies the N’th buffer data into the IOCTL buffer, completes it and starts writing data to the next buffer … and repeat …
Using this approach you a) don’t ever lose data, as it’s being cached in a nice, safe driver buffer by something operating above the level of Silverlight, b) are notified when something has come in and can pull the data when your usermode app is ready to work with the data and c) have a much lower impact on overall performance than something polling … that, as many others have stated, is the essence of the inverted call …
You can get much, much fancier of course: hi/lo watermarks on the buffers for completion, completion ports on the user thread for even better performance, backpressure through the IOCTL, etc. etc. but that’s the jist of it. As Don has mentioned there *are* some cases where shared memory segments are the right solution but the design you mention is almost a perfect fit for an inverted call approach …
Cheers!
that is not the way to do this. you do not need an in caller context callback. just put use normal WDFQUEUE processing. when you get the io request from your client, if there is no data, forward it to a manual queue, otherwise complete the request with data you stored away. when you have data from the service , try to remove a request from the manual queue. if successful, complete the request with the data, otherwise store away the data.
the last thing you want to do is infinitely synchronously block the sending thread that is asking for the information by using KeWaitForSingleObject or any other dispatcher object.
d
From: xxxxx@lists.osr.com [xxxxx@lists.osr.com] on behalf of xxxxx@hotmail.com [xxxxx@hotmail.com]
Sent: Thursday, March 24, 2011 8:01 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Shared Memory
That is OK. So: Please forgive me if I am and correct me.
The UMDF sends an IO request and waits for it to be completed.
The driver picks this up in EvtIoInCallerContext(); and puts the request into a wait mode.
EvtServiceCallback is called when new data is to be sent.
(How do I associate this function to filling and completing the request?)
The driver returns the data to the service.
You should provide the a number of requests so they can be filled. On the question of losing one, you can either provide a way to queue data in the driver if no request is available to fill, or else have a flag to indicate you did loose data then tune your service to have more requests. Don Burn (MVP, Windows DKD) Windows Filesystem and Driver Consulting Website: http://www.windrvr.com Blog: http://msmvps.com/blogs/WinDrvr “xxxxx@hotmail.com” wrote in message news:xxxxx@ntdev: > I won’t be doing that much. The code has already been writing, it is just adding extra code to record the movements of a mouse using the callback already developed. If a invert call can handle that then that is the method I will look at. How can I be sure that I am capturing all the data and not one is missed without waiting inside the driver for the service to issue the next inverted call. How many do you think I will need to be sure I am capturing all? > > — > > What is the intended use here? Shared memory has its place but you really have to be doing extremely high speed communications to make it worthwhile. You can on a typical current system do well over a million inverted calls a second if you push it. So how many hundreds of megabytes per second do you think you are going to be doing? Don Burn (MVP, Windows DKD) Windows Filesystem and Driver Consulting Website: http://www.windrvr.com Blog: http://msmvps.com/blogs/WinDrvr “xxxxx@hotmail.com” wrote in message news:xxxxx@ntdev: > Hi, > > I have been looking and most of the articles have found have been for WDM, how may I go about memory sharing with a user application in WDK? I am looking at actually using memory rather than a virtual file which the mirror driver uses. I’m open to suggestions of a better method too, I looked into inverted calls but I am worried because the driver may give a lot of small pieces of information that several inverted calls may not be enough.
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
I think I understand it I have worked with IOCTL before but the only piece I don’t fully understand is the waiting and how to complete a request in another thread - the ServiceCallback thread - when the IOCTL that came in when the buffer was empty without using a loop, whether in UMDF or - forbiddingly - KMDF.
How do you do it Doron, your answers are always perfect. Thankyou very much that’s exactly what I needed to know.
xxxxx@hotmail.com wrote:
How do you do it Doron, your answers are always perfect.
Because he works for Microsoft, and wrote a bunch of the code in KMDF.
He is an incredible resource, and we are fortunate that he participates
so much in this forum.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
And there are now new problems ;)…or at least, the same typical problems repeatedly show up in the same device classes 
d
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Tim Roberts
Sent: Thursday, March 24, 2011 9:38 AM
To: Windows System Software Devs Interest List
Subject: Re: [ntdev] Shared Memory
xxxxx@hotmail.com wrote:
How do you do it Doron, your answers are always perfect.
Because he works for Microsoft, and wrote a bunch of the code in KMDF.
He is an incredible resource, and we are fortunate that he participates so much in this forum.
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Doron Holan wrote:
And there are now new problems ;)…or at least, the same typical problems repeatedly show up in the same device classes
Yes. I once chatted with a woman who had been an emergency room
supervising nurse for 25 years. I commented on how stressful it must be
to have to handle so many different situations under that much
pressure. She said pretty much the same thing you just said: “After
you’ve been doing this job for a year or two, you have seen pretty much
everything you are ever going to see. You just have to follow the
recipes that work.”
–
Tim Roberts, xxxxx@probo.com
Providenza & Boekelheide, Inc.
I’m a constant reader of the lists. And yes, I really appreciate every post from Doron too! Beside the fact that my son also is named “Doron”! Also the other participates on this list make it a very valueable resouce!
Thank you!
Another technique, which I have applied when I had no control of the
lower-level driver, was to simply use asynchronous I/O, and pump down 50
ReadFile requests, all of which were held pending by the driver. That
IoStartNextPacket call or its equivalent is blindingly fast, and meant that
a pending buffer was dequeued immediately and could start accepting data. I
set the value of the number of preloads on a spin control; at 40 we were
having data overruns, and at 50 we ran for a day without a packet loss.
Sometimes there is no need to overdesign a driver to solve a problem that
can be solved by other means.
joe
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn
Sent: Thursday, March 24, 2011 9:19 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] Shared Memory
You should provide the a number of requests so they can be filled. On
the question of losing one, you can either provide a way to queue data
in the driver if no request is available to fill, or else have a flag to
indicate you did loose data then tune your service to have more
requests.
Don Burn (MVP, Windows DKD)
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
“xxxxx@hotmail.com” wrote in
message news:xxxxx@ntdev:
> I won’t be doing that much. The code has already been writing, it is just
adding extra code to record the movements of a mouse using the callback
already developed. If a invert call can handle that then that is the method
I will look at. How can I be sure that I am capturing all the data and not
one is missed without waiting inside the driver for the service to issue the
next inverted call. How many do you think I will need to be sure I am
capturing all?
>
> —
>
> What is the intended use here? Shared memory has its place but you really
have to be doing extremely high speed communications to make it worthwhile.
You can on a typical current system do well over a million inverted calls a
second if you push it. So how many hundreds of megabytes per second do you
think you are going to be doing? Don Burn (MVP, Windows DKD) Windows
Filesystem and Driver Consulting Website: http://www.windrvr.com Blog:
http://msmvps.com/blogs/WinDrvr “xxxxx@hotmail.com”
wrote in message news:xxxxx@ntdev: > Hi, > > I have been looking and most
of the articles have found have been for WDM, how may I go about memory
sharing with a user application in WDK? I am looking at actually using
memory rather than a virtual file which the mirror driver uses. I’m open to
suggestions of a better method too, I looked into inverted calls but I am
worried because the driver may give a lot of small pieces of information
that several inverted calls may n!
ot be enough.
—
NTDEV is sponsored by OSR
For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer
–
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.