generic questions about communicaiton between kernel mode and user mode

Hi,

I have generic (I suppose) question about communication possibilities between kernel mode and user mode.
Note! I’m asking about communication in BOTH directions: so usermode sends something to kernelmode and kernelmode wants to send something to usermode.

  1. Q: What possibilities we have at all? KM to UM - there is possibility to send IOCTLs (with different methods) - but what in case kernelmode want to send infomraiton to usermode? How to solve such case?
    Also if there is OTHER possibility (usermode->kernelmode) then IOCTLs than what kind?

  2. Q: To all above proposals (which you hopefully will advice here ) can you please attach some examples of such communication?

  3. Q: What kind of solution (kernel->user AND user->kernel) is lightest one?

  4. Q: What kind of solution is recommended by microsoft to solve such problems?

thank you for all suggestions/advices

For KM to UM use inverted calls if at all possible. Take a look at the
latest NT Insider on the OSRonline site to understand the model. There are
other approaches for communication, such as events, but these have a huge
amount of complexity for the performance gain (minimal) that they get.
Note, I have successfully pushed IOCTL’s to over 1 million / second on a 2
core system. So take a look at inverted calls.

Don Burn
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@gmail.com
Sent: Sunday, December 01, 2013 8:27 AM
To: Windows System Software Devs Interest List
Subject: [ntdev] generic questions about communicaiton between kernel mode
and user mode

Hi,

I have generic (I suppose) question about communication possibilities
between kernel mode and user mode.
Note! I’m asking about communication in BOTH directions: so usermode sends
something to kernelmode and kernelmode wants to send something to usermode.

  1. Q: What possibilities we have at all? KM to UM - there is possibility to
    send IOCTLs (with different methods) - but what in case kernelmode want to
    send infomraiton to usermode? How to solve such case?
    Also if there is OTHER possibility (usermode->kernelmode) then IOCTLs than
    what kind?

  2. Q: To all above proposals (which you hopefully will advice here ) can you
    please attach some examples of such communication?

  3. Q: What kind of solution (kernel->user AND user->kernel) is lightest one?

  4. Q: What kind of solution is recommended by microsoft to solve such
    problems?

thank you for all suggestions/advices


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Generally, if you think that your KM component needs to ‘communicate’ at all
you need to rethink your design. Generally, KM components respond to
requests from UM programs or services that take requests from local or
remote users. These requests usually come in the form of IOCTLs (calls to
DeeviceIoControl, ReadFile or WriteFile) and if your can possibly organize
your design to fall into this pattern your life will be much better. A
technique called inverted call exists to help recapitulate designs that
might need the concept of communicating from KM to UM. Note that if you
have KM code that might communicate with processes on remote machines, then
sockets are usually a better choice than IOCTLs and in that case the idea of
bidirectional communication becomes valid again.

wrote in message news:xxxxx@ntdev…

Hi,

I have generic (I suppose) question about communication possibilities
between kernel mode and user mode.
Note! I’m asking about communication in BOTH directions: so usermode sends
something to kernelmode and kernelmode wants to send something to usermode.

  1. Q: What possibilities we have at all? KM to UM - there is possibility to
    send IOCTLs (with different methods) - but what in case kernelmode want to
    send infomraiton to usermode? How to solve such case?
    Also if there is OTHER possibility (usermode->kernelmode) then IOCTLs than
    what kind?

  2. Q: To all above proposals (which you hopefully will advice here ) can you
    please attach some examples of such communication?

  3. Q: What kind of solution (kernel->user AND user->kernel) is lightest one?

  4. Q: What kind of solution is recommended by microsoft to solve such
    problems?

thank you for all suggestions/advices

>methods) - but what in case kernelmode want to send infomraiton to usermode?

The best way is the “inverted call”:

  • umode sends overlapped DeviceIoControl
  • kmode pends it to some queue for a long time till it will have a need to send some data to umode
  • when kmode wants to send this data, then it fills the data buffer of this IOCTL IRP and completes it
  • umode sees the usual overlapped completion (event-based or IOCP-based)

You can even use synchronous, not overlapped, IO in umode, it will just sacrifice the whole thread to the sole purpose of servicing such calls.

This is the classic way of K->U data transfers in both Windows and UNIXen (like the TUN and TAP network adapters in the latter, as also the old FreeBSD umode “natd” NAT).

You can wrap the inv. calls around to some library, which will create some kind of “communication ports” between user and kernel.

More so. If you are about FS filters, then FltMgr subsystem already has such ports implemented :slight_smile:


Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

On 01-Dec-2013 15:27, xxxxx@gmail.com wrote:

Hi,

I have generic (I suppose) question about communication possibilities between kernel mode and user mode

Suggestion: To answer such generic and frequently asked question
comprehensively, our host OSR offers seminars. Or look for similar
courses closer to your location.
– pa

> On 01-Dec-2013 15:27, xxxxx@gmail.com wrote:

> Hi,
>
> I have generic (I suppose) question about communication possibilities
> between kernel mode and user mode

Suggestion: To answer such generic and frequently asked question
comprehensively, our host OSR offers seminars. Or look for similar
courses closer to your location.
– pa

Not everyone has the ability to travel or the financial means to take
these courses, as good as they are said to be. Therefore, I think it is
perfectly fair to ask them here. The important piece of information the
OP should get from this is “Use inverted call” and thus eliminate tons of
pointless messages about how to share event handles, kernel memory, and
all those other horrors we are regularly subjected to. If a simple
generic question can be answered, it could avoid the horrors of those who
want Windows to work like their last RTOS (which had no user/kernel
distinction, for example)
joe


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

>Generally, if you think that your KM component needs to ‘communicate’ at all you need to rethink your design.

As most other generic sweeping statements that allow no compromise, the above one is simply wrong. More on it below…

Generally, KM components respond to requests from UM programs or services that take requests
from local or remote users.

This is 100% true for block devices, because they never generate data on their own initiative. However, when it comes to character and network devices, particularly the latter ones, the above statement may
be false. In the latter case, a driver/protocol/subsystem/etc may want to apply certain policies to incoming data(like, for example, packet filtering or routing decisions), and the userland happens to be the best place to
actually implement these policy decisions…

Anton Bassov

>

> On 01-Dec-2013 15:27, xxxxx@gmail.com wrote:
>> Hi,
>>
>> I have generic (I suppose) question about communication possibilities
>> between kernel mode and user mode
>
> Suggestion: To answer such generic and frequently asked question
> comprehensively, our host OSR offers seminars. Or look for similar
> courses closer to your location.
> – pa
>
>
Not everyone has the ability to travel or the financial means to take
these courses, as good as they are said to be. Therefore, I think it is
perfectly fair to ask them here. The important piece of information the
OP should get from this is “Use inverted call” and thus eliminate tons of
pointless messages about how to share event handles, kernel memory, and
all those other horrors we are regularly subjected to. If a simple
generic question can be answered, it could avoid the horrors of those who
want Windows to work like their last RTOS (which had no user/kernel
distinction, for example)

Once upon a time I reached the mistaken conclusion that an IOCTL for an inverted call had lots of system overhead and would be slow. Not sure why, but I’ve seen others reach that conclusion too.

James

>

>Generally, if you think that your KM component needs to ‘communicate’ at
> all you need to rethink your design.

As most other generic sweeping statements that allow no compromise, the
above one is simply wrong. More on it below…

> Generally, KM components respond to requests from UM programs or
> services that take requests
> from local or remote users.

This is 100% true for block devices, because they never generate data on
their own initiative. However, when it comes to character and network
devices, particularly the latter ones, the above statement may
be false. In the latter case, a driver/protocol/subsystem/etc may want to
apply certain policies to incoming data(like, for example, packet filtering or
routing decisions), and the userland happens to be the best place to
actually implement these policy decisions…

This is still UM driving the communication though isn’t it? UM says to KM “Complete my request when you have data”.

James

>>Generally, if you think that your KM component needs to ‘communicate’ at

> all you need to rethink your design.

As most other generic sweeping statements that allow no compromise, the
above one is simply wrong. More on it below…

> Generally, KM components respond to requests from UM programs or
> services that take requests
> from local or remote users.

This is 100% true for block devices, because they never generate data on
their own initiative. However, when it comes to character and network
devices, particularly the latter ones, the above statement may
be false. In the latter case, a driver/protocol/subsystem/etc may want to
apply certain policies to incoming data(like, for example, packet
filtering or routing decisions), and the userland happens to be the best
place to
actually implement these policy decisions…

Which does not preclude the use of the supported filter messages or the
use of inverted call as being vastly preferable to the gross hacks
required to try to emulate other models of communication.

So the sweeping statement should be more of the form “If you think your KM
component must initiate a data transfer to a process, rather than have the
process request data from your KM component…”

I like sweeping generalizations for those who are new to any form of
programming. It keeps them out of the quicksand pits, away from the
hungry lions, and avoids messy consequences, especially for those
quicksand bogs that have landmines sunk to the bottom. Anyone who asks a
question of the form asked is a beginner looking for good advice. And the
good advice is: “If you think that you need to have the kernel initiate a
data transfer to user space, or you feel a compulsion to share kernel
memory with a user app, or you think you want to share an event handle
with user space, be aware that those who are experts in this domain tend
to not only avoid such designs, but AGGRESSIVELY avoid such designs, and
these is a message here: don’t take this approach. The reasons the
experts avoid it are many, and sound, and part of it is that they know
exactly how many things can go wrong with it. Stick with simple
approaches, and you will find that it takes a fraction of the code, a tiny
fraction of the development time, and produces a solution vastly more
robust, that the one you are thinkng of.” But that’s too long. So the
simpler “Don’t do that” is used in its place.

Anyone who harbors the delusion that such complex designs are necessary
“for performance” should be referred to Don Burn’s observation of being
able to do a million IOCTLs per second. So “performance” is not an issue
to be overly concerned with.

Note that non-block-oriented devices will do internal buffering of data,
but only deliver it when requested. In the event of insufficient requests
per unit time, they will eventually run out of whatever buffering they
have chosen to allow themselves, and in any case, bad app design is the
usual cause. Synchronous I/O for non-block devices is generally an
insanely bad idea. If you need performance, FILE_FLAG_OVERLAPPED is your
New Best Friend.
joe

Anton Bassov


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

>>

> > On 01-Dec-2013 15:27, xxxxx@gmail.com wrote:
> >> Hi,
> >>
> >> I have generic (I suppose) question about communication possibilities
> >> between kernel mode and user mode
> >
> > Suggestion: To answer such generic and frequently asked question
> > comprehensively, our host OSR offers seminars. Or look for similar
> > courses closer to your location.
> > – pa
> >
> >
> Not everyone has the ability to travel or the financial means to take
> these courses, as good as they are said to be. Therefore, I think it is
> perfectly fair to ask them here. The important piece of information the
> OP should get from this is “Use inverted call” and thus eliminate tons
> of
> pointless messages about how to share event handles, kernel memory, and
> all those other horrors we are regularly subjected to. If a simple
> generic question can be answered, it could avoid the horrors of those
> who
> want Windows to work like their last RTOS (which had no user/kernel
> distinction, for example)

Once upon a time I reached the mistaken conclusion that an IOCTL for an
inverted call had lots of system overhead and would be slow. Not sure why,
but I’ve seen others reach that conclusion too.

It’s a common Urban Legend that Will Not Die.
joe

James


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

>>

> >Generally, if you think that your KM component needs to ‘communicate’
> at
> > all you need to rethink your design.
>
> As most other generic sweeping statements that allow no compromise, the
> above one is simply wrong. More on it below…
>
> > Generally, KM components respond to requests from UM programs or
> > services that take requests
> > from local or remote users.
>
> This is 100% true for block devices, because they never generate data
> on
> their own initiative. However, when it comes to character and network
> devices, particularly the latter ones, the above statement may
> be false. In the latter case, a driver/protocol/subsystem/etc may want
> to
> apply certain policies to incoming data(like, for example, packet
> filtering or
> routing decisions), and the userland happens to be the best place to
> actually implement these policy decisions…
>

This is still UM driving the communication though isn’t it? UM says to KM
“Complete my request when you have data”.

That’s right. That’s why it’s such a good idea. Doing it the other way
is unnatural, and results in gross hacks, time-wasting questions on this
NG which get the OP past the current bug, only to find, a day or week
later, another set of questions that should never have had to be answered
detailing problems which never needed to be solved, except for the initial
bad design decisions. And, sadly, their gross hack is rarely faster than
the simpler inverted call or IOCP model, it just takes much longer to get
right. And has a tendency to not reveal all the problems until
post-deployment, which is not the right time to have your customers
discover them. Create simple solutions unless you can PROVE, with SOLID
DATA, not Urban Legends, that the simpler approach cannot work.
joe

James


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

> So the sweeping statement should be more of the form "If you think your KM component must

initiate a data transfer to a process, rather than have the process request data from your KM component…"

S tick with simple approaches, and you will find that it takes a fraction of the code, a tiny fraction
of the development time, and produces a solution vastly more robust, that the one you are thinkng of."

Am I the only one who notices a huge contradiction between these two statements?

Ironically, in most cases simplicity happens to be exactly the reason why the KM code may want to transfer data to the userland for processing. In order to avoid doing complex things in the kernel you delegate the job to the userland, and solve the whole thing right on the spot by UM API that had been specifically designed for this purpose, rather than rolling your own implementation in the kernel. This is what I am talking about when I criticize “sweeping statements”. It obviously has nothing to do with the “efficiency” - it is (hopefully) obvious that this split KM-UM solution may be slightly less efficient, compared to 100% KM one…

Anton Bassov

>> So the sweeping statement should be more of the form "If you think your

> KM component must
> initiate a data transfer to a process, rather than have the process
> request data from your KM component…"

>S tick with simple approaches, and you will find that it takes a fraction
> of the code, a tiny fraction
>of the development time, and produces a solution vastly more robust, that
> the one you are thinkng of."

Am I the only one who notices a huge contradiction between these two
statements?

Ironically, in most cases simplicity happens to be exactly the reason
why the KM code may want to transfer data to the userland for processing.
In order to avoid doing complex things in the kernel you delegate the job
to the userland, and solve the whole thing right on the spot by UM API
that had been specifically designed for this purpose, rather than rolling
your own implementation in the kernel. This is what I am talking about
when I criticize “sweeping statements”. It obviously has nothing to do
with the “efficiency” - it is (hopefully) obvious that this split KM-UM
solution may be slightly less efficient, compared to 100% KM one…

Anton Bassov

You are confusing two barely-related concepts here. One is the simplicity
of doing filtering at the user level, and the other is the simplicity of
the implementation that makes that possible. If you want to use a
process, perhaps a system service, to receive data from the kernel, make
decisions based on it, and return it to the kernel, in order to avoid
doing complex things in the kernel, then it makes no sense whatsoever to
build an unnecessarily-complex interface to accomplish this simple task.
So I see no contradiction at all. If the goal is to reduce costs by
building the complex part in user space, then wasting weeks or months
learning the arcane mysteries of how to pass kernel buffers to user space,
or share events with user space, is foolishness. It accomplishes nothing
that could not be achieved more quickly by a very simple inverted call
interface, or by using the supported filter-driver messaging interface.
The only justification for making such a bizarre set of decisions is
simple ignorance. And, presumably, that’s why the original question.
Ignorance is curable. Stupidity is not. So it is important that we point
such people down the right path, before we get flooded with the “I got a
BSOD…” and it relates to some completely inappropriate sequence of code
whose purported purpose is to implement some bizarrely-complex solution to
a trivial problem. If you want to keep the solution simple, and one
component of that simplification is to put processing in user space, then
it is important to keep the communication with user space simple. Instead
of trying to replicate some historically-boring RTOS of the programmer’s
experience, the correct solution is to implement a solution that works
WITH, not AGAINST, the basic paradigms Windows supports. Sharing kernel
space with user space, sharing events between KM and UM, do NOT qualify as
solutions that are working WITH the design of the system. Since, as
several people have pointed out, there is rarely, if ever, a performance
gain, and even if so, it is countered by a loss in investment (it takes
much longer to implement the bizarre solution), robustness (it is very
likely to fail post-deployment as all the corner cases the implementor
never thought of come up in real environments), credibility of the product
line (remember Diamond graphics card? Yeah, two or three driver releases
PER DAY as they fought to fix all the bugs they deliberately introduced in
order to get performance, as in, “We don’t need no stinkin’ bounds checks;
the app will never give us data to write outside the graphical area”),
severe frustration to the programmer (who has to make the bizarre design
work, which involves knowing more arcana than the creators at Microsoft
even know), market share loss (missing crucial time-to-market windows),
etc. then There Is Something Wrong With This Picture. Write a simple
driver that works the way drivers are supposed to work, have it up and
running in a few weeks, not in a few quarters or even a few years. It is
straightforward, it comes up, it runs, and it can be maintained by
unskilled labor (the new hire, or its author a year later) because nobody
has to re-understand the arcana in order to figure out why it is getting a
BSOD on one customer’s machine(s) (and, I might add, under Murphy’s Law,
that is your single most important customer).

You have obviously misunderstood the difference between code factoring and
interface implementation.
joe


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

> If you want to use a process, perhaps a system service, to receive data from the kernel,

make decisions based on it, and return it to the kernel, in order to avoid doing complex things
in the kernel, then it makes no sense whatsoever to build an unnecessarily-complex interface
to accomplish this simple task.

You have obviously misunderstood the difference between code factoring and interface implementation.

Please read that statement that I am arguing about

This is what I am arguing about - there is no question of interface anywhere is sight. Sorry, but you are the only one who speaks about interfaces…

If the goal is to reduce costs by building the complex part in user space, then wasting weeks
or months learning the arcane mysteries of how to pass kernel buffers to user space,
or share events with user space, is foolishness.

In fact, I did not even suggest that one should use events…

In any case, there is nothing “mysterious” about events and buffers - IIRC, there is even WDK sample that demonstrates how to do exactly the above. Although there are some issues with this approach (mainly security-related ones), complexity of interface is NOT among them…

Sharing kernel space with user space, sharing events between KM and UM, do NOT qualify
as solutions that are working WITH the design of the system.

And this is wrong as well - if it was working the way you describe there would be no API that allows that.
Although this is not necessarily the best option available (and in most cases it is not), there is absolutely nothing in it that bends Windows rules and turns them inside out and upside down…

BTW, DR Joe, it it were up tome, I would nominate you for the “Master of FUD” title…

Anton Bassov

>> Generally, KM components respond to requests from UM programs or services that take requests

> from local or remote users.

This is 100% true for block devices, because they never generate data on their own initiative.

I would add: the design with the kmode service is only a “reflector” which represents the device in the OS, while the actual implementation of the device is done in user mode, is frequent, and used for block devices too.


Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

Propaganda and hyperbole at work here, eh?

At OSR we teach literally hundreds of newbies every year.

I have never found, in my long career of teaching students, mentoring team members, and explaining project designs to less-technical staffers that engineering truth should be an accepted casualty in the cause of simplicity or comprehension.

I don’t believe in teaching people dogma, or arbitrary rules to follow, without accompanying understanding.

While there is no doubt that inverted call is the preferred approach in most cases, it does not follow that other approaches are not supported by Windows. And if there are experienced kernel engineers who AGGRESSIVELY avoid sharing memory or event handles, I would suggest they return to first principles and reconsider the reasons for their beliefs.

Choosing an engineering design is about suitability for a given purpose, not absolute dictates or verbal tantrums. I’m doing an interface design right now that makes substantial use of shared events. Does that mean I’m stupid or that I don’t understand my Windows architecture?

Peter
OSR

> While there is no doubt that inverted call is the preferred approach in most cases, it does not follow

that other approaches are not supported by Windows.

Exactly so.

For instance, Windows-provided audio stuff uses shared memory (user-mode-allocated) approach, with a “set ring buffer head/tail” call from user to kernel. But this is a bit of a special case with DRM involved.


Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com

>Propaganda and hyperbole at work here, eh?

I would go a bit further than that and label it as FUD, i.e. false info that is based upon some true facts that, however, got misinterpreted,distorted, taken out of context and blown out of proportion by FUD originator
in order to suit his/her agenda…

Anton Bassov

Before we scatter the ashes of burning Dr. Newcomer in effigy, I would like
to say that in general the principal of not first jumping into things like
shared memory or events between user mode and kernel should be followed.
Way too often, I’ve encountered drivers with problems that use these
techniques, most of the time if I am asked to fix things, I rip them out for
the inverted call with no loss of performance (and in some case performance
improvements)!

With the environment we have today with things like WDF, the approach to a
driver you are concerned about performance on should be:

  1. Make the driver work with a clean interface model
  2. Measure the performance to see if there is a need to speed things up
  3. Profile the system to see where to make things faster
  4. Only then do you consider the less than usual interface models, and even
    then after seeing what else can be done.

Unfortunately, there is a myth that I hear way too often that Windows does
not have good performance versus another OS out there. So many developers
coming from the other environment start planning complex schemes to solve
problems that don’t exist.

Dr, Newcomer’s statements may have been too far, but the basic idea that a
new developer should initially avoid the more challenging interfaces until
proven they need them is sound.

Don Burn
Windows Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr