RE: {Disarmed} RE: Query regarding shared memory

NTFSD
1

Are you sure something this dangerous is really necessary? For one thing,
you’ll probably never get certification for the driver (I’ve been assured
that Microsoft really, really disapproves of this technique).

How do you plan to synchronize access to this shared memory? Note that it
probably involves a kernel call (whoops, slow! Or probably no faster than,
say, DeviceIoControl, ReadFile or WriteFile). How, exactly, do you plan to
“signal” the application that data is ready? IoCompleteRequest works very
well for this. SetEvent less so. And you are still at the mercy of the
scheduler (it is amazing how many programmers think that doing SetEvent
means “the thread will run immediately” when what SetEvent means is “this
thread is now feasible for scheduling”)

Actually, *why* do you say that IOCTL is “slow”? Are you accounting for the
scheduler, for example? What is going to happen when your user thread is
not running because it has either been preempted or has lost its timeslice?
Are you talking about actual kernel overhead, or scheduler delays, or can
you tell? Note that you may be creating a solution that in fact does not
solve the problem you are seeing, just gives you a warm fuzzy feeling that
it looks like some solution you once used in some other context in some
other operating system.

You are asking how to implement a solution without stating the problem you
are trying to solve. There is a perfectly fine bidirectional communication
mechanism between the kernel space and the user space: the IRP. Nice thing
is, you don’t have to add additional synchronization mechanisms to make it
work!

Or is this another example of an embedded realtime programmer trying to make
Windows work like an embedded OS? Every time I get this question in class,
it’s from someone whose background involves an OS that has either “RT”
somewhere in its acronymic name or has an acronymic name that ends in “OS”.
It usually runs without either mapped memory or the concept of user space.
joe

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Raghwendra Kshatriya
Sent: Thursday, December 18, 2008 12:28 AM
To: Windows File Systems Devs Interest List
Subject: {Disarmed} RE: [ntfsd] Query regarding shared memory

Here the objective of doing so is faster data communication between driver
(KERNEL MODE) and win32 application (USER MODE). Currently this we are doing
using IOCTL which is slow and one sided from Win32 —> to—> Driver. Using
share memory I want to update that memory in driver and simply wants to
signal win32 application to read that data.

Raghwendra

From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Skywing
Sent: Wednesday, December 17, 2008 10:06 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Query regarding shared memory

Why do you believe that this is necessary? It is almost always not, and it
is extremely easy to introduce subtle security and stability bugs that may
not show up right away with conventional (driver verifier) testing.

  • S

From: Suganya Nataraj
Sent: Wednesday, December 17, 2008 03:27
To: Windows File Systems Devs Interest List
Subject: [ntfsd] Query regarding shared memory

Hi Tony,
I need to create a shared memory which will be utilized by
both win32 application and kernal level.
Any changes or inputs given at oneside(either win32 or
kernel level) through shared memory, i should be able to get it at other
side.

So how could I create shared memory and how could I
communicate between the two levels…
http:</http:>
Regards,
http:</http:> Suganya Nataraj | E-Security|Tech Mahindra
Chandivali, Andheri (E), Mumbai-72.
http:</http:> Landline :+91 22 66882000 | Extn : 8092
http:</http:> Mobile: +91 9930244229
http:</http:> MailScanner has detected a possible fraud
attempt from “www.funonthenet.in” claiming to be Email:
xxxxx@techmahindra.com
http:</http:> MailScanner has detected a possible fraud
attempt from “www.funonthenet.in” claiming to be www.techmahindra.com

—
NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

============================================================================
================================================

Disclaimer:

This message and the information contained herein is proprietary and
confidential and subject to the Tech Mahindra policy statement, you may
review the policy at http://www.techmahindra.com/Disclaimer.html externally
and http://tim.techmahindra.com/Disclaimer.html internally within Tech
Mahindra.

============================================================================
================================================

—
NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

—
NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com
–
This message has been scanned for viruses and
dangerous content by http:</http:> MailScanner, and is
believed to be clean.
============================================================================
================================================

Disclaimer:

This message and the information contained herein is proprietary and
confidential and subject to the Tech Mahindra policy statement, you may
review the policy at http://www.techmahindra.com/Disclaimer.html externally
and http://tim.techmahindra.com/Disclaimer.html internally within Tech
Mahindra.

============================================================================
================================================

2

> I need to create a shared memory which will be utilized by both win32 application and kernal

level.

Usually, this is amazingly bad design, which usually cannot be justified.

Instead, send IOCTLs to the driver to send requests to it, and use inverted call IOCTLs to consume the data initiated by the driver.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

3

>win32 application (USER MODE). Currently this we are doing using IOCTL which is slow

Why do you think it’s slow? have you measured?

and one sided from Win32 -? to-? Driver. Using share memory I want to update that memory in driver
and simply wants to signal win32 application to read that data.

  1. Signalling will be slower then IOCTL, or around the same time.
  2. You cannot pass any data item with signalling if you do not use inverted call, which is IOCTL.
  3. Do not forget guarding the shared memory with locks. Without the locks, it is nearly never useful, and the locks will increase the overhead.

Once more: you already have everything working OK in a preferred method. Stop remaking it, unless this is a real (measured) performance hog.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

4

Maxim is right, IOCTL with direct I/O for ex would be the best choice here.
Shared memory has views, which will be mapped process dependent, and you will need to use the system wide dispatcher to handle concurrency to the memory most likely which will involve serious overhead.
Design your application so it is IOCTL request dependent, and you can get on moving with you real work instead of trying to achieve performance with VM.

which is slow and one sided from Win32 - to- Driver

By far that is not an issue, first of all you can make workarounds to avoid needing two ways communication, and secondly it will ease the overhead in your implementation this way. Think as the driver as a server serving requests to a client (your win32 app) Would you really like to make your win32 app be a server as well serving info for the driver ? I don’t think this is where you need to go.
I really suggest redesigning, by adding more feasibility to your control codes` internal design. Choose a good input data, and output data as well. Define as many flags as you need, as many enum states as you need, as many own status code as you need. This will help you not needing a two way communication.

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Friday, December 19, 2008 2:51 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Query regarding shared memory

win32 application (USER MODE). Currently this we are doing using IOCTL which is slow

Why do you think it’s slow? have you measured?

and one sided from Win32 -à to-à Driver. Using share memory I want to update that memory in driver
and simply wants to signal win32 application to read that data.

  1. Signalling will be slower then IOCTL, or around the same time.
  2. You cannot pass any data item with signalling if you do not use inverted call, which is IOCTL.
  3. Do not forget guarding the shared memory with locks. Without the locks, it is nearly never useful, and the locks will increase the overhead.

Once more: you already have everything working OK in a preferred method. Stop remaking it, unless this is a real (measured) performance hog.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

5

This is pretty much my view. A simple I/O architecture in the driver: =
start
the transfer; when the transfer is complete (which is unrelated to
interrupts; it might be one interrupt per transfer, or 200, who cares?)
return the IOCTL with IoCompleteRequest, after starting the next packet =
from
your queue. If you have eight slots (e.g., an 8-channel card) then =
there
would be eight queues. Not a complex design at all. If you need to =
“keep it
fed” let the application provide the data as a series of async IOCTLs. =
I
give the 1-queue sample to my students as a class exercise and they =
finish
the logic in 2-3 hours. Building gratuitously complicated solutions to
simple problems is not productive. The whole notion that “interrupts”
matter is already suggestive of the wrong approach. Interrupts matter =
only
to the driver, not to the application.

The notion of 1-sided vs. 2-sided communication suggests that there is a
misinterpretation of the role of application and driver. As already =
pointed
out, drivers are servers and apps are clients. This feels like the kind =
of
question I see in another NG that says “I want my server program to be =
able
to connect to my client programs and send them data” and ignores the
fundmental client/server asymmetry (as well as NAT, in the case of
networks). OTOH, I get design ideas like this all the time from =
realtime
embedded people, who treat apps and drivers as coroutines of each other
(they share the same address space, typically, have effectively =
zero-latency
switching times between driver and app, etc.) Unfortunately, these =
models
do not translate well to non-real-time systems (such as Windows, linux,
Unix, Mac OS X, Solaris, and several other operating systems I know of =
or
have worked in, most of them on what is now dead hardware).

The key to success is to understand the model that the system presents, =
and
build solutions that are as simple as possible within that model =
(Einstein
observed: “a theory should be as simple as possible, but no simpler”). =
If
we ignore the PnP/Power nonsense and use KMDF to hide this nonsense, =
then
presumably we are left solely with solving the problem.

I characterize driver solutions as being along a continuum:
User|…|Kernel. You pick a point on the continuum and decide =
how
much effort goes into user space and how much into kernel space. Always
keep in mind that the closer you draw that line to user space, the more =
you
have to put in the kernel, and every line in kernel space costs vastly =
more
than a corresponding line in user space. So you can build unbelievably
complex models in the kernel to handle this, at vast cost, and probably =
get
an unreliable, unmaintainable driver that costs a huge amount to =
develop, or
build the simplest possible driver in the kernel (but no simpler!), and =
put
the burden in user space. A common failure of thinking in this regard =
is to
think of the I/O API (ReadFile/WriteFile/DeviceIoControl) as the
programmer’s interface to the device; the proper approach is to design =
the
abstract interface to the device and then split the work between user =
space
(such as in a DLL) and kernel space. The programmer might never see a
ReadFile, WriteFile, or DeviceIoControl, but work entirely at a much =
higher
level. If, to work properly, the library has to queue 50 =
DeviceIoControls,
so what? Nobody cares, any more than a user-level programmer cares =
about
the concept of “interrupt”. The user should never see the hardware or =
have
a clue as to the details of how it works; the interface spec says what =
the
task is (“Read some data”) and the data gets read or an error occurs. =
What
happens below that level is uninteresting to the application writer.
joe

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Bercea Gabriel
Sent: Thursday, December 18, 2008 8:57 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Query regarding shared memory

Maxim is right, IOCTL with direct I/O for ex would be the best choice =
here.
Shared memory has views, which will be mapped process dependent, and you
will need to use the system wide dispatcher to handle concurrency to the
memory most likely which will involve serious overhead.
Design your application so it is IOCTL request dependent, and you can =
get on
moving with you real work instead of trying to achieve performance with =
VM.

which is slow and one sided from Win32 -? to-? Driver

By far that is not an issue, first of all you can make workarounds to =
avoid
needing two ways communication, and secondly it will ease the overhead =
in
your implementation this way. Think as the driver as a server serving
requests to a client (your win32 app) Would you really like to make your
win32 app be a server as well serving info for the driver ? I don=92t =
think
this is where you need to go.
I really suggest redesigning, by adding more feasibility to your control
codes` internal design. Choose a good input data, and output data as =
well.
Define as many flags as you need, as many enum states as you need, as =
many
own status code as you need. This will help you not needing a two way
communication.

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. =
Shatskih
Sent: Friday, December 19, 2008 2:51 AM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Query regarding shared memory

win32 application (USER MODE). Currently this we are doing using IOCTL=20
which is slow

Why do you think it’s slow? have you measured?

and one sided from Win32 -=E0 to-=E0 Driver. Using share memory I want =
to=20
update that memory in driver and simply wants to signal win32 =
application
to read that data.

  1. Signalling will be slower then IOCTL, or around the same time.
  2. You cannot pass any data item with signalling if you do not use =
    inverted
    call, which is IOCTL.
  3. Do not forget guarding the shared memory with locks. Without the =
    locks,
    it is nearly never useful, and the locks will increase the overhead.

Once more: you already have everything working OK in a preferred method.
Stop remaking it, unless this is a real (measured) performance hog.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:=20
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: =
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:=20
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: =
‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

–=20
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

6

>question I see in another NG that says "I want my server program to be able

to connect to my client programs and send them data" and ignores the
fundmental client/server asymmetry (as well as NAT, in the case of
networks).

…as well as lots of security issues listed by Howard/DeBlanc in their book.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

7

Ohhh…SECURITY! (Never mind that these people don’t know about SSH, key
exchanges, and all the fundamental concepts that matter…they want the
server to be able to connect to the client, and a significant percentage of
them say “And I want to use UDP because it is so much easier to program”)
joe

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Thursday, December 18, 2008 9:44 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Query regarding shared memory

question I see in another NG that says “I want my server program to be
able to connect to my client programs and send them data” and ignores
the fundmental client/server asymmetry (as well as NAT, in the case of
networks).

…as well as lots of security issues listed by Howard/DeBlanc in their
book.

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com

NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com

–
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.

8

> them say “And I want to use UDP because it is so much easier to program”)

ROTFLMAO!

–
Maxim S. Shatskih
Windows DDK MVP
xxxxx@storagecraft.com
http://www.storagecraft.com