in need of help sharing memory allocated by user program with ndis 5 IM

Hi everyone hope you are all well.
Using osrmemdrv.cpp as my guide I’m unable to access the memory allocated by the userland test program. The steps taken are as follows:
In the userland test program:
pAllocMem = ( long * ) malloc( BufSize );
strncpy ( (char *) pAllocMem, “METHOD_OUT_DIRECT”, 17 );
DeviceIoControl using overlapped I/O to pass driver pAllocMem
check ERROR_IO_PENDING which it is
In the driver:
using METHOD_OUT_DIRECT
pOutputBuffer = (MmGetSystemAddressForMdlSafe ( pIrp->MdlAddress, …
and save the pOutputBuffer into pAdapt->pMyBuffers
exit this DeviceIoControl by:
pAdapt->MypIrp = pIrp; // to be IoCompleteRequest’d in a diff
DeviceIoControl
NtStatus = STATUS_PENDING;
IoMarkIrpPending ( pIrp );
In another part of the driver:
if pAdapt->pMyBuffers not zero [ts initd value]
pnxtbuf = pAdapt->pMyBuffers;
// next will BSOD - when NOT MDL. METHOD_OUT_DIRECT and IOComplete early.
ach = *pnxtbuf;
I then DBGPRINT ach %X but it is not the value put there by the userland program.

What is this newbie doing wrong? Please help this misguided individual.

Your actual code, rather than your summary of your actual code, would be
more useful.

Mark Roddy

On Sun, Mar 30, 2014 at 2:27 PM, wrote:

> Hi everyone hope you are all well.
> Using osrmemdrv.cpp as my guide I’m unable to access the memory allocated
> by the userland test program. The steps taken are as follows:
> In the userland test program:
> pAllocMem = ( long * ) malloc( BufSize );
> strncpy ( (char *) pAllocMem, “METHOD_OUT_DIRECT”, 17 );
> DeviceIoControl using overlapped I/O to pass driver pAllocMem
> check ERROR_IO_PENDING which it is
> In the driver:
> using METHOD_OUT_DIRECT
> pOutputBuffer = (MmGetSystemAddressForMdlSafe ( pIrp->MdlAddress,
> …
> and save the pOutputBuffer into pAdapt->pMyBuffers
> exit this DeviceIoControl by:
> pAdapt->MypIrp = pIrp; // to be IoCompleteRequest’d in a
> diff
> DeviceIoControl
> NtStatus = STATUS_PENDING;
> IoMarkIrpPending ( pIrp );
> In another part of the driver:
> if pAdapt->pMyBuffers not zero [ts initd value]
> pnxtbuf = pAdapt->pMyBuffers;
> // next will BSOD - when NOT MDL. METHOD_OUT_DIRECT and IOComplete early.
> ach = *pnxtbuf;
> I then DBGPRINT ach %X but it is not the value put there by the userland
> program.
>
> What is this newbie doing wrong? Please help this misguided individual.
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Thank you for a speedy response Mark.
Thought I’d start less but see the sense in your statement.
As I’m also a C, C++ L plater I have just found my problem. I’m embarrassed to admit that
the intended shared memory block was released when I exited the function that set it up.
I apologise to all for wasting your time.

> Hi everyone hope you are all well.

Using osrmemdrv.cpp as my guide I’m unable to access the memory allocated
by the userland test program. The steps taken are as follows:
In the userland test program:
pAllocMem = ( long * ) malloc( BufSize );
strncpy ( (char *) pAllocMem, “METHOD_OUT_DIRECT”, 17 );

Note that this will fail disastrously if BufSize < 17. Note also that the
string in pAllocMem will not have a terminal NUL character.

DeviceIoControl using overlapped I/O to pass driver pAllocMem
check ERROR_IO_PENDING which it is

So what happens when it is completed? Who frees the buffer? Where is
your OVERLAPPED structure? Why are you using variables whose scope and
lifetime are indeterminate from the information presented?

In the driver:
using METHOD_OUT_DIRECT
pOutputBuffer = (MmGetSystemAddressForMdlSafe ( pIrp->MdlAddress, …
and save the pOutputBuffer into pAdapt->pMyBuffers
exit this DeviceIoControl by:
pAdapt->MypIrp = pIrp; // to be IoCompleteRequest’d in a diff
DeviceIoControl

Where is pAdapt declared? Note that it must not e a global variable. I
have no idea what that comment means.

NtStatus = STATUS_PENDING;
IoMarkIrpPending ( pIrp );

Many things missing here. What happens if another DeviceIoControl arrives
before this one finishes? Where do you enqueue it? What does your
dequeue routine do? What value to you return?

In another part of the driver:
if pAdapt->pMyBuffers not zero [ts initd value]
pnxtbuf = pAdapt->pMyBuffers;

This makes it look like you are using a global variable. As a beginner,
take this advice: as soon as you declarea global or static variable,
unless it has the const qualifier, you have made a serious coding error.

You may also ignore the people who jump in and say I’m wrong, that there
are valid uses for non-const global variables. They are experts. They
are also right. But as a beginner, you have no idea of the trouble you
are about to get into if you use a global variable. Those of us who are
experts know when to use them and how to use them, and more facts about
their use than you can imagine. Until you understand what is going on,
global variables are a path to total disaster. So the first thing you
have to do is remove them. That’s what device extensions, controller
extensions, and FILE_OBJECT extensions are for. And you should know the
implications for each one. For now, the device extension is the likeliest
place to put these values.

When putting upa code snippet, you MUST show the declarations of all
variables used, and, most importantly, WHERE they are declared. Scope and
extent matter. A lot.

// next will BSOD - when NOT MDL. METHOD_OUT_DIRECT and IOComplete early.
ach = *pnxtbuf;
I then DBGPRINT ach %X but it is not the value put there by the userland
program.

Yes, and so what? It doesn’t work, sure, but where does this appear?
Where is pnxtbuf declared? Where is it set (you show code about HOW it is
set, but don’t explain where that code is). It can’t be a global
variable, but even after you fix that mistake, we still need to know WHERE
that line of code is. And, given you are testing a variable for NULL,
where is that variable RESET to NULL?

You have not explained anything about the structure of your driver, and
there is no concept of an IRP being completed “early”. An IRP is
completed when your driver is done with it, and it is your responsibility
to see that all state relevant to that IRP is properly “reset”.
Furthermore, there should be no execution path that is possible after the
IRP has completed that requires any state set on behalf of that IRP.
You’re in your ISR, and there’s no IRP? Well, that’s your responsibility
to check for. If your ISR is using values associated with some IRP that
has already completed, that is a programming error. Fix it. And what are
you going to do with the data that just arrived? You’d better have an
answer for that (“throw it away” could be a valid answer, but if it isn’t,
you need to handle it). I could even ask, “Why is your device still
interrupt-enabled if there is no active IRP for it?” and you’d better have
a good answer that matches well with the “What do you do if you get an
interrupt and there is no active IRP.” question (they are linked).
Generally, you have a “current IRP” pointer which points to the most
recent IRP you have dequeued, and which is set to NULL right before you
complete the IRP. Testing *this* pointer is what keeps you from executing
code paths that would depend on having a valid IRP.

What I think I am seeing is a driver which contains a huge number of
design and coding errors. The major design error seems to be in enqueing
and dequeing IRPs, and maintaing state for the current active IRP, if any
(and “if any” is important!).
joe

What is this newbie doing wrong? Please help this misguided individual.


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

Thanks for that Joe. Very welcome and a great insight fpr me. I can see there is a lot more reading and understanding that I need to do.

I am truly amazed; not only with the exceptional technical knowledge [gained from years of hands on experience and an inner drive for personal achievement: I suspect] but mostly with the freely giving of this hard won knowledge. I have read a few posts here and thank you all in the extreme for your enlightenment. Without support from individuals such as yourselves life might well become overwhelming. Especially when you’re thrown in the deep end.
In short guys, thanks for being who you are.

Joe, having mulled over some of your points the first is the global variable one.
“unless it has the const qualifier, you have made a serious coding error.”
Unfortunately I have too many coding problems that I am happy to be made aware of.
“For now, the device extension is the likeliest place to put these values.”
I chose to place my variables in the pAdapt struc purely because it appeared to be the easiest area to get the address of; not based on any technical understanding; which by now is patently obvious I do not have. Also it seemed that as it was initialised once at the start of the ioctl interface which was set up by the miniport at device registration I assumed it would be around when I needed to access my bits. [yes I assumed?!]. I will take your advice and move it to the device extension.
Now for a silly question. Where is the struct for this?

Great phrase that I haven’t heard in years. I wonder how many on this forum will understand that one.

You create it yourself. The content and size of the Device Extension can be anything you want. Just define the struct and pass sizeof(YOUR_STRUCT) as the size of the device extension when you call IoCreateDevice (or whatever).

Since nobody has said it yet… you should really be doing this driver in KMDF and not in WDM. You’ll learn a useful skill, not an old dead one, and you’ll avoid zillions of potential problems.

Peter
OSR
@OSRDrivers

Frequently the act of asking for help itself provides the answer.

Mark Roddy

On Sun, Mar 30, 2014 at 7:07 PM, wrote:

> Thank you for a speedy response Mark.
> Thought I’d start less but see the sense in your statement.
> As I’m also a C, C++ L plater I have just found my problem. I’m
> embarrassed to admit that
> the intended shared memory block was released when I exited the function
> that set it up.
> I apologise to all for wasting your time.
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

google has wrecked trivia forever. I always knew what an L-plater was. Or
not.

Mark Roddy

On Mon, Mar 31, 2014 at 9:12 AM, wrote:

>


>
> Great phrase that I haven’t heard in years. I wonder how many on this
> forum will understand that one.
>
>
>


>
> You create it yourself. The content and size of the Device Extension can
> be anything you want. Just define the struct and pass sizeof(YOUR_STRUCT)
> as the size of the device extension when you call IoCreateDevice (or
> whatever).
>
> Since nobody has said it yet… you should really be doing this driver in
> KMDF and not in WDM. You’ll learn a useful skill, not an old dead one, and
> you’ll avoid zillions of potential problems.
>
> Peter
> OSR
> @OSRDrivers
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>

Thanks for the info. KMDF - well pleased you mentioned that. The WINDDK only has one ? sample sdiosample.htm. Can you offer a good starting place?
Nice one folks.

Just so I [think] I’m on the right track:-
device object pointer returned by IoCreateDevice/Secure etc?
contains a pointer to the device extension viz:
PVOID DeviceExtension;
of size determined by sizeof(what-you-need).
I’ll start moving my data to a DeviceExtension now.
Now the next Q is; does the system have any info at this memory and ours is tacked on the end?
or is it for a particular driver writers sole use? Hmmm! I can’t see anything such as an offset to beginning of user defined part so I guess the latter is true. Well I suppose the kernel could use -ve offsets.
Anyway guys thanks.

Just thought I’d share this.
http://j00ru.vexillium.org/?p=1220

It is rare that a NDIS IM or NDIS LWF actually uses device extension. In
this area a global data structure is commonly used. See the WDK NDIS IM
driver samples.

Thomas F. Divine
http://www.pcausa.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of xxxxx@hotmail.co.uk
Sent: Monday, March 31, 2014 10:40 AM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] in need of help sharing memory allocated by user program
with ndis 5 IM

Just so I [think] I’m on the right track:- device object pointer returned by
IoCreateDevice/Secure etc?
contains a pointer to the device extension viz:
PVOID DeviceExtension;
of size determined by sizeof(what-you-need).
I’ll start moving my data to a DeviceExtension now.
Now the next Q is; does the system have any info at this memory and ours is
tacked on the end?
or is it for a particular driver writers sole use? Hmmm! I can’t see
anything such as an offset to beginning of user defined part so I guess the
latter is true. Well I suppose the kernel could use -ve offsets.
Anyway guys thanks.


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

I’ll certainly check that out thanks.
Nothing is straight forward.
The driver I’m playing with uses NdisMRegisterDevice and I then found this
http://www.osronline.com/showThread.CFM?link=85802 with google; so a lot more RT?M is required which is no bad thing. [at least in the end].
keep happy.

> I am truly amazed; not only with the exceptional technical knowledge

[gained from years of hands on experience and an inner drive for personal
achievement: I suspect] but mostly with the freely giving of this hard
won knowledge. I have read a few posts here and thank you all in the
extreme for your enlightenment. Without support from individuals such as
yourselves life might well become overwhelming. Especially when you’re
thrown in the deep end.
In short guys, thanks for being who you are.

Joe, having mulled over some of your points the first is the global
variable one.
“unless it has the const qualifier, you have made a serious coding error.”
Unfortunately I have too many coding problems that I am happy to be made
aware of.
“For now, the device extension is the likeliest place to put these
values.”
I chose to place my variables in the pAdapt struc purely because it
appeared to be the easiest area to get the address of; not based on any
technical understanding; which by now is patently obvious I do not have.

For every complex problem, there is an answer which is simple, obvious,
and wrong.

It is no harder to get the address of a value in the device extension than
it is to get the address of a global variable; the & operator (prefix)
works very well in all cases.

Global variables are a disaster if two threads are accessing them…and
not just the case where synchronization is needed. The correctness of the
data must be maintained on a per-thread basis. So you can properly
synchronize access with a spin lock, but if the new data you write causes
some other thread that expected its data to be there to fail, then your
driver is just wrong. They also fail horribly when you can have two or
more instances of your device on a machine. All instances of your device
share the same one (1) location!

Also it seemed that as it was initialised once at the start of the ioctl
interface which was set up by the miniport at device registration I
assumed it would be around when I needed to access my bits. [yes I
assumed?!].

This is not single-threaded app programming. In a driver, you will
typically deal with a MINIMUM of three threads of control, which must be
well-behaved, but there is no maximum value (except the limits of the
computer memory); you could have 100 threads talking to your device. But
even if you open it exclusive, there is no limit, because the open lets
every thread of your app have access, and although there is an enforced
handle-count limit on threads, it is sufficiently large that from your
driver’s viewpoint there is no limit. And, even if you restrict the
action to a single thread…and this is a Very Bad Idea, because it cannot
be easily enforced without disrupting application expectations, there are
still your passive thread, the DPC, and the ISR to worry about.

Those of us who have been doing multithreading for years understand these
issues. But a tiny mistake can do wonders for data integrity. I’ve seen
driver errors that have rendered the machine unbootable, in a test driver
with no hardware (if you are working on a file system, you assume that all
bugs are fatal this way, and you will be safe). Assuming worst-case is
good; assuming ideal-case or best-case is dangerous. In a driver, you
have to assume the worst possible situations will arise, and be prepared
to deal with them in a non-BSOD way.

Data persistence is a serious concern (often called “extent” of the data
lifetime). Global variables have indefinite persistence, which is good,
but that means that you can find yourself in a context in which your
persistent data has persisted beyond its use-by date, and the IRP that
generated that data is long gone. The data is stale, but still there.
And that is the least of the problems of global variables. It gets far
worse.

I will take your advice and move it to the device extension.
Now for a silly question. Where is the struct for this?

Whatever you want. You declare a struct holding whatever values you want
to hold, and pass its sizeof() into the IoCreateDevice. You will have a
void* pointer to it in the device object, which you will have to cast to
your struct type. So one of the “boilerplate” lines in a function is

PMYDEVICEEXTENSION ext = (PMYDEVICEEXTENSION)dev->DeviceExtension;

Now the address of any value you want is trivial:

&ext->whatever

is the address of the variable. Be careful passing pointers around,
though; you have to ensure that the data location referenced by the
pointer still exists at the time the pointer is used (extent, again). For
device extensions this is easy, but general pointer-passing is far, far
less forgiving than in a single-threaded application.
joe


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

“For every complex problem, there is an answer which is simple, obvious, and wrong.”
This is so true, still, had to laugh - knowingly.
Phew! lots to keep in mind with this driver writing. Well it’s onwards I go.
Thanks for helping me think on a deeper level.

Driver writing is not for the faint of heart. As an app programmer, I can
sustain, and for the last 30 years have been able to sustain, writing 200
source lines per hour. Plus or minus 20. I would tell my students, “If
I’m writing driver code at a tenth of that speed, I’m coding far too fast
and making sloppy-coding errors”. Some folks have said that on a good day
they end up with a positive lines-of-code-per-day number; on a bad day, it
can go to large negative numbers. And these are /experienced/ driver
writers!
joe

“For every complex problem, there is an answer which is simple, obvious,
and wrong.”
This is so true, still, had to laugh - knowingly.
Phew! lots to keep in mind with this driver writing. Well it’s onwards I
go.
Thanks for helping me think on a deeper level.


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

+1

Probably a target system reboot and driver reload per line of code when all
is said and done…

Thomas

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of
xxxxx@flounder.com
Sent: Monday, March 31, 2014 9:24 PM
To: Windows System Software Devs Interest List
Subject: RE:[ntdev] in need of help sharing memory allocated by user program
with ndis 5 IM

Driver writing is not for the faint of heart. As an app programmer, I can
sustain, and for the last 30 years have been able to sustain, writing 200
source lines per hour. Plus or minus 20. I would tell my students, “If I’m
writing driver code at a tenth of that speed, I’m coding far too fast and
making sloppy-coding errors”. Some folks have said that on a good day they
end up with a positive lines-of-code-per-day number; on a bad day, it can go
to large negative numbers. And these are /experienced/ driver writers!
joe

“For every complex problem, there is an answer which is simple,
obvious, and wrong.”
This is so true, still, had to laugh - knowingly.
Phew! lots to keep in mind with this driver writing. Well it’s onwards
I go.
Thanks for helping me think on a deeper level.


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer


NTDEV is sponsored by OSR

Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev

OSR is HIRING!! See http://www.osr.com/careers

For our schedule of WDF, WDM, debugging and other seminars visit:
http://www.osr.com/seminars

To unsubscribe, visit the List Server section of OSR Online at
http://www.osronline.com/page.cfm?name=ListServer

The full implications of your advice do elude me but that will become evident as I progress. However, I’m likely to make less errors I hope.
When I can ask an intelligent question I’ll be back.
Thanks everyone,
daunted. but not put off yet.

Hello guys,
hope you’re all well, especially now the warmer weather is here.
I’m being caught out with a basic misunderstanding and the hidden traps for young players. Although I don’t get a BSOD the machine does decide to reboot, ?randomly?, after many hours, I think it’s when the processing of a TCP packet i.e. when I use the shared mem in quick succession. When only a few TCP packets arrive/leave all appears well. In other cases the Rx/Tx of many packets in quick succession causes no problem?? Although the purpose is to queue the packets I simply RtlCopyMemory all packets on top of each other. i.e. I don’t update the queue pointers, yet.
[as an aside I didn’t realize opening a local folder search window caused TCP packets to 137.117.86.84 viz: GET /sasearch/lclAdv.xml HTTP/1.1]. Might be speaking too soon but opening a search window with its few TCP packets doesn’t cause a prob.Yet. As my starting point I have taken the extended Passthru example and attempted to add ‘my flavour’ of shared memory based on the other example from [I believe some of yourselves.] osrmemdrv.cpp. My initial attempt to take your advice and move my ‘global’ data failed as the IoCreateDeviceSecure is not used instead replaced with NdisMRegisterDevice that doesn’t have a DeviceExtensionSize param and together with http://www.osronline.com/showThread.CFM?link=85802 put a kybosh on it! So now I’m wondering if the problem is ‘data extent/lifetime’ as Joe pointed out. i.e. the shared memory not being available. However, I would have expected a definite BSOD for this. My implementation is certainly not as elegant as that in osrmemdrv.cpp but the basic idea is much the same.
I’ve just got rid of 40 odd compiler warnings to do with casts; wondering hopefully but not expectantly that t will help. Haaha Of course, not surprisingly, it didn’t! Not getting to the bottom of this problem isn’t going to bring about mass destruction or cause me to end up destitute; it IS however, causing me sleepless nights.
I haven’t even got 2 instances of test programs running yet! The fact that the CreateFile doesn’t specify sharing we can overlook for now]. Heck, I haven’t even got round to adding [or learning about] the cancel side of things yet.
Well you’ve noticed I haven’t asked a question yet. That’s because I haven’t got an intelligent one I’m afraid. The best I can do is to ask “should one expect a BSOD if the MmGetSystemAddressForMdlSafe memory was accessed if it was out of scope?” It had been accessed many times. The driver definitely hadn’t finished with it; proved by my DbgView log. As I stated earlier the machine reboots rather than bug-checking. I’m hoping this might be a clue?
I’m sure you are all pretty much fed up with such indeterminate queries like this so I won’t be offended if it goes straight to the Trash can. I’m not sure how to proceed. Excuse my rambling on.
Naturally it is very intermittent. It was running for over 24 hours before it last occurred.
Any pointers will be gratefully accepted.
All the best,
ian t