race conditions

NTFSD
1

I have a question regarding data manipulation in race conditions.
For example in my filter driver I can see that I have a lot of DispatchCreate calls, also a lot of DispatchRead calls.
I want to know if these calls can cause race conditions, if I process them shyncronously (completion routine, etc. . .)

By race conditions I mean, if my dipatch read routine “comunicates” with a driver entry created system thread (because I want to access pageable data with the thread that is running at PASSIVE_LEVEL, and my dispatch read which may run at DISPATCH_LEVEL I can’t. In my dispatch read I let all IRP_PAGING_IO and IRP_SYNCHRONOUS_PAGING_IO pass through), with the help of a globally decalred event and a locked nonpaged Mdl, can I have the risk that when, in a read request, that I want to pass some data to the thread and wait for it on the event for the thread to finish, another request is made in this time, and changes the event state, or tries to access the the mdl and modify it, and my “first” request get’s junk data because of other concurent request.
Is this possible or the IO manager waits for the Irp’s to be completed before calling the dispatch routines, for the next Irp.

I tried to use FAST_MUTEX for this issue but eventualy the driver slowly freezez the computer. What can happen if I acquire a fast mutex, get the control of the fast_mutex, and in the time I try to process data more then 2 threads try to acquire the mutex, this means they go in the wait state, but which of the threads will gain control of the mutex first alfter I release it ?
How do you handle such problems generally in a filter driver ?
thank you.

2

To be honest, I don’t really understand what you are asking about. Look at your description

[begin quote]

…if my dipatch read routine “comunicates” with a driver entry created system thread (because I want to access pageable data with > the thread that is running at PASSIVE_LEVEL, and my dispatch read which may run at DISPATCH_LEVEL I can’t. In my dispatch read I let all IRP_PAGING_IO and
IRP_SYNCHRONOUS_PAGING_IO pass through), with the help of a globally decalred
event and a locked nonpaged Mdl…

[end quote]

Therefore, according to the above description, your Dispatch Read routine may run at DPC level, and, hence, cannot access pageable emomory. In order to solve the problem, you make it “communicate” with a thread that runs at IRQL==PASSIVE_LEVEL.

Now look at what you are asking about:

[begin quote]

… can I have the risk that when, in a read request, that I want to pass some data to the thread and wait for it on the event for the thread to finish…

[end quote]

Taking into account your description of a situation (i.e. the fact that Dispatch Read runs at DPC level), the only possible answer to your question stands as following:

“No, there is no risk of race condition whatsoever - you are guaranteed to BSOD right upon your call
to KeWaitForSingleObject() with non-zero timeout at DPC level, so that the question of race condition just has no chance to arise, in the first place”…

Anton Bassov

3

I have doubts about that
Let me give u a piece o code to see what I mean.

NTSTATUS DispatchCreate(…)
{
//as I said I pass through all the Irp’s that have something to do with the paging file
//so
irpSp->Flags == SL_OPEN_PAGING_FILE
//passes through

//all other requests have to execute this code

KeWaitForSingleObject(&CreateEvent,…);
//the CreateEvent variable is a global KEVENT that is initaly signaled so the first request passes
KeResetEvent(&CreateEvent); //so future request will have to wait on this one

// now I want to fill a structure with the create parameters to send to my worker thread
/*
typedef struct _createParameters
{
PFILE_OBJECT FileObjectPointer;
//UNICODE_STRING FileName;
ACCESS_MASK DesiredAccess;
PNL_DEVICE_EXTENSION_HEADER NLExtensionHeader;
KEVENT RequestEvent;
}CREATE_PARAMETERS, *PCREATE_PARAMETERS;

*/
CreateParams = MmGetSystemAddressForMdlSafe(GlobalSharedMdl,…);

CreateParams->FileObjectPointer = irpSp->FileObject;
CreateParams->…

// this is the most delicate part because I want this data written to the MDL to stay there for all the //processing time

KeSetEvent(&ThreadEvent,…);
//thread event is a global variable that is handled by the worker thread. the thread wait on this
//event for data to process
//now the worker thread knows it should process some information, store it in a buffer, etc…
KeWaiForSingleObject(&threadEvent,…);
//wait for the thread to finish processing
//the thread does additional writting to the mdl so it can tell the dispatch routine if it should pass the //IRP succesfully or not
//ex

PBOOLEAN HasCreateFileAccess = MmGetSystemAddressForMdlSafe(…);
if (*HasCreateFileAccess)
{
IoCompleteRequest(…)
return STATUS_SUCCESS
}
else
{…}

//after all this processing is done I release the initial event
KeSetEvent(&CreateEvent);
//now another thread that was waiting to make a request can go through

}

I showed u this more like algorithmically so you can undestand my issue better, because as you could easily see this is not actual code.
Now my question is that in the time that I call
KeResetEvent(&CreateEvent); //in the beginning
and
KeSetEvent(&CreateEvent); //in the end
more than one thread waits to make a request, how will they react after I signal the CreateEvent event.
which thread will get to write data in the globally shared Mdl, will they all try to do it at the same time :frowning:
Is this a good aproach of the problem ?
how can I implement a better thread queue ?

4

> I have doubts about that

Well, if, after a year of posting questions in NGs, you still have some doubts about BSOD after calling KeWaitXXX at elevated IRQL, I am afraid it just does not make sense to continue this discussion. Furthermore, as long as we are speaking about FS filter driver, it did not make sense to even start it, in the first place, as it follows from the quote below:

[begin quote]

Constraints on Dispatch Routines

The following guidelines briefly discuss how to avoid common programming
errors in dispatch routines for file system filter drivers.

IRQL-Related Constraints

Note: For information about which types of IRPs are used in paging I/O, see
Dispatch Routine IRQL and Thread Context.

Dispatch routines in the paging I/O path should never call IoCallDriver at
any IRQL above APC_LEVEL. If a dispatch routine raises IRQL, it must lower it
before calling IoCallDriver.

Dispatch routines in the paging path, such as read and write, cannot safely
call any kernel-mode routines that require callers to be running at IRQL
PASSIVE_LEVEL.

Dispatch routines that are in the paging file I/O path cannot safely call
any kernel-mode routines that require a caller to be running at IRQL <
DISPATCH_LEVEL.

Dispatch routines that are not in the paging I/O path should never call
IoCallDriver at any IRQL above PASSIVE_LEVEL. If a dispatch routine raises
IRQL, it must lower it before calling IoCallDriver.

…

[end quote]

This paragraph alone is sufficient for making a conclusion that your
driver’s Dispatch routines are just not going to get called at elevated IRQL.
You cannot cause *any* page faults while processing paging IO to the paging
file, and you cannot cause page faults on code (although you can cause them
on data) while processing paging IO to the “normal” file. This is the only
reason why IRQL has been mentioned in the above paragraph ( once you pass all
paging IO requests down the stack straight away, it does not apply to you
anyway). However, Dispatch routine itself will always be called at IRQL<
DISPATCH_LEVEL…

To summarize, you just have to redesign the whole driver…

Anton Bassov

5

Anton it seems that you have a problem with my post on every forum I post.
Ok I can see that you don’t like my questions, just avoid them.
If they too stupid for you, please keep going to the real challenges.
I appreciate the fact that you keep track of my posting over the time, and you try to analyze my evolution, but really now, if you see any of my posts again IGNORE them.
I can’t beleive I can’t come on a forum and ask for suggestions, opinions on a topic.
All you want is !analyze -v and code dump ?
If you read all the books about the topic, or if you are a GOD in the area, please, I admit I have a lot to learn, and I don’t need people like you in the way.
As for your MSDN dump, thank you very much, you didn’t understand the question, no help for me. I am already aware of what you wrote.
As I recall I asked you about RACE CONDITION, thread queues, not about elevated IRQL in dispatch routines. I don’t want IRQL. Do you want a picture for that ? You can’t read?
As I can see you problems are bigger than mine, and I see you, are not properly documented on the topic.
bye
I still wait for suggestions regarding race conditions in dispatch routines of file system filter drivers.
thank you

6

> -----Original Message-----

From: xxxxx@lists.osr.com [mailto:bounce-294915-
xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Tuesday, July 24, 2007 1:46 PM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] race conditions

Anton it seems that you have a problem with my post on every forum I
post.
[PCAUSA] I would agree with you that Anton is sometimes a little too harsh is replies.

Having said that, the points that Anton raised are essential for you to understand if you are ever to solve your problem.

You said “I don’t want IRQL.”

I read that as saying that you don’t care about what “IRQL” means and how it influences your code. You cannot solve your problem without understanding IRQL and how it influences coding.

Do you understand the IRQL concept and do you understand that it is simply impossible to use KeWaiForSingleObject to wait for an event to complete when you are at elevated IRQL?

Once you understand that you CANNOT wait at the desired point you may be able to find a solution to your problem or at least understand why your goal is not achievable.

Thomas F. Divine

7

No the problem is the following, I know IRQL is essential for my coding, and I know how to handle this problem (the IRQL problem).
When I said that I don’t care about the IRQL I meant that my question wasn’t about that.
The IRQL is not an issue right now for me.
If someone posts a question about a certain thing on the forum you don’t answer him what ever come in you mind, but what he asked you about, and I didn’t ask about the IRQL.
I only said that I what to prevent pageable code to be called at not an elevated IRQL , and I found the solution a worker thread, as I says it also says in the documentation.
The problem I am facing is different, and I ask for a good, efficient solution, that won’t decrease the computer performance.

The problem is I don’t get a substantial answer, as I see the answers rely on other dumb answers, as they should rely on the main questions.
I had other replies that agreed with the fact that several threads can make a Create request almost at the same time. This is my question. How can I efficiently “linearize” these calls, so they don’t make race conditions on my main shared memory between the worker thread and the dispatch routine.
I don’t say this is the only way to implement it, with a worker thread, but this is the way I read in the documentation it is done in filter drivers.
I am opened to hear any suggestions.
I believe this is an interesting issue.
How would you implement this, give me an algorithm as I wrote in my previous reply, anything that could help.
thank you.

8

> -----Original Message-----

From: xxxxx@lists.osr.com [mailto:bounce-294944-
xxxxx@lists.osr.com] On Behalf Of xxxxx@gmail.com
Sent: Tuesday, July 24, 2007 7:36 PM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] race conditions

No the problem is the following, I know IRQL is essential for my
coding, and I know how to handle this problem (the IRQL problem).
When I said that I don’t care about the IRQL I meant that my question
wasn’t about that.
The IRQL is not an issue right now for me.
If someone posts a question about a certain thing on the forum you
don’t answer him what ever come in you mind, but what he asked you
about, and I didn’t ask about the IRQL.
I only said that I what to prevent pageable code to be called at not an
elevated IRQL , and I found the solution a worker thread, as I says it
also says in the documentation.
The problem I am facing is different, and I ask for a good, efficient
solution, that won’t decrease the computer performance.

The problem is I don’t get a substantial answer, as I see the answers
rely on other dumb answers, as they should rely on the main questions.
I had other replies that agreed with the fact that several threads can
make a Create request almost at the same time. This is my question. How
can I efficiently “linearize” these calls, so they don’t make race
conditions on my main shared memory between the worker thread and the
dispatch routine.
[PCAUSA] Consider queuing the information from your dispatch routine. Dequeue in your worker thread. You will be implementing a FIFO.

You will need to allocate a small “context” structure that will be allocated and initialized in dispatch routine and freed in worker thread. Consider using Lookaside APIs for this sort of repetitive fixed-size allocations.

Consider using a SList for the queue.

Good luck,

Thomas F. Divine

I don’t say this is the only way to implement it, with a worker thread,
but this is the way I read in the documentation it is done in filter
drivers.
I am opened to hear any suggestions.
I believe this is an interesting issue.
How would you implement this, give me an algorithm as I wrote in my
previous reply, anything that could help.
thank you.

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@pcausa.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

9

Thomas,

[PCAUSA] I would agree with you that Anton is sometimes a little too harsh is replies.

I really hope you would agree that the above statement surely does not apply to this particular OP - in actuality, I think that, in this particular case, I was a way too soft…

To be honest, I am quite annoyed with posters who don’t listen to what they are advised to do and
who are too lazy to read documentation (which is obvious from their posts), and, instead, choose to abuse the NG by posting countless questions that mainly repeat themselves (because posters either ignore answers that they get or just unable to understand these answers, due to their unwillingness to read documentation). When you deal with someone who just does not want to learn, you have a feeling that you are sending data to the server that is permanently shut down, i.e just wasting your time. Therefore, sometimes I may give these posters rather harsh replies (sometimes a bit too harsh - I admit).

Up to this point I thought of the OP as of one of these posters. However, his latest reply to your post puts him into a basically different “weight class” - he seems to be the first person I’ve come across in so far who actually tries to justify his unwillingness to learn. He does not really care that his question in itself just does not make sense in context he asks it, and discards reasonable and logical arguments as " dumb answers" - he just counters them with “this is not what I was asking about”…

I am afraid “idiot” is the only term that applies to this particular OP - although the term in itself may sound offensive, it is just an accurate description of the OP’s situation…

Anton Bassov

10

Thomas,

[PCAUSA] Consider queuing the information from your dispatch routine. Dequeue in
your worker thread. You will be implementing a FIFO.

Actually, in such case it is, probably, easier just to queue a workitem, and pass it everything you want as a context. In case if workitems access some global resources you also have to make sure their access to these resources is synchronized (once workitem routines run at PASSIVE_LEVEL, it is really easy to do so)…

The funniest thing here is that the OP just does not need to care about it, simply because, as it follows from the excerpt that I have quoted earlier, his Dispatch routines will never get invoked at elevated IRQL anyway, and, according to him, he does not process IRPs on the paging IO path, so that he does not need to worry about avoiding page faults. However, this is already a different story…

Anton Bassov

11

There certainly have been more and more folks recently who 1.) do keep asking the same question in hopes that somehow the answer will change to suit their needs and 2.) seem to think that the folks who respond are a paid substitute for actually doing research of available material.

Thomas F. Divine

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-294951-
xxxxx@lists.osr.com] On Behalf Of xxxxx@hotmail.com
Sent: Tuesday, July 24, 2007 10:10 PM
To: Windows File Systems Devs Interest List
Subject: RE:[ntfsd] race conditions

Thomas,

> [PCAUSA] I would agree with you that Anton is sometimes a little too
harsh is replies.

I really hope you would agree that the above statement surely does not
apply to this particular OP - in actuality, I think that, in this
particular case, I was a way too soft…

To be honest, I am quite annoyed with posters who don’t listen to what
they are advised to do and
who are too lazy to read documentation (which is obvious from their
posts), and, instead, choose to abuse the NG by posting countless
questions that mainly repeat themselves (because posters either ignore
answers that they get or just unable to understand these answers, due
to their unwillingness to read documentation). When you deal with
someone who just does not want to learn, you have a feeling that you
are sending data to the server that is permanently shut down, i.e just
wasting your time. Therefore, sometimes I may give these posters rather
harsh replies (sometimes a bit too harsh - I admit).

Up to this point I thought of the OP as of one of these posters.
However, his latest reply to your post puts him into a basically
different “weight class” - he seems to be the first person I’ve come
across in so far who actually tries to justify his unwillingness to
learn. He does not really care that his question in itself just does
not make sense in context he asks it, and discards reasonable and
logical arguments as " dumb answers" - he just counters them with
“this is not what I was asking about”…

I am afraid “idiot” is the only term that applies to this particular OP

  • although the term in itself may sound offensive, it is just an
    accurate description of the OP’s situation…

Anton Bassov

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@pcausa.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

12

Hmmm . . .
Anton,

I have have the feeling you have some kind of egocentric attitude.
Ok first of all I don’t think that “folks” here are paid to do research on my work
I research my own work with the resources I have. I respect every one who comes here, who either answers posts, or puts posts on.
I see this as a community, where no one is obliged to answer any post that might sound ridiculous, etc. . . .
Anton my a question is not necessarily stupid if you consider it so.
And about my countless questions. . . . hmm it seems you have issues, I didn’t post questions on the forums for almost an year. What countless questions ?
Another thing.
I am not lazy to read documentation. I really do things as they are done professionally and that’s why my questions might sound ridiculous.
I don’t know why asking about how to implement a simple and elegant thread queue in kernel mode is so stupid and proves that I didn’t read a bit about anything ? I have made a queue like this, I think it is slow, I want to hear suggestions, because I am sure that I am not the only one who had this problem, or the only one that will have it.
Or you are just used to answer to the posts that have !analyze -v dumped ? Ohh this makes you very smart indeed. That’s what’s all about, that is the real thing, if you dump MSDN stuff or code you are the most documented and most equilibred guy right ?
These proves that you are an ignorant “macho”, who thinks, he’s the grandfather of all knowledge.
I am sickened by people like you.
Get a life . . .
And ofcourse, you always give me the usual bla bla bla:

Dispatch routines will never get invoked at elevated IRQL . . . .
he does not process IRPs on the paging IO path, so that he does not need to worry about avoiding page faults.
Dude I am aware of that. I know what I am doing.
I know I avoid page faults. Ok you a are a genius. You have discovered I let paging Irp’s pass through, and even so I what to implement a thread queue. Do you want a medal for that ? Ok I will make big shiny card with your discovery and send it to you. just give me the shipping address.
why do you always bring up something that is not in discussion ?
and if you think I am undocumented please recommend me some documentation, oh grandfather of all knowledge

13

*THIS IS OFF -TOPIC ISSUE*

I don’t know if someone keeps a track of funny statements that were made in the NG, but if someone does, I would suggest paying a special attention to the one below - this is a true masterpiece

[begin quote]

I really do things as they are done professionally and that’s why my questions might sound ridiculous.

[end quote]

Sorry, guys, but I just could not resist…

Anton Bassov

14

I meant, I want things to be done professionally, I am sorry.

15

Now children, children… Play nice, or daddy will have to get the whip.

Mr. Bassov: The repeated less-“erudite” questions annoy me also, so I just don’t bother to answer them. You might give that technique a try.

Mr. Bercea… Please actually READ the replies you’ve been given, and avoid dragging whatever history you have with Mr. Bassov onto this list.

EVERYBODY: If you don’t like the answer you get from somebody on the list, either: (a) ignore it, or (b) reply directly to the person and trade insults with them via direct email.

In ALL cases, we DO ask that you help us by reporting any truely hateful, racist, biggoted, xenophobic, or cruel replies directly to the List Slaves (xxxxx@osr.com). Such behavior will not be tolerated, as described in the joining notes. I do not consider this thread to have risen anywhere NEAR that level.

Most everybody likes a little flaming now and then. It keeps us all awake. Let’s work together to make sure it doesn’t detract from our mission on the list or make lots of folks feel unwelcome.

Peter
List Slave Technical Fellow

16

>You will need to allocate a small “context” structure that will be allocated
and

initialized in dispatch routine and freed in worker thread. Consider using
Lookaside APIs for this sort of repetitive fixed-size allocations.

If 4 pointers are enough - Tail.Overlay.DriverContext can be used instead.

–
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com