Hello, I have the following pseudo-code implemented in a driver:
RegisterProcessNotifyRoutine()
TakeProcessTableLock(EXCLUSIVE)
RegisterThreadNotifyRoutine()
CollectCurrentProcessesAndThreadFromTheSystem()
ReleaseProcessTableLock()
In my process notification routine I take the process lock exclusively and
I insert the new process created in my process table.
In my thread notification routine I take the process lock shared when
looking up the process the thread belongs to in the table.
The CollectCurrentProcessesAndThreadFromTheSystem function uses
ZwQuerySystemInformation to retrieve the active processes in the system and
add them to my process table.
Because I’ve registered the process notify routine before collecting the
active processes from the system and because until I collect all the
processes the thread notification routines are ‘stuck’ trying to acquire
the process lock I see no way in which the following scenario can happen:
The thread notify routine is called for a new thread for a process which I
DON’T have in my process table. Also, when this nonexistent process
terminates (my process routine notify is called) it’s still not present in
the process table.
This scenario is easily reproduced using 5 batch scripts which every 100 ms
create a new process and wait for its termination.
Am I missing something or this behavior shouldn’t be happening if my code
actually works as I describe it? 
Calling the complex paths like ZwQuerySystemInformation under the lock is usually a no-no.
Do the following:
- create the table and start gathering info using Ps callbacks
- then take a snapshot with ZwQuerySystemInformation, grab the table lock, and merge the snapshot to the table under the lock.
- you need to specially treat the “recently deleted” objects, which were deleted between ZwQuerySystemInformation and your lock grab.
–
Maxim S. Shatskih
Microsoft MVP on File System And Storage
xxxxx@storagecraft.com
http://www.storagecraft.com
“Gurzou Alexandru” wrote in message news:xxxxx@ntdev…
Hello, I have the following pseudo-code implemented in a driver:
RegisterProcessNotifyRoutine()
TakeProcessTableLock(EXCLUSIVE)
RegisterThreadNotifyRoutine()
CollectCurrentProcessesAndThreadFromTheSystem()
ReleaseProcessTableLock()
In my process notification routine I take the process lock exclusively and I insert the new process created in my process table.
In my thread notification routine I take the process lock shared when looking up the process the thread belongs to in the table.
The CollectCurrentProcessesAndThreadFromTheSystem function uses ZwQuerySystemInformation to retrieve the active processes in the system and add them to my process table.
Because I’ve registered the process notify routine before collecting the active processes from the system and because until I collect all the processes the thread notification routines are ‘stuck’ trying to acquire the process lock I see no way in which the following scenario can happen: The thread notify routine is called for a new thread for a process which I DON’T have in my process table. Also, when this nonexistent process terminates (my process routine notify is called) it’s still not present in the process table.
This scenario is easily reproduced using 5 batch scripts which every 100 ms create a new process and wait for its termination.
Am I missing something or this behavior shouldn’t be happening if my code actually works as I describe it?
You could also fire a workitem with an event and a payload as context and
wait for the workitem to finish with your lock held like so
having a context like this:
WORKITEM_CTX
{
PKEVENTE WorkITemDone;
PIO_WORKITEM CurrentItem;
PVOID Data; // out
ULONG DataLength //out
NTSTATUS rc; //last error code
}
TakeProcessTableLock(EXCLUSIVE)
RegisterThreadNotifyRoutine()
CollectCurrentProcessesAndThreadFromTheSystem() - this done in a worktitem
with the above ctx or similar and wait for it to end via the Event
ReleaseProcessTableLock()
But nevertheless the bug or manifestation you see must be somewhere in your
code.
You could always do what Maxim suggested, that is starting with the
PsCallbacks and let them do their bidding as normal and then “sync” what is
missing via a function call of QuerySystemInfo and cleanup
what is not there anymore.
It would be much easier if your driver would start at boot because there
are only some system threads that you are missing and the system process
Gabriel
On Tue, Sep 22, 2015 at 6:42 AM, Maxim S. Shatskih
wrote:
> Calling the complex paths like ZwQuerySystemInformation under the
> lock is usually a no-no.
>
> Do the following:
>
> - create the table and start gathering info using Ps callbacks
> - then take a snapshot with ZwQuerySystemInformation, grab the table
> lock, and merge the snapshot to the table under the lock.
> - you need to specially treat the “recently deleted” objects, which were
> deleted between ZwQuerySystemInformation and your lock grab.
>
> –
> Maxim S. Shatskih
> Microsoft MVP on File System And Storage
> xxxxx@storagecraft.com
> http://www.storagecraft.com
>
> “Gurzou Alexandru” wrote in message
> news:xxxxx@ntdev…
> Hello, I have the following pseudo-code implemented in a driver:
>
> RegisterProcessNotifyRoutine()
>
> TakeProcessTableLock(EXCLUSIVE)
> RegisterThreadNotifyRoutine()
> CollectCurrentProcessesAndThreadFromTheSystem()
> ReleaseProcessTableLock()
>
>
> In my process notification routine I take the process lock exclusively and
> I insert the new process created in my process table.
>
> In my thread notification routine I take the process lock shared when
> looking up the process the thread belongs to in the table.
>
> The CollectCurrentProcessesAndThreadFromTheSystem function uses
> ZwQuerySystemInformation to retrieve the active processes in the system and
> add them to my process table.
>
> Because I’ve registered the process notify routine before collecting the
> active processes from the system and because until I collect all the
> processes the thread notification routines are ‘stuck’ trying to acquire
> the process lock I see no way in which the following scenario can happen:
> The thread notify routine is called for a new thread for a process which I
> DON’T have in my process table. Also, when this nonexistent process
> terminates (my process routine notify is called) it’s still not present in
> the process table.
>
> This scenario is easily reproduced using 5 batch scripts which every 100
> ms create a new process and wait for its termination.
>
> Am I missing something or this behavior shouldn’t be happening if my code
> actually works as I describe it?
>
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
–
Bercea. G.
really notification of new process create called just when first thread in process created. in context of creator thread. so, when first thread in process created, always first called ProcessNotifyRoutine and then from same thread called ThreadNotifyRoutine. because win32 always create first thread in process in suspended state (for notify csrss before thread resume) - new threads will be not created in process, until process creator not returned from ProcessNotifyRoutine and ThreadNotifyRoutine notifications. as result we cannot got ThreadNotifyRoutine before we got ProcessNotifyRoutine for thread process. process exit notify called in context of last process thread after called thread exit notify for all threads in process. so even not need any synchronization between process and threads notify routines(this auto done by os). only synchronization for access process/threads DB. may be you mistake in realization of process table ?
@Maxim:
I also thought it wasn’t the best idea to call ZwQuerySystemInformation
with the lock held but temporarily it seemed like a good idea.
@Gabriel Bercea:
Thanks for the suggestion, I don’t know why the thought of deferring
proc/thread notifications to a worker thread hasn’t occurred to me, this
seems like the cleanest and most efficient method possible.
The final result will be a boot driver but it is a lot easier to test by
stopping and re-starting the driver rather then rebooting the system each
time.
@harald24:
I re-read the documentation a few times before posting the question to make
sure this couldn’t happen (I even had an NT_ASSERT in the code if I
encounter that case and that’s how I realized it was happening). Well that
is what I’ll look into: synchronization issues and process/thread table
management (I’m using RtlAvlTable* functions for the first time in a
project but I’m fairly confident I’m using them correctly).
Thanks everyone for your suggestions, I appreciate your help and when I’ll
find the bug in my code I’ll let you know.
On 22 September 2015 at 15:38, wrote:
> really notification of new process create called just when first thread in
> process created. in context of creator thread. so, when first thread in
> process created, always first called ProcessNotifyRoutine and then from
> same thread called ThreadNotifyRoutine. because win32 always create first
> thread in process in suspended state (for notify csrss before thread
> resume) - new threads will be not created in process, until process creator
> not returned from ProcessNotifyRoutine and ThreadNotifyRoutine
> notifications. as result we cannot got ThreadNotifyRoutine before we got
> ProcessNotifyRoutine for thread process. process exit notify called in
> context of last process thread after called thread exit notify for all
> threads in process. so even not need any synchronization between process
> and threads notify routines(this auto done by os). only synchronization for
> access process/threads DB. may be you mistake in realization of process
> table ?
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
>I’m using RtlAvlTable*
why not use LIST_ENTRY list for process and threads list, like this do os by self ?
Because when a new thread is created/terminated and I’m looking for its
process in the table I prefer O(logn) search time rather than a O(n).
On 22 September 2015 at 21:29, wrote:
> >I’m using RtlAvlTable*
> why not use LIST_ENTRY list for process and threads list, like this do os
> by self ?
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>
> OSR is HIRING!! See http://www.osr.com/careers
>
> For our schedule of WDF, WDM, debugging and other seminars visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
After some more debugging I realized that this behavior happens only for
the threads which belong to the process which starts the driver service
(sc.exe), i.e. I’m using ‘sc start MyDriver’.
It was normal for my driver not to get a process creation notification for
sc because it was started before my driver was loaded, but it wasn’t normal
for the sc process not to be reported by ZwQuerySystemInformation.
Of course, as you may guess the API works properly and after debugging my
code I found the problem: when I was iterating over the
SYSTEM_PROCESS_INFORMATION entries retrieved by the API I was exiting the
loop prematurely (without processing the last entry). Everything works fine
now.
On 22 September 2015 at 21:37, Gurzou Alexandru
wrote:
> Because when a new thread is created/terminated and I’m looking for its
> process in the table I prefer O(logn) search time rather than a O(n).
>
> On 22 September 2015 at 21:29, wrote:
>
>> >I’m using RtlAvlTable*
>> why not use LIST_ENTRY list for process and threads list, like this do os
>> by self ?
>>
>> —
>> NTDEV is sponsored by OSR
>>
>> Visit the list at: http://www.osronline.com/showlists.cfm?list=ntdev
>>
>> OSR is HIRING!! See http://www.osr.com/careers
>>
>> For our schedule of WDF, WDM, debugging and other seminars visit:
>> http://www.osr.com/seminars
>>
>> To unsubscribe, visit the List Server section of OSR Online at
>> http://www.osronline.com/page.cfm?name=ListServer
>>
>
>