Virtual memory manager threads

NTDEV
1

Hello,

I’m writing a FS filter that hooks to FS device,
and sends a flush IRP to the device.

During the flush, I need to block all new writes except for those who
came as part of the flush process (identified by thread ID).

My problem is that with stress during the flush (many writes), it appears
that worker threads also participate in the flush process, so if I block
them - I block the whole flush, until timeout is reached.

Is there a way to tell which threads belong to the flush process, besides
my own thread ?

Regards,
Dror.

2

Do you want a good ideea ? Never Ever block threads created by kernel just
like that. In this case, threads created by Cc manager can take an active
part in flushing. There is no real way to take them appart from other
threads in system… And apart that, the FS synchronizes itslef the flushing
operations, regardless they are on files , directories, or volumes, so what
is the problem ?

----- Original Message -----
From:
To: “NT Developers Interest List”
Sent: Thursday, September 12, 2002 11:52 AM
Subject: [ntdev] Virtual memory manager threads

> Hello,
>
> I’m writing a FS filter that hooks to FS device,
> and sends a flush IRP to the device.
>
> During the flush, I need to block all new writes except for those who
> came as part of the flush process (identified by thread ID).
>
> My problem is that with stress during the flush (many writes), it appears
> that worker threads also participate in the flush process, so if I block
> them - I block the whole flush, until timeout is reached.
>
> Is there a way to tell which threads belong to the flush process, besides
> my own thread ?
>
> Regards,
> Dror.
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to %%email.unsub%%
>

3

I need to synchronize the FS to a point in time, and for that I need to
block any new writes except for the flush.
Any idea ?

Regards,
Dror.

At 02:17 PM 9/12/2002 +0300, you wrote:
Do you want a good ideea ? Never Ever block threads created by kernel just
like that. In this case, threads created by Cc manager can take an active
part in flushing. There is no real way to take them appart from other
threads in system… And apart that, the FS synchronizes itslef the
flushing
operations, regardless they are on files , directories, or volumes, so
what
is the problem ?

----- Original Message -----
From:
To: “NT Developers Interest List”
Sent: Thursday, September 12, 2002 11:52 AM
Subject: [ntdev] Virtual memory manager threads

> Hello,
>
> I’m writing a FS filter that hooks to FS device,
> and sends a flush IRP to the device.
>
> During the flush, I need to block all new writes except for those who
> came as part of the flush process (identified by thread ID).
>
> My problem is that with stress during the flush (many writes), it appears
> that worker threads also participate in the flush process, so if I block
> them - I block the whole flush, until timeout is reached.
>
> Is there a way to tell which threads belong to the flush process, besides
> my own thread ?
>
> Regards,
> Dror.
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to %%email.unsub%%
>

—
You are currently subscribed to ntdev as: xxxxx@e-market.co.il
To unsubscribe send a blank email to %%email.unsub%%

4

The answer is, you shouldn’t block paging i/o - you will quickly end up
blocking all the threads, and counter Mm’s attempts to maintain fair
memory availability. Apart from the simple fact of course, that you will
hang the system. I’d also not block any other i/o where the TopLevelIrp
is non-NULL (apart from the paging i/o rule) - indicating that i/o was
not a user top-level write.
Ravi

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@e-market.co.il [mailto:xxxxx@e-market.co.il]
Sent: Thursday, September 12, 2002 8:16 AM
To: NT Developers Interest List
Subject: [ntdev] Re: Virtual memory manager threads

I need to synchronize the FS to a point in time, and for that I need to
block any new writes except for the flush. Any idea ?

Regards,
Dror.

At 02:17 PM 9/12/2002 +0300, you wrote:
Do you want a good ideea ? Never Ever block threads created by kernel
just like that. In this case, threads created by Cc manager can take an
active part in flushing. There is no real way to take them appart from
other threads in system… And apart that, the FS synchronizes itslef the
flushing operations, regardless they are on files , directories, or
volumes, so what is the problem ?

----- Original Message -----
From:
To: “NT Developers Interest List”
Sent: Thursday, September 12, 2002 11:52 AM
Subject: [ntdev] Virtual memory manager threads

> Hello,
>
> I’m writing a FS filter that hooks to FS device,
> and sends a flush IRP to the device.
>
> During the flush, I need to block all new writes except for those who
> came as part of the flush process (identified by thread ID).
>
> My problem is that with stress during the flush (many writes), it
> appears that worker threads also participate in the flush process, so
> if I block them - I block the whole flush, until timeout is reached.
>
> Is there a way to tell which threads belong to the flush process,
> besides my own thread ?
>
> Regards,
> Dror.
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@rdsor.ro To
> unsubscribe send a blank email to %%email.unsub%%
>

—
You are currently subscribed to ntdev as: xxxxx@e-market.co.il To
unsubscribe send a blank email to %%email.unsub%%

—
You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to %%email.unsub%%

5

wrote in message news:xxxxx@ntdev…
>
>
> I need to synchronize the FS to a point in time, and for that I need to
> block any new writes except for the flush.
> Any idea ?
>
> Regards,
> Dror.
>

The best idea would be to take this topic to the NTFSD list, where people
who specialize in file systems will answer you.

My second best idea would be to use the native Volume Snapshot capabilities
built into Windows, starting with XP and .NET 2003.

Peter
OSR

6

You don’t. As a matter of fact, you should let all writes with IRP_NOCACHE
flag go through.

Think about the caller of the flush, he just want to make sure the all the
data he worte flush to disk. You don’t have to worry about if someone else
also write to the same area because this is not your problem. It is the
application’s problem. If someone want data to disk to be consistent to what
he wrote, he should use byte lock.

Bi

-----Original Message-----
From: xxxxx@e-market.co.il [mailto:xxxxx@e-market.co.il]
Sent: Thursday, September 12, 2002 8:16 AM
To: NT Developers Interest List
Subject: [ntdev] Re: Virtual memory manager threads

I need to synchronize the FS to a point in time, and for that I need to
block any new writes except for the flush.
Any idea ?

Regards,
Dror.

At 02:17 PM 9/12/2002 +0300, you wrote:
Do you want a good ideea ? Never Ever block threads created by kernel just
like that. In this case, threads created by Cc manager can take an active
part in flushing. There is no real way to take them appart from other
threads in system… And apart that, the FS synchronizes itslef the
flushing
operations, regardless they are on files , directories, or volumes, so
what
is the problem ?

----- Original Message -----
From:
To: “NT Developers Interest List”
Sent: Thursday, September 12, 2002 11:52 AM
Subject: [ntdev] Virtual memory manager threads

> Hello,
>
> I’m writing a FS filter that hooks to FS device,
> and sends a flush IRP to the device.
>
> During the flush, I need to block all new writes except for those who
> came as part of the flush process (identified by thread ID).
>
> My problem is that with stress during the flush (many writes), it appears
> that worker threads also participate in the flush process, so if I block
> them - I block the whole flush, until timeout is reached.
>
> Is there a way to tell which threads belong to the flush process, besides
> my own thread ?
>
> Regards,
> Dror.
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to %%email.unsub%%
>

—
You are currently subscribed to ntdev as: xxxxx@e-market.co.il
To unsubscribe send a blank email to %%email.unsub%%

—
You are currently subscribed to ntdev as: xxxxx@appstream.com
To unsubscribe send a blank email to %%email.unsub%%

7

If you acquire mutex for vcb, will you get PIT copy for the volume? I mean
flush the volume and try to acquire the mutex, once you get hold of this,
you can be sure that no I/O activities are going to happen on this volume.
You get your snapshot.
correct me, if this won’t work.

thanks
pashupati

-----Original Message-----
From: Ravisankar Pudipeddi [mailto:xxxxx@windows.microsoft.com]
Sent: Thursday, September 12, 2002 9:35 AM
To: NT Developers Interest List
Subject: [ntdev] Re: Virtual memory manager threads

The answer is, you shouldn’t block paging i/o - you will quickly end up
blocking all the threads, and counter Mm’s attempts to maintain fair
memory availability. Apart from the simple fact of course, that you will
hang the system. I’d also not block any other i/o where the TopLevelIrp
is non-NULL (apart from the paging i/o rule) - indicating that i/o was
not a user top-level write.
Ravi

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@e-market.co.il [mailto:xxxxx@e-market.co.il]
Sent: Thursday, September 12, 2002 8:16 AM
To: NT Developers Interest List
Subject: [ntdev] Re: Virtual memory manager threads

I need to synchronize the FS to a point in time, and for that I need to
block any new writes except for the flush. Any idea ?

Regards,
Dror.

At 02:17 PM 9/12/2002 +0300, you wrote:
Do you want a good ideea ? Never Ever block threads created by kernel
just like that. In this case, threads created by Cc manager can take an
active part in flushing. There is no real way to take them appart from
other threads in system… And apart that, the FS synchronizes itslef the
flushing operations, regardless they are on files , directories, or
volumes, so what is the problem ?

----- Original Message -----
From:
To: “NT Developers Interest List”
Sent: Thursday, September 12, 2002 11:52 AM
Subject: [ntdev] Virtual memory manager threads

> Hello,
>
> I’m writing a FS filter that hooks to FS device,
> and sends a flush IRP to the device.
>
> During the flush, I need to block all new writes except for those who
> came as part of the flush process (identified by thread ID).
>
> My problem is that with stress during the flush (many writes), it
> appears that worker threads also participate in the flush process, so
> if I block them - I block the whole flush, until timeout is reached.
>
> Is there a way to tell which threads belong to the flush process,
> besides my own thread ?
>
> Regards,
> Dror.
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@rdsor.ro To
> unsubscribe send a blank email to %%email.unsub%%
>

—
You are currently subscribed to ntdev as: xxxxx@e-market.co.il To
unsubscribe send a blank email to %%email.unsub%%

—
You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to %%email.unsub%%

—
You are currently subscribed to ntdev as: xxxxx@Legato.COM
To unsubscribe send a blank email to %%email.unsub%%

8

No you won’t.Among one of the many things wrong with a naive approach
like that is that you don’t get a semantically consistent filesystem
snapshot.
And the VCB is a private filesystem structure - you shouldn’t be peeking
into it, and acquiring its locks. You would deadloc the system prety
fast, and your driver will not be compatible version to version etc.

MS has built-in snapshot support for Windows XP which includes a volume
snapshot writer, which gives you a consistent volume snapshot.
Please use that.
Ravi
-----Original Message-----
From: Pashupati Kumar [mailto:xxxxx@legato.com]
Sent: Sunday, September 15, 2002 9:31 PM
To: NT Developers Interest List
Subject: [ntdev] Re: Virtual memory manager threads

If you acquire mutex for vcb, will you get PIT copy for the volume? I
mean flush the volume and try to acquire the mutex, once you get hold of
this, you can be sure that no I/O activities are going to happen on this
volume. You get your snapshot.
correct me, if this won’t work.
thanks
pashupati

-----Original Message-----
From: Ravisankar Pudipeddi [mailto:xxxxx@windows.microsoft.com]
Sent: Thursday, September 12, 2002 9:35 AM
To: NT Developers Interest List
Subject: [ntdev] Re: Virtual memory manager threads

The answer is, you shouldn’t block paging i/o - you will quickly end up
blocking all the threads, and counter Mm’s attempts to maintain fair
memory availability. Apart from the simple fact of course, that you will

hang the system. I’d also not block any other i/o where the TopLevelIrp
is non-NULL (apart from the paging i/o rule) - indicating that i/o was
not a user top-level write.
Ravi

This posting is provided “AS IS” with no warranties, and confers no
rights.

-----Original Message-----
From: xxxxx@e-market.co.il [mailto:xxxxx@e-market.co.il]
Sent: Thursday, September 12, 2002 8:16 AM
To: NT Developers Interest List
Subject: [ntdev] Re: Virtual memory manager threads

I need to synchronize the FS to a point in time, and for that I need to
block any new writes except for the flush. Any idea ?
Regards,
Dror.

At 02:17 PM 9/12/2002 +0300, you wrote:
Do you want a good ideea ? Never Ever block threads created by kernel
just like that. In this case, threads created by Cc manager can take an
active part in flushing. There is no real way to take them appart from
other threads in system… And apart that, the FS synchronizes itslef the

flushing operations, regardless they are on files , directories, or
volumes, so what is the problem ?
----- Original Message -----
From:
To: “NT Developers Interest List”
Sent: Thursday, September 12, 2002 11:52 AM
Subject: [ntdev] Virtual memory manager threads

> Hello,
>
> I’m writing a FS filter that hooks to FS device,
> and sends a flush IRP to the device.
>
> During the flush, I need to block all new writes except for those who
> came as part of the flush process (identified by thread ID).
>
> My problem is that with stress during the flush (many writes), it
> appears that worker threads also participate in the flush process, so
> if I block them - I block the whole flush, until timeout is reached.
>
> Is there a way to tell which threads belong to the flush process,
> besides my own thread ?
>
> Regards,
> Dror.
>
>
> —
> You are currently subscribed to ntdev as: xxxxx@rdsor.ro To
> unsubscribe send a blank email to %%email.unsub%%
>

—
You are currently subscribed to ntdev as: xxxxx@e-market.co.il To
unsubscribe send a blank email to %%email.unsub%%
—
You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to %%email.unsub%%

—
You are currently subscribed to ntdev as: xxxxx@Legato.COM
To unsubscribe send a blank email to %%email.unsub%%
—
You are currently subscribed to ntdev as: xxxxx@windows.microsoft.com
To unsubscribe send a blank email to %%email.unsub%%