Paging IO question

NTFSD
1

Hello,

I am developing a file system filter for Windows XP and later.

My task is to perform backup of files before they get written
(Copy-on-write). However, I am not sure wheter or not to ignore
IRP_PAGING_IO in my IRP_MJ_WRITE dispatch routine? My idea is to only making
backups of writes WITHOUT the IRP_PAGING_IO flag set. By doing this I
obviously miss paging io writes but I am thinking of hooking
PreAcquireForSectionSynchronization and perform a file backup there instead.

Am I sure not to miss file content changes with this solution? Or do I also
need to hook PreAcquireForModifiedPageWriter and/or handle paging io anyway?

Regars
Alexander

2

Windows Cache manager uses paging IO. So you should not ingnore it.

Dan

----- Original Message -----
From: “Alexander”
Newsgroups: ntfsd
To: “Windows File Systems Devs Interest List”
Sent: Monday, January 03, 2005 9:14 PM
Subject: [ntfsd] Paging IO question

> Hello,
>
> I am developing a file system filter for Windows XP and later.
>
> My task is to perform backup of files before they get written
> (Copy-on-write). However, I am not sure wheter or not to ignore
> IRP_PAGING_IO in my IRP_MJ_WRITE dispatch routine? My idea is to only
> making backups of writes WITHOUT the IRP_PAGING_IO flag set. By doing this
> I obviously miss paging io writes but I am thinking of hooking
> PreAcquireForSectionSynchronization and perform a file backup there
> instead.
>
> Am I sure not to miss file content changes with this solution? Or do I
> also need to hook PreAcquireForModifiedPageWriter and/or handle paging io
> anyway?
>
> Regars
> Alexander
>
>
> —
> Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> To unsubscribe send a blank email to xxxxx@lists.osr.com

3

Thanks for you reply.

But is it possible for me to detect when a file gets cached in the cache
manager like I can detect that a file gets memory mapped with
PreAcquireForSectionSynchronization?

I have been told that I cannot perform fileIO during a paging io write,
(even if TopLevelIrp = NULL) is this true? If yes have should I make a copy
of a file with the IRP_PAGING_IO flag set?

Regards
Alexander
“Dan Partelly” skrev i meddelandet
news:xxxxx@ntfsd…
> Windows Cache manager uses paging IO. So you should not ingnore it.
>
> Dan
>
> ----- Original Message -----
> From: “Alexander”
> Newsgroups: ntfsd
> To: “Windows File Systems Devs Interest List”
> Sent: Monday, January 03, 2005 9:14 PM
> Subject: [ntfsd] Paging IO question
>
>
>> Hello,
>>
>> I am developing a file system filter for Windows XP and later.
>>
>> My task is to perform backup of files before they get written
>> (Copy-on-write). However, I am not sure wheter or not to ignore
>> IRP_PAGING_IO in my IRP_MJ_WRITE dispatch routine? My idea is to only
>> making backups of writes WITHOUT the IRP_PAGING_IO flag set. By doing
>> this I obviously miss paging io writes but I am thinking of hooking
>> PreAcquireForSectionSynchronization and perform a file backup there
>> instead.
>>
>> Am I sure not to miss file content changes with this solution? Or do I
>> also need to hook PreAcquireForModifiedPageWriter and/or handle paging io
>> anyway?
>>
>> Regars
>> Alexander
>>
>>
>> —
>> Questions? First check the IFS FAQ at
>> https://www.osronline.com/article.cfm?id=17
>>
>> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
>> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>

4

Backup on first CREATE with WRITE bit in DesiredAccess.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Alexander”
Newsgroups: ntfsd
To: “Windows File Systems Devs Interest List”
Sent: Monday, January 03, 2005 10:14 PM
Subject: [ntfsd] Paging IO question

> Hello,
>
> I am developing a file system filter for Windows XP and later.
>
> My task is to perform backup of files before they get written
> (Copy-on-write). However, I am not sure wheter or not to ignore
> IRP_PAGING_IO in my IRP_MJ_WRITE dispatch routine? My idea is to only making
> backups of writes WITHOUT the IRP_PAGING_IO flag set. By doing this I
> obviously miss paging io writes but I am thinking of hooking
> PreAcquireForSectionSynchronization and perform a file backup there instead.
>
> Am I sure not to miss file content changes with this solution? Or do I also
> need to hook PreAcquireForModifiedPageWriter and/or handle paging io anyway?
>
> Regars
> Alexander
>
>
>
> —
> Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

5

Once more: make a decision on CREATE with WRITE set in DesiredAccess, and
forget forever all these monstrosities with paging IO.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Alexander”
Newsgroups: ntfsd
To: “Windows File Systems Devs Interest List”
Sent: Monday, January 03, 2005 11:50 PM
Subject: Re:[ntfsd] Paging IO question

> Thanks for you reply.
>
> But is it possible for me to detect when a file gets cached in the cache
> manager like I can detect that a file gets memory mapped with
> PreAcquireForSectionSynchronization?
>
> I have been told that I cannot perform fileIO during a paging io write,
> (even if TopLevelIrp = NULL) is this true? If yes have should I make a copy
> of a file with the IRP_PAGING_IO flag set?
>
> Regards
> Alexander
> “Dan Partelly” skrev i meddelandet
> news:xxxxx@ntfsd…
> > Windows Cache manager uses paging IO. So you should not ingnore it.
> >
> > Dan
> >
> > ----- Original Message -----
> > From: “Alexander”
> > Newsgroups: ntfsd
> > To: “Windows File Systems Devs Interest List”
> > Sent: Monday, January 03, 2005 9:14 PM
> > Subject: [ntfsd] Paging IO question
> >
> >
> >> Hello,
> >>
> >> I am developing a file system filter for Windows XP and later.
> >>
> >> My task is to perform backup of files before they get written
> >> (Copy-on-write). However, I am not sure wheter or not to ignore
> >> IRP_PAGING_IO in my IRP_MJ_WRITE dispatch routine? My idea is to only
> >> making backups of writes WITHOUT the IRP_PAGING_IO flag set. By doing
> >> this I obviously miss paging io writes but I am thinking of hooking
> >> PreAcquireForSectionSynchronization and perform a file backup there
> >> instead.
> >>
> >> Am I sure not to miss file content changes with this solution? Or do I
> >> also need to hook PreAcquireForModifiedPageWriter and/or handle paging io
> >> anyway?
> >>
> >> Regars
> >> Alexander
> >>
> >>
> >> —
> >> Questions? First check the IFS FAQ at
> >> https://www.osronline.com/article.cfm?id=17
> >>
> >> You are currently subscribed to ntfsd as: xxxxx@rdsor.ro
> >> To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
>
>
>
> —
> Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

6

The poster was asking for a copy on write semantics, not a copy on access.
There are many times where a file will be opened with write access but a
write will never occur, hence you have reduced the copy on write semantics
to copy on first access with write access semantics.

Your solution for queuing a request to copy the file during the
syncforsection processing will cover the case for memory mapped files,
though I would suggest to simply enqueue the copy request at this point and
then handle the write pathway based upon whether the copy is completed yet.

Pete

Peter Scott
Windows Filesystem and Device Driver Consulting
www.KernelDrivers.com
(303)546-0300

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-197489-
xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
Sent: Wednesday, January 05, 2005 5:59 AM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Paging IO question

Backup on first CREATE with WRITE bit in DesiredAccess.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Alexander”
> Newsgroups: ntfsd
> To: “Windows File Systems Devs Interest List”
> Sent: Monday, January 03, 2005 10:14 PM
> Subject: [ntfsd] Paging IO question
>
>
> > Hello,
> >
> > I am developing a file system filter for Windows XP and later.
> >
> > My task is to perform backup of files before they get written
> > (Copy-on-write). However, I am not sure wheter or not to ignore
> > IRP_PAGING_IO in my IRP_MJ_WRITE dispatch routine? My idea is to only
> making
> > backups of writes WITHOUT the IRP_PAGING_IO flag set. By doing this I
> > obviously miss paging io writes but I am thinking of hooking
> > PreAcquireForSectionSynchronization and perform a file backup there
> instead.
> >
> > Am I sure not to miss file content changes with this solution? Or do I
> also
> > need to hook PreAcquireForModifiedPageWriter and/or handle paging io
> anyway?
> >
> > Regars
> > Alexander
> >
> >
> >
> > —
> > Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
> >
> > You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
> —
> Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@kerneldrivers.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

7

I think that the copy is only done if the file is “not copied yet”.

So, the first CREATE will check for this version of the file in the store
and do the actual copy. Then subsequent CREATEs will just check for “not copied
yet”, get their “yes it was” and continue.

Also opening a file with WRITE access and not writing is a strange idea.
Also dealing with paging IO for such a stupid task is surely an overkill.

Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

----- Original Message -----
From: “Peter Scott”
To: “Windows File Systems Devs Interest List”
Sent: Wednesday, January 05, 2005 7:51 PM
Subject: RE: [ntfsd] Paging IO question

>
> The poster was asking for a copy on write semantics, not a copy on access.
> There are many times where a file will be opened with write access but a
> write will never occur, hence you have reduced the copy on write semantics
> to copy on first access with write access semantics.
>
> Your solution for queuing a request to copy the file during the
> syncforsection processing will cover the case for memory mapped files,
> though I would suggest to simply enqueue the copy request at this point and
> then handle the write pathway based upon whether the copy is completed yet.
>
> Pete
>
> Peter Scott
> Windows Filesystem and Device Driver Consulting
> www.KernelDrivers.com
> (303)546-0300
>
>
> > -----Original Message-----
> > From: xxxxx@lists.osr.com [mailto:bounce-197489-
> > xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
> > Sent: Wednesday, January 05, 2005 5:59 AM
> > To: Windows File Systems Devs Interest List
> > Subject: Re: [ntfsd] Paging IO question
> >
> > Backup on first CREATE with WRITE bit in DesiredAccess.
> >
> > Maxim Shatskih, Windows DDK MVP
> > StorageCraft Corporation
> > xxxxx@storagecraft.com
> > http://www.storagecraft.com
> >
> > ----- Original Message -----
> > From: “Alexander”
> > Newsgroups: ntfsd
> > To: “Windows File Systems Devs Interest List”
> > Sent: Monday, January 03, 2005 10:14 PM
> > Subject: [ntfsd] Paging IO question
> >
> >
> > > Hello,
> > >
> > > I am developing a file system filter for Windows XP and later.
> > >
> > > My task is to perform backup of files before they get written
> > > (Copy-on-write). However, I am not sure wheter or not to ignore
> > > IRP_PAGING_IO in my IRP_MJ_WRITE dispatch routine? My idea is to only
> > making
> > > backups of writes WITHOUT the IRP_PAGING_IO flag set. By doing this I
> > > obviously miss paging io writes but I am thinking of hooking
> > > PreAcquireForSectionSynchronization and perform a file backup there
> > instead.
> > >
> > > Am I sure not to miss file content changes with this solution? Or do I
> > also
> > > need to hook PreAcquireForModifiedPageWriter and/or handle paging io
> > anyway?
> > >
> > > Regars
> > > Alexander
> > >
> > >
> > >
> > > —
> > > Questions? First check the IFS FAQ at
> > https://www.osronline.com/article.cfm?id=17
> > >
> > > You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> > —
> > Questions? First check the IFS FAQ at
> > https://www.osronline.com/article.cfm?id=17
> >
> > You are currently subscribed to ntfsd as: xxxxx@kerneldrivers.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
> —
> Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com

8

> Also opening a file with WRITE access and not writing is a strange idea.

Yes, but not impossible and not forbidden. I’ve seen many cases
when application opens the file for full access and then queries
attributes only.

L.

9

Yeah, like explorer. I have seen it do this a lot. Thus, using Max’s
solution, you would end up with entire directory trees which have taken copy
events when no actual writes have occurred to files in these nodes.

Also, I would beg to differ that a copy on write semantics filesystem is
‘stupid’. These are highly complex filesystems to get them right and
efficient.

Pete

Peter Scott
Windows Filesystem and Device Driver Consulting
www.KernelDrivers.com
(303)546-0300

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-197591-
xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Thursday, January 06, 2005 3:24 AM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Paging IO question

> Also opening a file with WRITE access and not writing is a strange
idea.

Yes, but not impossible and not forbidden. I’ve seen many cases
when application opens the file for full access and then queries
attributes only.

L.

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@kerneldrivers.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

10

If you chose a true logging model for a file system, COW is not required;
just set some anchor points.

The solution to the original problem is to filter the open and the close
operations. On close, compare the results of the file to the last results
(the copied version). If there are changes, make the backup.

I worked on a file system filter driver about 8 years ago that did just what
is being described. On close, if the file contents were modified, save the
diffs. Reconstruction the original file (versions) is quite easy).

Jamey

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Peter Scott
Sent: Thursday, January 06, 2005 10:27 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] Paging IO question

Yeah, like explorer. I have seen it do this a lot. Thus, using Max’s
solution, you would end up with entire directory trees which have taken copy
events when no actual writes have occurred to files in these nodes.

Also, I would beg to differ that a copy on write semantics filesystem is
‘stupid’. These are highly complex filesystems to get them right and
efficient.

Pete

Peter Scott
Windows Filesystem and Device Driver Consulting
www.KernelDrivers.com
(303)546-0300

-----Original Message-----
From: xxxxx@lists.osr.com [mailto:bounce-197591-
xxxxx@lists.osr.com] On Behalf Of Ladislav Zezula
Sent: Thursday, January 06, 2005 3:24 AM
To: Windows File Systems Devs Interest List
Subject: Re: [ntfsd] Paging IO question

> Also opening a file with WRITE access and not writing is a strange
idea.

Yes, but not impossible and not forbidden. I’ve seen many cases
when application opens the file for full access and then queries
attributes only.

L.

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@kerneldrivers.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@tfb.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

11

If I perform a copy on syncforsection (mem mapped files) and on IRP_MJ_WRITE
(not paging io) do I miss to make backups? If yes, are there other
notifications that I should look for?

I do not understand your proposed solution for enqueing the copy request
while in syncforsection processing. Do you mean that I should start the file
copy on syncforsection, for example in a work item, and if I see a
IRP_MJ_WRITE on the file I should pend the IRP until my copy is completed?

“Peter Scott” skrev i meddelandet
news:xxxxx@ntfsd…
>
> The poster was asking for a copy on write semantics, not a copy on access.
> There are many times where a file will be opened with write access but a
> write will never occur, hence you have reduced the copy on write semantics
> to copy on first access with write access semantics.
>
> Your solution for queuing a request to copy the file during the
> syncforsection processing will cover the case for memory mapped files,
> though I would suggest to simply enqueue the copy request at this point
> and
> then handle the write pathway based upon whether the copy is completed
> yet.
>
> Pete
>
> Peter Scott
> Windows Filesystem and Device Driver Consulting
> www.KernelDrivers.com
> (303)546-0300
>
>
>> -----Original Message-----
>> From: xxxxx@lists.osr.com [mailto:bounce-197489-
>> xxxxx@lists.osr.com] On Behalf Of Maxim S. Shatskih
>> Sent: Wednesday, January 05, 2005 5:59 AM
>> To: Windows File Systems Devs Interest List
>> Subject: Re: [ntfsd] Paging IO question
>>
>> Backup on first CREATE with WRITE bit in DesiredAccess.
>>
>> Maxim Shatskih, Windows DDK MVP
>> StorageCraft Corporation
>> xxxxx@storagecraft.com
>> http://www.storagecraft.com
>>
>> ----- Original Message -----
>> From: “Alexander”
>> Newsgroups: ntfsd
>> To: “Windows File Systems Devs Interest List”
>> Sent: Monday, January 03, 2005 10:14 PM
>> Subject: [ntfsd] Paging IO question
>>
>>
>> > Hello,
>> >
>> > I am developing a file system filter for Windows XP and later.
>> >
>> > My task is to perform backup of files before they get written
>> > (Copy-on-write). However, I am not sure wheter or not to ignore
>> > IRP_PAGING_IO in my IRP_MJ_WRITE dispatch routine? My idea is to only
>> making
>> > backups of writes WITHOUT the IRP_PAGING_IO flag set. By doing this I
>> > obviously miss paging io writes but I am thinking of hooking
>> > PreAcquireForSectionSynchronization and perform a file backup there
>> instead.
>> >
>> > Am I sure not to miss file content changes with this solution? Or do I
>> also
>> > need to hook PreAcquireForModifiedPageWriter and/or handle paging io
>> anyway?
>> >
>> > Regars
>> > Alexander
>> >
>> >
>> >
>> > —
>> > Questions? First check the IFS FAQ at
>> https://www.osronline.com/article.cfm?id=17
>> >
>> > You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
>> > To unsubscribe send a blank email to xxxxx@lists.osr.com
>>
>>
>> —
>> Questions? First check the IFS FAQ at
>> https://www.osronline.com/article.cfm?id=17
>>
>> You are currently subscribed to ntfsd as: xxxxx@kerneldrivers.com
>> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>