>>a) Is it a good idea to persist the CBT metadata (essentially a bitmap)
onto one of the volumes ?
[AB] Yes, where else will you keep it?
>b) Is the OS volume a good choice for the same ? Or the information
should be persisted on the volume to which it is relevant ?
[AB] Depends on product expectations. Both have their pros and cons. If
say, you plan to support volume migration from one computer to another,
like say a removable HDD, then you have to migrate the CBT info with the
volume. Other such examples are, a multiboot windows scenarion, where the
same fixed HDD is used by two versions of windows (multiboot). I say, it is
a degisn decision because it can become complicated if the volume in
question is attached to a node which doesnt have the softeware installed,
and those IOs then would go unaccounted for. Somethign to consider.
>c) Is ZwReadFile() and ZwWriteFile() the correct choice for reading and
writing the persistent information onto a file from Filter driver ?
[AB] In short, no. There are times when you will have to perform IO on
those volumes, when the FS is frozen, unavailable.
>d) Should I also handle other cases of volume being removed and
re-attached like disk where the volume resides can be made offline and
online, device remove and add etc ?
[AB] You have to, else you will see weird data loss cases infield.
>e) When the volume is re-attached, how do I determine that the volume was
removed safely ? One way could be to clear the persistent state information
when volume appears and then persist the details only when it is gracefully
removed. Are there better alternatives ?
[AB] Avoid lazy flush, sync the IO and the CBT flush. But that will have
performance inpacts (for every IO if you trigger another one for meta data
sync) so you will have to come up with a solution.
>f) How to ensure that no writes happened onto the volume behind the
curtains (for example, by attaching the volume to some other machine where
the CBT driver is not in the stack) ?
[AB]There are clever ways to find some of them, but not all cases are
possible. Refer to my ans for #b. You will have to clearly state the
limitations of the product, and also work so that data loss doesnt happen.
Failing the CBT and having to restart from Base is much better than having
a backup which is corrupted.
>g) When the volume is seen again, how to ensure that there are no writes
happen onto the volume before the persisted bitmap has been read ? One way
would be to start tracking the changes on the volume that appeared and then
merge the persistent bitmap once the volume on which the persistent changes
are stored is available for reads. Are there better alternatives ?
[AB] your driver has to be boot start and has to load and init itself
before the FS comes up. If you are a volume driver you can perform IO to
the volume without the FS APIS (check ans for #c), in that case there is no
question of merging and syncing, your tracking IO is always persisted when
you please.
>h) Is it not a catch-22 situation when the filter driver will be
filtering IO requests to the volume on which it will be persisting the
state information ?1
[AB] not if you are a volume ‘filter’ and not an fa ‘filter’
>i) What optimizations can be performed for cases where the volume removal
was not clean ? One optimization could be based on the file system bitmap
wen last backup was performed and the file system bitmap when the volume is
seen the next time. Are there other optimizations possible ?
[AB] yes, some, I suggest you get the other parts of the puzzle figured out
first, and then this will ans itself.
Cheers
Amit
On Mon, Nov 21, 2016 at 10:17 AM, wrote:
> I have a Change Block Tracking (CBT) driver for tracking changes on
> volumes. It is a KMDF Upper Filter driver. Right now, it does not save any
> persistent information onto any disk / volume and I am planning to
> implement the functionality of CBT across reboots.
> I have some questions around the same. My search shows limited discussion
> around this topic on OSR, so pardon me if you think these have been
> discussed earlier and kindly point me to appropriate threads is that is the
> case :
> a) Is it a good idea to persist the CBT metadata (essentially a bitmap)
> onto one of the volumes ?
>
> b) Is the OS volume a good choice for the same ? Or the information should
> be persisted on the volume to which it is relevant ?
>
> c) Is ZwReadFile() and ZwWriteFile() the correct choice for reading and
> writing the persistent information onto a file from Filter driver ?
>
> d) Should I also handle other cases of volume being removed and
> re-attached like disk where the volume resides can be made offline and
> online, device remove and add etc ?
>
> e) When the volume is re-attached, how do I determine that the volume was
> removed safely ? One way could be to clear the persistent state information
> when volume appears and then persist the details only when it is gracefully
> removed. Are there better alternatives ?
>
> f) How to ensure that no writes happened onto the volume behind the
> curtains (for example, by attaching the volume to some other machine where
> the CBT driver is not in the stack) ?
>
> g) When the volume is seen again, how to ensure that there are no writes
> happen onto the volume before the persisted bitmap has been read ? One way
> would be to start tracking the changes on the volume that appeared and then
> merge the persistent bitmap once the volume on which the persistent changes
> are stored is available for reads. Are there better alternatives ?
>
> h) Is it not a catch-22 situation when the filter driver will be filtering
> IO requests to the volume on which it will be persisting the state
> information ?1
>
> i) What optimizations can be performed for cases where the volume removal
> was not clean ? One optimization could be based on the file system bitmap
> wen last backup was performed and the file system bitmap when the volume is
> seen the next time. Are there other optimizations possible ?
>
> —
> NTDEV is sponsored by OSR
>
> Visit the list online at: http:> showlists.cfm?list=ntdev>
>
> MONTHLY seminars on crash dump analysis, WDF, Windows internals and
> software drivers!
> Details at http:
>
> To unsubscribe, visit the List Server section of OSR Online at <
> http://www.osronline.com/page.cfm?name=ListServer>
>
–
- ab</http:></http:>