Using the in-box Microsoft VSS provider, is there a way to query only what has changed between an ‘n’ and ‘n+1’ snapshot? When I do a vssadmin -q I see that all of the snaps my system has taken are marked ‘differential’. But my understanding of differential is that it is relative to a full, or base, snapshot. I do not see any such snapshot - every one is marked differential. So I’m not sure if I’m just misunderstanding what differential means in this context or what. Further, even if these are differential in terms of their behind-the-scenes allocation/implementation, is this metadata exposed to the user in any way? I’d like to perform remote backups to sync a series of local VSS snapshots, but obviously do not want to re-backup the entire volume on each subsequent snap if it hasn’t changed. I believe ‘incremental’ is what I’d ideally like to have achieve (though I can’t determine whether the ITB provider supports this backup type) but differential would at least be better than doing full backups on each snapshot.
You’ll need to develop a device driver to do this for generic volumes
And, having done this exact project here at OSR multiple times, I can tell you with some confidence that this is not simple. The IOCTLs that you need to know about to successfully interact with VolSnap are not all documented. Sigh…
I’ve built VSS providers in the past and I agree that it’s not fun. I was looking at this product below, which claims to provide all changes between two VSS snapshots without a driver, and wondering if perhaps I were missing something basic about VSS capabilities regarding incremental/differential snaps:
One thought I had was that you could potentially look at the USN stuff for the n and n+1 snaps and see what has changed that way. Of course, to my knowledge, I don’t think this would capture any direct volume write activity, only regular FS writes, right?
I’m needing a complete volume mirroring solution, so purely file-based is likely not feasible. But I am intrigued by that Hystax product and how they could achieve that sans driver, if in fact they do.
It maybe one of the most complex drivers to build outside of a full-blown FSD. There is a point where the snapshot transitions between read-only, read-write, and back to read-only. That is where you have to do your magic (GPT attributes are your friend). You really don’t need to use any of the undocumented VSS IOCTLs; except the data structure passed to FLUSH_AND_HOLD. I guess you could really fancy up your API using VSS get and set app data IOCTLs, but it is not required.
According to their website, they are making diffs between snapshots, not using real-time change tracking:
"Library reads deltas directly from VSS snapshots, no long API calls or data parsing
On another note: UrBackup now has a paid option to get CBT.
I will report anything I can find, yes. I am still a bit skeptical of their claims, in particular the ones about volume-level backups. I don’t see any possible way a user mode component can track raw volume writes. Even if they had reverse engineered the format of the MS storage area (the stuff inside of \System Volume Information) I don’t see how they could access that from user mode.
Caller can instantly get deltas in a specific file or on a disk with volume-based offsets
Clearly their marketing team got a bit overzealous on this. We certainly know that regardless of the implementation their product can’t provide a list of changed blocks “instantly”, at best it would be linearly scaled with the number of changes. Compared to a brute force, full snap-to-snap diff, sure, it should be much better. But this and the comments you have both made are why I’m viewing their claims through a skeptical lens.