Is there a way to detect if a NTFS volume has changed since the last shutdown in kernel mode? For example if the driver was taken to another system? I think of following solutions but please suggest if there is a better one:
- Calculate checksum of the volume and compare it. Very expensive.
- Calculate checksum of the MFT and compare it. Still expensive.
- Check access timestamp of some file (could be MFT) that is guaranteed to be accessed when a volume is acccesed. Is there any such file other than MFT?
Is there any better and low overhead solution? Thank you.
Have you looked at the USN journal ? It might be what you’re looking for…
See this article http://www.microsoft.com/msj/0999/journal/journal.aspx
What are you trying to do ?
Thanks,
Alex.
Thank you Mr. Alex. But is USN journal always available (enabled)? IIRC, it is optional and may be disabled by default. Let me to do more study on this. Thank you.
Alex, do you think this would work if someone had modified the NTFS volume under Mac OS X? Or if USN weren’t enabled?
On the other hand, perhaps computing a checksum of the MFT and bitmap might be sufficient (since the MFT would indicate if any files/directories were modified and the bitmap would indicate if any allocation changed, even if file attributes did not change.) It sure would be faster than computing a checksum of the entire volume. Honestly, I suspect the algorithm would need some fine tuning, since even booting off the volume will change its contents (so you might need to exclude portions of the MFT from your computation. The registry hives seem to be the first thing that comes to my mind.) Alternatively, it might be possible to just do the checksum during boot start-up.
Tony
OSR
Hope to see everyone September 19 for the next Developing File Systems for Windows seminar (http://www.osr.com/fsd.html)
-----Original Message-----
From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com] On Behalf Of Alex Carp
Sent: Monday, August 22, 2011 10:32 AM
To: ntfsd redirect
Subject: RE: [ntfsd] NTFS volume change
Have you looked at the USN journal ? It might be what you’re looking for…
See this article http://www.microsoft.com/msj/0999/journal/journal.aspx
What are you trying to do ?
Thanks,
Alex.
NTFSD is sponsored by OSR
For our schedule of debugging and file system seminars visit:
http://www.osr.com/seminars
To unsubscribe, visit the List Server section of OSR Online at http://www.osronline.com/page.cfm?name=ListServer
Mr. Tony, if USN is not enabled. Can I enabled it? And will it stay enabled when it is accessed under different environment? Also are the NTFS drivers of the alternate environment required to keep updating the USN? Thank you.
Hi Tony,
No, I don’t think it would work in those cases… That’s why I asked what
the OP was trying to do.
If I wanted to ensure that a volume hasn’t be modified for security reasons,
then I think that block level HMAC is pretty much the way to go. This should
detect bad sectors as well as any attempt to tamper with the volume. Block
level encryption might also validate this in a different way… HMAC might
allow mounting the volume read-only on a different system whereas encryption
will not allow even that without a key so again it depends on the
requirements.
An MFT hash might detect changes to the volume that the MFT knows about but
not necessarily changes to the user data, so does that mean that the “NTFS
volume has changed since the last shutdown” or not ?
So OP, please state your requirements.
Thanks,
Alex.
You can enable it. If you can run a driver you can enable it.
Please define different environment. For example does a disk editor count as
a different environment?
As to other NTFS drivers of different environment, frankly I don’t think
there are any requirements… I don’t think there are any that are actually
sanctioned by MS so I doubt there’s anyone to enforce requirements…
Thanks,
Alex.
My different environments are Linux, WinPE, Another system with Windows 7, Windows recovery dvd.
[AC] An MFT hash might detect changes to the volume that the MFT knows about but
not necessarily changes to the user data,
[NK] If a file is changed, won’t the “last modified time” saved in MFT be changed also?
[AC] so does that mean that the “NTFS volume has changed since the last shutdown” or not ?
[NK] Yes, the volume has changed.
Checksum/hash may have problems if the shutdown was not clean (a sudden power loss scenario). As the question is when to calculate the reference checksum?
It’s safe to assume that WinPE, Win7 and Windows recovery DVD will update
the USN journal properly. I don’t know about the Linux driver you have in
mind.
“If a file is changed, won’t the “last modified time” saved in MFT be
changed also?” -> it depends on the driver. Hopefully it will be changed.
But I was thinking about someone changing the file at block level, not
necessarily using the driver.
Since NTFS has built-in transactions and journaling it is possible that as
part of mounting the volume NTFS will undo some changes in order to bring
the volume back to a consistent state. I don’t think you can intercept these
operations at file system level.
If you have a block level filter driver you should see all IO to the volume
and you can update and validate checksums on the fly.
Thanks,
Alex.
In order for an MFT hash to fail to detect a change, it would require that the attributes of the file not change. For example, if one overwrite the data with comparable size data AND did not update the last modified time, then it would not be detected. That’s certainly a possibility, but suggests an intelligent adversary - in which case they could change anything and just recomputed the hash and write it wherever it is that you save it.
In the case of a power failure, you’d be forced to assume the volume had changed. If you wish to protect against this, you make the problem much more challenging. Logically, this is the same reason one has to run autochk on a volume (for example) and that guarantees the volume will change in any case (at least the dirty bit will be cleared.) Building a system that can detect this sort of failure is something I would approach by keeping a log (it is how NTFS avoids running autochk in most cases anyway.)
As Alex points out, the difficulty in answering your question is that it’s sufficiently vague that it isn’t clear what you’re trying to accomplish (e.g., what threats are you trying to protect against.) If someone is willing to use a disk level modification tool to change the disk, they’re quite capable of changing your computed checksum (assuming it’s stored on the volume somewhere) as well.
Perhaps it’s the file system developer in me, but I view the file system meta data to be the critical piece to protect here, not the user data. If you REALLY want to know about all the user data, you will need to track it directly
Tony
OSR
Hope to see everyone September 19 for the next Developing File Systems for Windows seminar (http://www.osr.com/fsd.html)