I’m trying to do some measurement of NTFS, and it would be a great help if I could programatically correlate read/write requests issued to a disk back to the file being modified. I’m interesting in windows versions XP and later.
I realize that in some cases this will be problematic, so I’d be happy with a partial solution, particularly one that is sound (even if not complete). This is for measurement, not production, so I can live with a dirty hack. I have a lot of latitude in being able to modify the system being measured with filter or other drivers.
I imagine that one possible approach would be to delay requests in a class driver while a user mode process scans the file table looking for the appropriate block range. This seems complicated and may take a while to get right though.
Does anyone have any ideas or can someone point me in a better direction?
What exactly is it that you’re trying to do? Coming up with a general
solution could be pretty damn tricky, but do you have control over the
application creating/writing the file that you want to measure? If so, you
could have the application create/extend the file, get the extents using
Mark’s article:
http://www.wd-3.com/archive/luserland.htm
And send the extents to the disk filter driver. Then you could let the app
start firing off the writes and you’d be able to recognize them in the
filter (sort of like what you proposed, just much more constrained).
-scott
–
Scott Noone
Consulting Associate
OSR Open Systems Resources, Inc.
http://www.osronline.com
wrote in message news:xxxxx@ntfsd…
> I’m trying to do some measurement of NTFS, and it would be a great help if
> I could programatically correlate read/write requests issued to a disk
> back to the file being modified. I’m interesting in windows versions XP
> and later.
>
> I realize that in some cases this will be problematic, so I’d be happy
> with a partial solution, particularly one that is sound (even if not
> complete). This is for measurement, not production, so I can live with a
> dirty hack. I have a lot of latitude in being able to modify the system
> being measured with filter or other drivers.
>
> I imagine that one possible approach would be to delay requests in a class
> driver while a user mode process scans the file table looking for the
> appropriate block range. This seems complicated and may take a while to
> get right though.
>
> Does anyone have any ideas or can someone point me in a better direction?
>
Thanks Scott,
I’m trying to gather data on how various workloads translate into
block-level requests. For example, I’d like to be able to answer questions
like: what percentage of block level requests are coming from writing out
the pagefile to disk, versus files in /user/dutch, versus files with
FILE_ATTRIBUTE_HIDDEN? Essentially, i’d like to be able to tag as many
block level requests as possible with the name of the file they are
modifying.
So you have a creative insight there, but modifying the application level is
one of the few things I cannot do, since I’m interested in system-wide
behavior.
I’ve had some success by recording the read/write buffer and the MDL
addresses for reads and writes at the minifilter level, and comparing these
to the system buffer and MDL in a storage class driver above the disk.
Often for the same I/O the same buffer is used at both levels. However,
it’s far from perfect. For many system related updates ($LOG and $MFT) I
don’t see any buffer provided at the minifilter level, which confuses me.
Also this seems pretty hard to get working for fragmented and sparse files,
though perhaps not impossible.
–Dutch
On Thu, Aug 5, 2010 at 9:05 AM, Scott Noone wrote:
> What exactly is it that you’re trying to do? Coming up with a general
> solution could be pretty damn tricky, but do you have control over the
> application creating/writing the file that you want to measure? If so, you
> could have the application create/extend the file, get the extents using
> Mark’s article:
>
> http://www.wd-3.com/archive/luserland.htm
>
> And send the extents to the disk filter driver. Then you could let the app
> start firing off the writes and you’d be able to recognize them in the
> filter (sort of like what you proposed, just much more constrained).
>
> -scott
>
> –
> Scott Noone
> Consulting Associate
> OSR Open Systems Resources, Inc.
> http://www.osronline.com
>
>
> wrote in message news:xxxxx@ntfsd…
>
> I’m trying to do some measurement of NTFS, and it would be a great help if
>> I could programatically correlate read/write requests issued to a disk back
>> to the file being modified. I’m interesting in windows versions XP and
>> later.
>>
>> I realize that in some cases this will be problematic, so I’d be happy
>> with a partial solution, particularly one that is sound (even if not
>> complete). This is for measurement, not production, so I can live with a
>> dirty hack. I have a lot of latitude in being able to modify the system
>> being measured with filter or other drivers.
>>
>> I imagine that one possible approach would be to delay requests in a class
>> driver while a user mode process scans the file table looking for the
>> appropriate block range. This seems complicated and may take a while to get
>> right though.
>>
>> Does anyone have any ideas or can someone point me in a better direction?
>>
>>
> —
> NTFSD is sponsored by OSR
>
> For our schedule of debugging and file system seminars
> (including our new fs mini-filter seminar) visit:
> http://www.osr.com/seminars
>
> To unsubscribe, visit the List Server section of OSR Online at
> http://www.osronline.com/page.cfm?name=ListServer
>
There are some Event Tracing for Windows (ETW) tracepoints that identify the file with which (many) block-level I/Os are associated.
It’s been, quite literally, many years since I’ve touched this… but you can probably fool a bit with TraceView and the system-provided trace points to get an idea of the data that’s available. THEN, you could write you own tracing program to collect this data and correlate it.
Like I said, I’ve actually DONE this for a research project, but it was a long time ago. Knowing a bit about file systems and file system filter drivers here, we generally try to avoid writing one whenever possible. And the ETW approach worked quite well for us.
Peter
OSR