test!

A most excellent suggestion for anyone who develops file systems and
file system filter drivers. I’m trying to put together some articles on
testing file systems and file system filter drivers, so anyone who can
contribute, please send me a note. I’ll take the suggestions,
questions, thoughts, issues, concerns and other comments I receive to
help shape some material we can use on the list for pointers on testing.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com


From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Brilly Wu (HangZhou)
Sent: Tuesday, July 06, 2004 1:39 AM
To: ntfsd redirect
Subject: [ntfsd] test!


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Hi Tony,

This is one more greate idea, just like the Filter doc, and faq doc from
osr. Many thanks !

Now how is your plan: in terms of by when you want infos from us; is it
going to be in a series or one shot etc. so that we could provide our
testing ideas etc ahead of time.

If you want the infos to be directed to you, it might be nice to have the
email address published though I’m sure anyone can find that out from
osronline site !. Or should we post the testing(s) we do here on this list.

-pro

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Tony Mason
Sent: Tuesday, July 06, 2004 6:37 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] test!
Importance: High

A most excellent suggestion for anyone who develops file systems and file
system filter drivers. I’m trying to put together some articles on testing
file systems and file system filter drivers, so anyone who can contribute,
please send me a note. I’ll take the suggestions, questions, thoughts,
issues, concerns and other comments I receive to help shape some material we
can use on the list for pointers on testing.

Regards,

Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com



From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com]
On Behalf Of Brilly Wu (HangZhou)
Sent: Tuesday, July 06, 2004 1:39 AM
To: ntfsd redirect
Subject: [ntfsd] test!


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Tony,

I have a couple of suggestions which would have helped me out eons ago
when I first started:

  1. Show how to properly use prefast. I was in awe when Neil Christensen
    showed it to me at Driver DevCon last year; I didn’t even know it
    existed. I think it really hit home because while he was showing prefast
    to me, prefast caught a bug in their new filter manager code which he
    was able to fix immediately and then check in. That sold me right then
    and there and its now part of my nant build environment for anything
    that gets checked in.

  2. Describe how to use Driver Verifier to fully stress test your driver.
    I have come across a couple of filter drivers now that haven’t done
    this, and fail if you even enable driver verifier. I don’t think people
    realize the value of the Driver Verifier Manager.

These are tools that exist in the OS and DDK, and are NOT being properly
utilized by a lot of new developers entering the field. I don’t blame
them for this; like me, most of them didn’t even know they exist! Your
work could include this and educate the masses on the benefits of such
great tools!


Regards,
Dana Epp
[Blog: http://silverstr.ufies.org/blog/]

Hi Prokash,

Thanks for the great questions. I expect that this information will be
disseminated both via a series of articles in The NT Insider as well as
a new document to supplement the FAQ.

Since I know that most of us work best with deadlines, I’ll use what you
can provide me with by the middle of July (15 July 2004). Once the
initial round is done, I’ll likely come back in 6 months and ask again.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com


From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Prokash Sinha
Sent: Tuesday, July 06, 2004 10:52 AM
To: ntfsd redirect
Subject: RE: [ntfsd] test!

Hi Tony,

This is one more greate idea, just like the Filter doc, and faq doc from
osr. Many thanks !

Now how is your plan: in terms of by when you want infos from us; is it
going to be in a series or one shot etc. so that we could provide our
testing ideas etc ahead of time.

If you want the infos to be directed to you, it might be nice to have
the email address published though I’m sure anyone can find that out
from osronline site !. Or should we post the testing(s) we do here on
this list.

-pro

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Tony Mason
Sent: Tuesday, July 06, 2004 6:37 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] test!
Importance: High

A most excellent suggestion for anyone who develops file systems
and file system filter drivers. I’m trying to put together some
articles on testing file systems and file system filter drivers, so
anyone who can contribute, please send me a note. I’ll take the
suggestions, questions, thoughts, issues, concerns and other comments I
receive to help shape some material we can use on the list for pointers
on testing.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com


From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Brilly Wu (HangZhou)
Sent: Tuesday, July 06, 2004 1:39 AM
To: ntfsd redirect
Subject: [ntfsd] test!


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to
xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@garlic.com
To unsubscribe send a blank email to
xxxxx@lists.osr.com


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Dana,

Thanks for the suggestions. I know we’ve written about Prefast before,
but it bears mentioning again. I know someone here at OSR (not me) is
writing a document on verifier as well.

Your comments also suggests a tutorial document - something like “So you
want to build a file system (or file system filter driver)”.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Dana Epp
Sent: Tuesday, July 06, 2004 11:05 AM
To: ntfsd redirect
Subject: Re: [ntfsd] test!

Tony,

I have a couple of suggestions which would have helped me out eons ago
when I first started:

  1. Show how to properly use prefast. I was in awe when Neil Christensen
    showed it to me at Driver DevCon last year; I didn’t even know it
    existed. I think it really hit home because while he was showing prefast
    to me, prefast caught a bug in their new filter manager code which he
    was able to fix immediately and then check in. That sold me right then
    and there and its now part of my nant build environment for anything
    that gets checked in.

  2. Describe how to use Driver Verifier to fully stress test your driver.

I have come across a couple of filter drivers now that haven’t done
this, and fail if you even enable driver verifier. I don’t think people
realize the value of the Driver Verifier Manager.

These are tools that exist in the OS and DDK, and are NOT being properly
utilized by a lot of new developers entering the field. I don’t blame
them for this; like me, most of them didn’t even know they exist! Your
work could include this and educate the masses on the benefits of such
great tools!


Regards,
Dana Epp
[Blog: http://silverstr.ufies.org/blog/]


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

A month or two ago, probably after downloading the latest Hardware
Compatibility Test suite from the Microsoft website, I was asked to fill in
a questionaire (from the HCT developers, I guess) concerning the future
direction of the HCT. One of the questions they asked was whether it would
be helpful for third-party developers to have access to the source code of
the HCT. Yes, it would!

An open-source HCT would be an outstanding enhancement over the current
release. I’m referring in particular to the IFSTEST portion of the HCT,
which we run frequently as part of our filter driver test and development
process. At present, the logs produced by IFSTEST are rather cryptic, and
it’s often very hard to tell exactly what went wrong during a given step in
the test (or what to do about it). If we had the HCT/IFSTEST source code to
examine – or better, if we could build IFSTEST and run it in a debugger
while testing the filter drivers – it would be a vast improvement over the
present situation. Then we’d be able to see exactly what has failed (and
probably have a lot better insight into what was “supposed” to happen).
Many other benefits would follow from this – we could incorporate HCT (or
HCT-like) source code and functionality directly into our product test
plans, and so forth.

Not to mention other benefits (to Microsoft as well as we developers), such
as: users finding bugs in the test, questioning the need or validity of
certain tests, make suggestions for more or better tests. The list goes on
an on…

I hope Microsoft will do this “for the good of all mankind”. Open source –
what a concept!

  • Dale.

From: Tony Mason [mailto:xxxxx@osr.com]
Sent: Tuesday, July 06, 2004 6:37 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] test!
Importance: High

A most excellent suggestion for anyone who develops file systems and file
system filter drivers. I’m trying to put together some articles on testing
file systems and file system filter drivers, so anyone who can contribute,
please send me a note. I’ll take the suggestions, questions, thoughts,
issues, concerns and other comments I receive to help shape some material we
can use on the list for pointers on testing.

Regards,

Tony

Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com


From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com]
On Behalf Of Brilly Wu (HangZhou)
Sent: Tuesday, July 06, 2004 1:39 AM
To: ntfsd redirect
Subject: [ntfsd] test!


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@veritas.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Here is my intial thought, and I would soon (before 15th) elaborate along
this line —

  1. The scenairo, as I tried to post it the other day is “How best we can do
    a filter” that would control accessing a) volume that could possiblly be on
    multiple partitions b) folders c) a file/doc. And this has to be on a
    portable volume so that (if I’m correct) even if someone from a different
    domain want to import the volume(s) would only have the accesses given to an
    user. Filter might need help from other usr lvl component(may be a service),
    but to have a control at those level, it seems like the three main fields
    ( Disposition, Option, Access , flags … and others ) of the fileobject can
    be used. I’ve seen a half-page doc on osronline, but a fairly complete
    picture of how to correleate win32 priv. to the file-system level privilages
    would of immense help.

  2. How to, if any way(s), to control caching mechanism(s) from a filter ?.
    This has to do with media server planning. There are a wide variety of
    Adaptive algorithm(s) that can play a major role if we could use caching in
    a semi-customed way ( detail later ). Also there are other places on the
    storage stacks who takes their role in terms of caching.

Again may be the details are already laid out on some nt insiders, but as I
said I need to look at it. And finally, the tests, at least in my
experience(s) depends on how best these design issues and interaction
w/other componenets are understood.

So I will surely post some of the test scenairos before 15th, and hope to
have detail insight from you at the palo-alto seminar of yours…

-pro
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Tony Mason
Sent: Tuesday, July 06, 2004 8:12 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] test!

Hi Prokash,

Thanks for the great questions. I expect that this information will be
disseminated both via a series of articles in The NT Insider as well as a
new document to supplement the FAQ.

Since I know that most of us work best with deadlines, I’ll use what you
can provide me with by the middle of July (15 July 2004). Once the initial
round is done, I’ll likely come back in 6 months and ask again.

Regards,

Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com



From: xxxxx@lists.osr.com [mailto:xxxxx@lists.osr.com]
On Behalf Of Prokash Sinha
Sent: Tuesday, July 06, 2004 10:52 AM
To: ntfsd redirect
Subject: RE: [ntfsd] test!

Hi Tony,

This is one more greate idea, just like the Filter doc, and faq doc from
osr. Many thanks !

Now how is your plan: in terms of by when you want infos from us; is it
going to be in a series or one shot etc. so that we could provide our
testing ideas etc ahead of time.

If you want the infos to be directed to you, it might be nice to have the
email address published though I’m sure anyone can find that out from
osronline site !. Or should we post the testing(s) we do here on this list.

-pro

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Tony Mason
Sent: Tuesday, July 06, 2004 6:37 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] test!
Importance: High

A most excellent suggestion for anyone who develops file systems and
file system filter drivers. I’m trying to put together some articles on
testing file systems and file system filter drivers, so anyone who can
contribute, please send me a note. I’ll take the suggestions, questions,
thoughts, issues, concerns and other comments I receive to help shape some
material we can use on the list for pointers on testing.

Regards,

Tony
Tony Mason
Consulting Partner
OSR Open Systems Resources, Inc.
http://www.osr.com


From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Brilly Wu (HangZhou)
Sent: Tuesday, July 06, 2004 1:39 AM
To: ntfsd redirect
Subject: [ntfsd] test!


Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to
xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@osr.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17

You are currently subscribed to ntfsd as: xxxxx@garlic.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Hi, Tony,

This might be one suggestion to your “Test!” project:

I’ll write a filter that modifies file data when read/written
(e.g. compression filter).

The filter must intercept paging IOs only
(because this is the only way how to catch all reads/writes).

So this implies the scheme

  1. ReadFile
  2. NtReadFile
  3. I/O manager (cached read)
  4. Filter1 (cached read, pass through)
  5. Filter2 (cached read, pass through)
  6. FSD (calls some of the CC interfaces for reading)
  7. Cache manager (page fault)
  8. Memory manager (I/O paging read)
  9. Filter1 (noncached paging read)
  10. Filter2 (noncached paging read)
  11. FSD (converts to disk sectors)
  12. Lower disk devices

Let’s say I’m the “Filter2” and I’m intercepting noncached I/Os
only at point 10.

The question is: Did you ever seen a FSD or a redirector
for which this does not work ? E.g. as response to cached read,
it will not call the cache manager ?

I think that the file system must not call the cache manager
in the shown way, but does really exist such one ?

L.