Why does blocking the Paging I/O request lead to deadlock?

Hi Rod!

" Why is it critical? What are you protecting against? Are you trying to
set up some sort of trasactional section?
Is it a data recovery issue? You have given me a functional requirement,
what is the user requirment?"

Actually, since i am planning to store the encryption header at the end in
encrypted files, i thought that it would be a good idea to store it
temporarily in a log file, and when the Close request to the encrypted file
comes, i will write the header at the end. The header contains some
information that needs to be updated when specific requests come to that
encrypted file. And thus i have to update the header present in the log.
What if i am unable to do so?
The file may end up unrecoverable.
What should i say to my professor? This happens even in professional
products?

Thanks!

  • K. Dev.

And Don has given you a great information for a starting point. Allow
me to throw out a couple ideas, based almost entirely on a belief that
academics have very different focus than most people on this list, for
you to bounce around that fall under the category of things to consider
but not ask directly.

In my opinion, what Don is describing is more or less the basis of any
typical production journaling file system. As he points out, this is
not necessarily going to be the easiest thing. Personally, I think that
this sort of arrangement, especially considering that it is only part of
the project, is, minimally, a very ambitious goal for something to be
completed in the time frame of class project by someone who is not
expected to be an expert. While what your professor is tasking you with
is presumably not expected to be production quality or complete for
these reasons, nor is it a full file system, nevertheless it still seems
to me to be a pretty unreasonable exercise for the amount of time given.
Don, Rod, Slava or Dejan among others could give you a much better feel
for the time required for something like this than I, but if you accept
my contention, then I would wonder if your professor is also aware of
this. It’s quite possible that he or she is aware of this specific
chicken or the egg aspect, and that is why he or she chose it, and may
only be expecting some one to identify it as a specific issue. It also
quite possible that he or she does not know the internals of NT file
systems, as opposed to, say, the Linux equivalents.

In either case, I guess my thought would be to go to him or her, and as
diplomatically as possible lay out your argument as to what you think
the issues are, which seem to basically be summarized in the last e-mail
you sent to Slava. In my pocket, I would keep the time estimate,
factoring in that the estimate is for someone who has experience with
this stuff. My personal belief is that most students are not going to
get anywhere as close to understanding the issue as well as you have
earned and clearly demonstrated. While this thread was rather
fragmented and argumentative (not your fault), one thing that I think
everyone agreed on was that the documentation sucks, which pretty much
means that in addition to one book, this list, and those like it are the
only sources of information on the subject, and even then you’re going
to have to fight for it, as you commendably did. So, as I see it, by
doing this, you might get a better idea of what he or she is really
expecting to actually materialize, if that is something specific, as
well as the general brownie points that come from asking thoughtful,
intelligent questions. If the goal proves to be unrealistic for all
involved, if nothing else, you will have demonstrated that you
understood the fundamental issue thoroughly, which I think will both be
very unusual among your peers, as well very difficult to demonstrate by
any means other than direct conversation - just look how this thread
went. I have always found academics found of asking questions that they
think no one can answer completely, and normalize the results with
partial credit. That’s fine for a lot of things, but my concern would
be that this is a difficult one to grade granularly.

Good luck,

mm

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Don Burn
Sent: Thursday, August 02, 2007 11:08
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Re:Re:RE:Why does blocking the Paging I/O request
lead to deadlock?

Rod has given you the hint. Consider creating the log file allocating a

number of blocks to it. Then get the retrieval pointers for the actual
disk blocks for the file. Manage the log yourself writing to the disk
rather than the file system.

This is not anywhere near as simple as just writing a log, but it does
allow you to do the write first, and do the writes in the paging path.


Don Burn (MVP, Windows DDK)
Windows 2k/XP/2k3 Filesystem and Driver Consulting
Website: http://www.windrvr.com
Blog: http://msmvps.com/blogs/WinDrvr
Remove StopSpam to reply

“Kernel Developer” wrote in message
news:xxxxx@ntfsd…
>>>you should also take care of your log file. You should close it in
case
>>>of
volume lock or dismount. Look at metadatamanager sample in wdk.<<<

I have already planned to do this.
The whole situation is that there is no way to ensure that Write to my
log
file happens before the write to the actual file.

And for some reasons, the data present in the log is critical. For
example,
i store extra information in log file that i finally purge into the
original file when it is closed. And this information is critical.

My concern is that if i am not able to write to my log file and the
system
crashes. Even though it is a college project, i want it to look
professional.

1. What should i do?
2. Just say that a system crash had occurred and your files may be in
corrupted/ unrecoverable state?
3. How do you guys handle it in a commercial/ real life product?

Thanks!
-K. Dev.


NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: xxxxx@evitechnology.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Thanks Everybody!

Well, I see no point going to my professor and trying to explain her the
stuff…
Like most of the other professors ( which they are actually not atleast in
my college ), she also does not know anything about Windows Internals, leave
File System Filters/ Minifilters aside.

I think its better to do the stuff without all these complexities ( Log
file, etc. ) at first; and if the time permits try to play with them in the
second version…
All that i can do is “Keep the design Extensible…”. Right?
:slight_smile:

Thanks again!
-K. Dev.

>went. I have always found academics found of asking questions that they

think no one can answer completely, and normalize the results with
partial credit.

I agree completely.

“+1”, as it is said in Russian-speaking Internet.

If the task is not from the standard taskbook and not solved - the this is
usually OK for the professor - just show him all of the approaches you have
tried to solve, all partial results you have achieved, and all knowledge gains
you got during this work.


Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

> Like most of the other professors ( which they are actually not atleast in

my college ), she also does not know anything about Windows Internals, leave
File System Filters/ Minifilters aside.

Correct.

Just explain her all of the complexities of Windows filesystems in some good
looking document.


Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

> Just explain her all of the complexities of Windows filesystems in some good

looking document.

Well, if he explains *all* the complexities of Windows filesystems , it will be already not a doc but a book of reputation comparable to that of Nagar’s one. The “only” thing left is to write it…

Anton Bassov

Maxim:

Just out of curiosity, what does “+1” mean?

mm

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S.
Shatskih
Sent: Tuesday, August 07, 2007 16:26
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Re:Re:RE:Why does blocking the Paging I/O request
lead to deadlock?

went. I have always found academics found of asking questions that
they
think no one can answer completely, and normalize the results with
partial credit.

I agree completely.

“+1”, as it is said in Russian-speaking Internet.

If the task is not from the standard taskbook and not solved - the this
is
usually OK for the professor - just show him all of the approaches you
have
tried to solve, all partial results you have achieved, and all knowledge
gains
you got during this work.


Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com


NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: xxxxx@evitechnology.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

“+1” means - “I second this”, “I completely agree”.


Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

“Martin O’Brien” wrote in message
news:xxxxx@ntfsd…
Maxim:

Just out of curiosity, what does “+1” mean?

mm

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S.
Shatskih
Sent: Tuesday, August 07, 2007 16:26
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Re:Re:RE:Why does blocking the Paging I/O request
lead to deadlock?

>went. I have always found academics found of asking questions that
they
>think no one can answer completely, and normalize the results with
>partial credit.

I agree completely.

“+1”, as it is said in Russian-speaking Internet.

If the task is not from the standard taskbook and not solved - the this
is
usually OK for the professor - just show him all of the approaches you
have
tried to solve, all partial results you have achieved, and all knowledge
gains
you got during this work.


Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com


NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: xxxxx@evitechnology.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

I like it. I’m also glad I asked, as my guess was something to due with
partial credit, which, retrospectively, not a very good one.

Thanks,

mm

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S.
Shatskih
Sent: Wednesday, August 08, 2007 06:37
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Re:Re:RE:Why does blocking the Paging I/O request
lead to deadlock?

“+1” means - “I second this”, “I completely agree”.


Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com

“Martin O’Brien” wrote in message
news:xxxxx@ntfsd…
Maxim:

Just out of curiosity, what does “+1” mean?

mm

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Maxim S.
Shatskih
Sent: Tuesday, August 07, 2007 16:26
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] Re:Re:RE:Why does blocking the Paging I/O request
lead to deadlock?

>went. I have always found academics found of asking questions that
they
>think no one can answer completely, and normalize the results with
>partial credit.

I agree completely.

“+1”, as it is said in Russian-speaking Internet.

If the task is not from the standard taskbook and not solved - the this
is
usually OK for the professor - just show him all of the approaches you
have
tried to solve, all partial results you have achieved, and all knowledge
gains
you got during this work.


Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com


NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: xxxxx@evitechnology.com
To unsubscribe send a blank email to xxxxx@lists.osr.com


NTFSD is sponsored by OSR

For our schedule debugging and file system seminars
(including our new fs mini-filter seminar) visit:
http://www.osr.com/seminars

You are currently subscribed to ntfsd as: xxxxx@evitechnology.com
To unsubscribe send a blank email to xxxxx@lists.osr.com