When creating my own IRP_MJ_WRITE that writes to a previous created file on
the disk(I have the FO of the same) from inside my file system filter
driver, what whould I put in the ioStackLocation->DeviceObject filed? The
next device in the stack, that is the one to which my filter is attached,
or that retuened by IoGetRelatedDeviceObject(FO) ?
Also when I call IO call driver I think the same device name is needed,
right?
Do I need to set a completion routine for this IRP, the OSR article states
that the cases where a completion routine is required is not clear!!!
–
If you are creating and issuing your own IRPs, you definitely need to set a
completion routine. Otherwise, how will you know when to free the IRP?
The DeviceObject field in the next-lower I/O stack location is set by
IoCallDriver. You don’t need to set it.
– arlie
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of amitr0
Sent: Friday, March 24, 2006 12:45 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IRP_MJ_WRITE
When creating my own IRP_MJ_WRITE that writes to a previous created file on
the disk(I have the FO of the same) from inside my file system filter
driver, what whould I put in the ioStackLocation->DeviceObject filed? The
next device in the stack, that is the one to which my filter is attached, or
that retuened by IoGetRelatedDeviceObject(FO) ?
Also when I call IO call driver I think the same device name is needed,
right?
Do I need to set a completion routine for this IRP, the OSR article states
that the cases where a completion routine is required is not clear!!!
–
It is not always the case.
If you created IRP using IoBuildSynchronousFsdRequest you must not free the
IRP yourself - IoManager should do it for you. You may want to have
completion routine (for example to know when IRP was completed) in this case
but it is not mandatory.
Alexei.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Arlie Davis
Sent: Friday, March 24, 2006 9:51 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IRP_MJ_WRITE
If you are creating and issuing your own IRPs, you definitely need to set a
completion routine. Otherwise, how will you know when to free the IRP?
The DeviceObject field in the next-lower I/O stack location is set by
IoCallDriver. You don’t need to set it.
– arlie
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of amitr0
Sent: Friday, March 24, 2006 12:45 PM
To: Windows File Systems Devs Interest List
Subject: [ntfsd] IRP_MJ_WRITE
When creating my own IRP_MJ_WRITE that writes to a previous created file on
the disk(I have the FO of the same) from inside my file system filter
driver, what whould I put in the ioStackLocation->DeviceObject filed? The
next device in the stack, that is the one to which my filter is attached, or
that retuened by IoGetRelatedDeviceObject(FO) ?
Also when I call IO call driver I think the same device name is needed,
right?
Do I need to set a completion routine for this IRP, the OSR article states
that the cases where a completion routine is required is not clear!!!
–
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: xxxxx@vmware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com
Yes, of course. If you are using IoBuildSynchronousFsdRequest, you must NOT
call IoSetCompletionRoutine, because that would overwrite the completion
routine set by IoBSFR.
– arlie
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Alexei Jelvis
Sent: Friday, March 24, 2006 12:59 PM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IRP_MJ_WRITE
It is not always the case.
If you created IRP using IoBuildSynchronousFsdRequest you must not free the
IRP yourself - IoManager should do it for you. You may want to have
completion routine (for example to know when IRP was completed) in this case
but it is not mandatory.
Alexei.
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]On Behalf Of Arlie Davis
Sent: Friday, March 24, 2006 9:51 AM
To: Windows File Systems Devs Interest List
Subject: RE: [ntfsd] IRP_MJ_WRITE
If you are creating and issuing your own IRPs, you definitely need to set a
completion routine. Otherwise, how will you know when to free the IRP?
The DeviceObject field in the next-lower I/O stack location is set by
IoCallDriver. You don’t need to set it.
– arlie
AFAIK none of the IoBuildXxx routines set a completion routine in the IRP,
they just let the IRP get cleaned up as part of normal I/O completion.
Just double checked with the debugger, here’s an example IRP returned from
this call (note that “Completion” is NULL):
0: kd> !irp @eax
Irp is active with 1 stacks 2 is current (= 0x8184f8b4)
No Mdl Thread 81841020: Irp is completed.
cmd flg cl Device File Completion-Context
[1b, 0] 0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
-scott
–
Scott Noone
Software Engineer
OSR Open Systems Resources, Inc.
http://www.osronline.com
“Arlie Davis” wrote in message
news:xxxxx@ntfsd…
> Yes, of course. If you are using IoBuildSynchronousFsdRequest, you must
> NOT
> call IoSetCompletionRoutine, because that would overwrite the completion
> routine set by IoBSFR.
>
> – arlie
>
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com] On Behalf Of Alexei Jelvis
> Sent: Friday, March 24, 2006 12:59 PM
> To: Windows File Systems Devs Interest List
> Subject: RE: [ntfsd] IRP_MJ_WRITE
>
> It is not always the case.
> If you created IRP using IoBuildSynchronousFsdRequest you must not free
> the
> IRP yourself - IoManager should do it for you. You may want to have
> completion routine (for example to know when IRP was completed) in this
> case
> but it is not mandatory.
>
> Alexei.
>
> -----Original Message-----
> From: xxxxx@lists.osr.com
> [mailto:xxxxx@lists.osr.com]On Behalf Of Arlie Davis
> Sent: Friday, March 24, 2006 9:51 AM
> To: Windows File Systems Devs Interest List
> Subject: RE: [ntfsd] IRP_MJ_WRITE
>
>
> If you are creating and issuing your own IRPs, you definitely need to set
> a
> completion routine. Otherwise, how will you know when to free the IRP?
>
> The DeviceObject field in the next-lower I/O stack location is set by
> IoCallDriver. You don’t need to set it.
>
> – arlie
>
>
>
>
Achhh, silly me. Sorry.
– arlie
-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Scott Noone
Sent: Friday, March 24, 2006 2:44 PM
To: Windows File Systems Devs Interest List
Subject: Re:[ntfsd] IRP_MJ_WRITE
AFAIK none of the IoBuildXxx routines set a completion routine in the IRP,
they just let the IRP get cleaned up as part of normal I/O completion.
Just double checked with the debugger, here’s an example IRP returned from
this call (note that “Completion” is NULL):
0: kd> !irp @eax
Irp is active with 1 stacks 2 is current (= 0x8184f8b4) No Mdl Thread
81841020: Irp is completed.
cmd flg cl Device File Completion-Context
[1b, 0] 0 0 00000000 00000000 00000000-00000000
Args: 00000000 00000000 00000000 00000000
-scott
Thanks aeverybody for your suggestions.
Hi experts,
I was experimenting with writing files from inside FS filters. I had to roll
my own IRP_MJ_WRITE (if you remember my previous threads). For
experimentation I used the Filespy code. Instead of sending the Logs up to
user mode, I modified the code of the driver to send the logs to a kernel
mode thread which rolled out custom IRP_MJ_WRITES and wrote the data to a
file on the disk.
Here are the issues i face…
-
The system become too damn slow! To rectify the problem I consolidated
multiple log records, and then dumped the buffer in one go, through a single
IRP_MJ_WRITE. Still the system stalls.
-
Also, I used a simple counter architecture that kept increasing till a
threashold was reached, at that point the thread would wake up and get the
logs from the logrecords and dump them (after consolidating). What I find is
that the driver runs out of memory too soon, and no new log record
structures are allocated. The otehr alternative would be to check
IoStatus->Information, if it is zero, make the thread sleep fro some time
and retry.
Can someone please advice me as to how I can omplement this method properly?
Thanks in advance…
amitr0
Did I get this right? You are copying writes to cache onto disk?
“amitr0” wrote in message news:xxxxx@ntfsd…
Hi experts,
I was experimenting with writing files from inside FS filters. I had to roll
my own IRP_MJ_WRITE (if you remember my previous threads). For
experimentation I used the Filespy code. Instead of sending the Logs up to
user mode, I modified the code of the driver to send the logs to a kernel
mode thread which rolled out custom IRP_MJ_WRITES and wrote the data to a
file on the disk.
Here are the issues i face…
1. The system become too damn slow! To rectify the problem I consolidated
multiple log records, and then dumped the buffer in one go, through a single
IRP_MJ_WRITE. Still the system stalls.
2. Also, I used a simple counter architecture that kept increasing till a
threashold was reached, at that point the thread would wake up and get the
logs from the logrecords and dump them (after consolidating). What I find is
that the driver runs out of memory too soon, and no new log record
structures are allocated. The otehr alternative would be to check
IoStatus->Information, if it is zero, make the thread sleep fro some time
and retry.
Can someone please advice me as to how I can omplement this method properly?
Thanks in advance…
amitr0
Lyndon,
I didn’t properly understand waht you said. Can you explain a bit further.
Right. Consider IRP_MJ_WRITE where Irp->Flags does not have IRP_NOCACHE bit
set; so data is being written to file cache. Do you write a copy of this
data to disk (file) in your driver?
“amitr0” wrote in message news:xxxxx@ntfsd…
Lyndon,
I didn’t properly understand waht you said. Can you explain a bit further.
Look Lyndon,
I am not writing IRP logs to file on a one to one basis. If you have gone
through filespy you would know that it has a function called SpyGetLog(…)
which is used when the user mode app sends an IOCTL to the driver to get the
logs the driver has put into the various log record data structures.
All I do is make a seperate thread, which calls this function in kernel mode
itself, instead of the user mdoe app, and fetches the logs from the internal
datastructures. Then it pools these logs togetehr (to make the writing
operation more efficient) and send am write IRP down to the FS with the
buffer.
I don’t think, paging/caching comes into picture here. Or am I mistaken?
amitr0
Look here amitr0
I understood your previous post in this thread. I did not assert or ask
whether you write “IRP logs to file on a one to one basis”. You should
assume I am familar with filespy.
The obvious point is that if you attempt to log IRP access of file cache
(memory) to file (disk) then it should not surprise you to find that the
system performance is very poor.
Lyndon
“amitr0” wrote in message news:xxxxx@ntfsd…
Look Lyndon,
I am not writing IRP logs to file on a one to one basis. If you have gone
through filespy you would know that it has a function called SpyGetLog(…)
which is used when the user mode app sends an IOCTL to the driver to get the
logs the driver has put into the various log record data structures.
All I do is make a seperate thread, which calls this function in kernel mode
itself, instead of the user mdoe app, and fetches the logs from the internal
datastructures. Then it pools these logs togetehr (to make the writing
operation more efficient) and send am write IRP down to the FS with the
buffer.
I don’t think, paging/caching comes into picture here. Or am I mistaken?
amitr0
“The obvious point is that if you attempt to log IRP access of file cache
(memory) to file (disk) then it should not surprise you to find that the
system performance is very poor.”
This is not waht is clear to me. Filespy logs all IRPs right? Whether
Irp->Flags has caching bit on or off. So the records would go to it’s data
structures. Which is what I am accessing and writing out. I cannot
understand the issue, please forgive me for my ignorance, I am not a veteran
here.
Do you say the performance will be poor, because of the sheer volume of IRP
logs generated? Beg your pardon, if my repetative questions bothers you, but
I really cannot see things from where you see them. Please help if you can.
amitr0
Just think about the difference between memory speeds and disk speeds. It is
not rocket science 
“amitr0” wrote in message news:xxxxx@ntfsd…
“The obvious point is that if you attempt to log IRP access of file cache
(memory) to file (disk) then it should not surprise you to find that the
system performance is very poor.”
This is not waht is clear to me. Filespy logs all IRPs right? Whether
Irp->Flags has caching bit on or off. So the records would go to it’s data
structures. Which is what I am accessing and writing out. I cannot
understand the issue, please forgive me for my ignorance, I am not a veteran
here.
Do you say the performance will be poor, because of the sheer volume of IRP
logs generated? Beg your pardon, if my repetative questions bothers you, but
I really cannot see things from where you see them. Please help if you can.
amitr0
amitr0,
Curious, but how large are you growing the log files before attempting
to write them. Lyndon is correct as you know,
HD’s are very slow. If your doing a lot of little writes, that will
increase the performance hit the HD is causing by multiples.
Lyndon J. Clarke wrote:
Just think about the difference between memory speeds and disk speeds. It is
not rocket science
“amitr0” wrote in message news:xxxxx@ntfsd…
>“The obvious point is that if you attempt to log IRP access of file cache
>(memory) to file (disk) then it should not surprise you to find that the
>system performance is very poor.”
>
>This is not waht is clear to me. Filespy logs all IRPs right? Whether
>Irp->Flags has caching bit on or off. So the records would go to it’s data
>structures. Which is what I am accessing and writing out. I cannot
>understand the issue, please forgive me for my ignorance, I am not a veteran
>here.
>
>Do you say the performance will be poor, because of the sheer volume of IRP
>logs generated? Beg your pardon, if my repetative questions bothers you, but
>I really cannot see things from where you see them. Please help if you can.
>
>amitr0
>
>
>
>—
>Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: xxxxx@comcast.net
>To unsubscribe send a blank email to xxxxx@lists.osr.com
>
>
>
Matt, Lyndon,
Actually that is also an issue here. The threshold when I rally should start
to dump is yet undecided. Typically I write once the buffer size is close to
4 KBs. Is this wrong?
Don’t know the answer. IIRC, 4k is the size of a single page; the OS
writes multiple pages at a time to disk for performance reasons… Try
32, 64, 128, 256 writes. Start with the highest and then scale back if
the system can’t handle it.
Good luck and let me know what the best write size is that you find.
M
amitr0 wrote:
Matt, Lyndon,
Actually that is also an issue here. The threshold when I rally should
start to dump is yet undecided. Typically I write once the buffer size
is close to 4 KBs. Is this wrong?
— Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17 You are currently
subscribed to ntfsd as: unknown lmsubst tag argument: ‘’ To
unsubscribe send a blank email to xxxxx@lists.osr.com
Sure Matt, I will let you know.
Also I have a simple doubt regarding this…
Writing to a file from user land should be more expensive than writing to
one from our own IRP. Is this true? I mean, if implemented correctly, the
kernel mode way to write logs to a file should be a beter approach than
sending the logs to user land, and then again writing them to the disk.
I think the time taken will be dominated by the disk writes; if all else is
well implemented it ought to be irrelevant.
“amitr0” wrote in message news:xxxxx@ntfsd…
Sure Matt, I will let you know.
Also I have a simple doubt regarding this…
Writing to a file from user land should be more expensive than writing to
one from our own IRP. Is this true? I mean, if implemented correctly, the
kernel mode way to write logs to a file should be a beter approach than
sending the logs to user land, and then again writing them to the disk.