Also, as far as i know, well known manufacturers of storage devices,
have also come out with drives that have huge in built caches (8MB
atleast), which they fill and maintaqin at the firmware level.
On 8/18/05, Eric Thiebaut-George wrote:
> Yes, I’m aware of that. It’s the case for some SATA drives in
> particular.
>
> Eric
>
> On Wed, 17 Aug 2005 10:52:21 -0400, “Arlie Davis”
> said:
> > Don’t forget that some devices do better when they know the full set of
> > active IRPs, because they can re-order them. I believe even some disk
> > hardware controllers will reorder requests to reduce seek time. So you
> > may
> > even be slowing down the total throughput, without benefiting the
> > “privileged” apps at all.
> >
> > – arlie
> >
> >
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of Eric
> > Thiebaut-George
> > Sent: Wednesday, August 17, 2005 8:41 AM
> > To: Windows File Systems Devs Interest List
> > Subject: SPAM-LOW: Re: [ntfsd] Disk I/O priority
> >
> > > > Depending on the PID which generated the original request, I have to
> > > > decide which request should go straight to the underlying driver,
> > > > and which request should be delayed.
> > >
> > > Well, that was exactly, what we were trying out, the priviledged
> > > application got more IRPs through us, than the non priviledged ones.
> > > How do you decide the threshold? The ratio of priviledged IRPs to non
> > > proviledged ones. Also, blocking certain IRPs or delaying them makes
> > > the syustem unstable. You will also have to take care of IRPs getting
> > > cancelled because they got delayed (I am sure, with your experience
> > > you know all this).
> >
> > We haven’t completely figured out the algorithm yet, but at this stage
> > it’s
> > more of a proof of concept…
> >
> > > Well, process specific information is stripped down at FS leve. There
> > > might be a crooked way though, I am not aware of it. Writing a two
> > > stage driver, attaching one at the FS level that passes the driver
> > > attached to the disk the correct PIDs as IOCTLs…may be might work,
> > > not sure.
> >
> > That was pretty much what I was thinking. It does look messy but it might
> > be
> > a candidate if there is no other way…
> >
> > Eric
> >
> > —
> > Questions? First check the IFS FAQ at
> > https://www.osronline.com/article.cfm?id=17
> >
> > You are currently subscribed to ntfsd as: xxxxx@stonestreetone.com To
> > unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> >
> >
> > —
> > Questions? First check the IFS FAQ at
> > https://www.osronline.com/article.cfm?id=17
> >
> > You are currently subscribed to ntfsd as: xxxxx@dungorm.co.uk
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
>
> —
> Questions? First check the IFS FAQ at https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@gmail.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
–
- Developer
It doesn’t stop there, either.
You should also consider drives exposed through SCSI, Fibre Channel,
iSCSI and soon to be SAS. There’s a good chance that although these
appear to the host as drives they are probably some virtual construct
on the storage array. Decisions you make in your driver could very
easily be counter-intuitive to how those devices cache data and
process commands.
Such attached devices might traditionally be rare in the lower end of
the market, but especially with iSCSI they are making rapid inroads.
Mark
At 06:08 AM 8/18/2005, Developer wrote:
Also, as far as i know, well known manufacturers of storage devices,
have also come out with drives that have huge in built caches (8MB
atleast), which they fill and maintaqin at the firmware level.
On 8/18/05, Eric Thiebaut-George wrote:
> > Yes, I’m aware of that. It’s the case for some SATA drives in
> > particular.
> >
> > Eric
> >
> > On Wed, 17 Aug 2005 10:52:21 -0400, “Arlie Davis”
> > said:
> > > Don’t forget that some devices do better when they know the full set of
> > > active IRPs, because they can re-order them. I believe even some disk
> > > hardware controllers will reorder requests to reduce seek time. So you
> > > may
> > > even be slowing down the total throughput, without benefiting the
> > > “privileged” apps at all.
> > >
> > > – arlie
> > >
> > >
> > > -----Original Message-----
> > > From: xxxxx@lists.osr.com
> > > [mailto:xxxxx@lists.osr.com] On Behalf Of Eric
> > > Thiebaut-George
> > > Sent: Wednesday, August 17, 2005 8:41 AM
> > > To: Windows File Systems Devs Interest List
> > > Subject: SPAM-LOW: Re: [ntfsd] Disk I/O priority
> > >
> > > > > Depending on the PID which generated the original request, I have to
> > > > > decide which request should go straight to the underlying driver,
> > > > > and which request should be delayed.
> > > >
> > > > Well, that was exactly, what we were trying out, the priviledged
> > > > application got more IRPs through us, than the non priviledged ones.
> > > > How do you decide the threshold? The ratio of priviledged IRPs to non
> > > > proviledged ones. Also, blocking certain IRPs or delaying them makes
> > > > the syustem unstable. You will also have to take care of IRPs getting
> > > > cancelled because they got delayed (I am sure, with your experience
> > > > you know all this).
> > >
> > > We haven’t completely figured out the algorithm yet, but at this stage
> > > it’s
> > > more of a proof of concept…
> > >
> > > > Well, process specific information is stripped down at FS leve. There
> > > > might be a crooked way though, I am not aware of it. Writing a two
> > > > stage driver, attaching one at the FS level that passes the driver
> > > > attached to the disk the correct PIDs as IOCTLs…may be might work,
> > > > not sure.
> > >
> > > That was pretty much what I was thinking. It does look messy but it might
> > > be
> > > a candidate if there is no other way…
> > >
> > > Eric
> > >
> > > —
> > > Questions? First check the IFS FAQ at
> > > https://www.osronline.com/article.cfm?id=17
> > >
> > > You are currently subscribed to ntfsd as: xxxxx@stonestreetone.com To
> > > unsubscribe send a blank email to xxxxx@lists.osr.com
> > >
> > >
> > >
> > >
> > > —
> > > Questions? First check the IFS FAQ at
> > > https://www.osronline.com/article.cfm?id=17
> > >
> > > You are currently subscribed to ntfsd as: xxxxx@dungorm.co.uk
> > > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> > —
> > Questions? First check the IFS FAQ at
> https://www.osronline.com/article.cfm?id=17
> >
> > You are currently subscribed to ntfsd as: xxxxx@gmail.com
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
> >
>
>
>–
>
>- Developer
>
>—
>Questions? First check the IFS FAQ at
>https://www.osronline.com/article.cfm?id=17
>
>You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
>To unsubscribe send a blank email to xxxxx@lists.osr.com
You cannot do this. Most write disk requests are from Cache Manager, and
the requesting process context is 100% lost in them.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
----- Original Message -----
From: “Eric Thiebaut-George”
To: “Windows File Systems Devs Interest List”
Sent: Wednesday, August 17, 2005 5:22 AM
Subject: [ntfsd] Disk I/O priority
> Hi,
>
> I am trying to write a filter driver which will prioritize requests sent
> to the disk. I’m only interested in actual disk hits and not cache hits
> (and I’m not concerned with network, so that’s one thing off my back!).
>
> Depending on the PID which generated the original request, I have to
> decide which request should go straight to the underlying driver, and
> which request should be delayed. I know that paging writes cannot be
> delayed, but I believe I should be able to delay everything else?
>
> I have already done an encryption filter over the last few years, so I
> know some of the gotchas, but I’m wondering what type of filter I should
> write:
>
> * Above the disk driver
> Pros:
> easier to write, and I only get what I am interested in
> Cons:
> Can I actually get the PID of the original caller?
>
> * Above the FS driver
> Pros:
> I should be able to get (most of the time?) the PID of the original
> process. But what about antivirus softwares that post the requests and
> perform the actual READs and WRITEs from another context?
> Cons:
> I have to deal with the cache manager…
>
> Another possibility would be a combination: “tag” requests at the FS
> level and do the actual delaying at the disk level. But can I get the
> original request at the disk level?
>
> So the main question is: Is there a way to “reliably” get the PID of the
> original request? At what level? What’s the best way to get it? I don’t
> need 100% accuracy, but I’d like to be close enough to have some effect
> on disk I/O priority globally.
>
> Thanks!
>
> Eric
>
> —
> Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@storagecraft.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
This is mainly a protocol’s issue. If modern SATA and Windows ATAPI driver
support tagged queue - then this is really an issue.
It is always an issue with SCSI.
Maxim Shatskih, Windows DDK MVP
StorageCraft Corporation
xxxxx@storagecraft.com
http://www.storagecraft.com
----- Original Message -----
From: “Developer”
To: “Windows File Systems Devs Interest List”
Sent: Thursday, August 18, 2005 9:08 AM
Subject: Re: SPAM-LOW: Re: [ntfsd] Disk I/O priority
Also, as far as i know, well known manufacturers of storage devices,
have also come out with drives that have huge in built caches (8MB
atleast), which they fill and maintaqin at the firmware level.
On 8/18/05, Eric Thiebaut-George wrote:
> Yes, I’m aware of that. It’s the case for some SATA drives in
> particular.
>
> Eric
>
> On Wed, 17 Aug 2005 10:52:21 -0400, “Arlie Davis”
> said:
> > Don’t forget that some devices do better when they know the full set of
> > active IRPs, because they can re-order them. I believe even some disk
> > hardware controllers will reorder requests to reduce seek time. So you
> > may
> > even be slowing down the total throughput, without benefiting the
> > “privileged” apps at all.
> >
> > – arlie
> >
> >
> > -----Original Message-----
> > From: xxxxx@lists.osr.com
> > [mailto:xxxxx@lists.osr.com] On Behalf Of Eric
> > Thiebaut-George
> > Sent: Wednesday, August 17, 2005 8:41 AM
> > To: Windows File Systems Devs Interest List
> > Subject: SPAM-LOW: Re: [ntfsd] Disk I/O priority
> >
> > > > Depending on the PID which generated the original request, I have to
> > > > decide which request should go straight to the underlying driver,
> > > > and which request should be delayed.
> > >
> > > Well, that was exactly, what we were trying out, the priviledged
> > > application got more IRPs through us, than the non priviledged ones.
> > > How do you decide the threshold? The ratio of priviledged IRPs to non
> > > proviledged ones. Also, blocking certain IRPs or delaying them makes
> > > the syustem unstable. You will also have to take care of IRPs getting
> > > cancelled because they got delayed (I am sure, with your experience
> > > you know all this).
> >
> > We haven’t completely figured out the algorithm yet, but at this stage
> > it’s
> > more of a proof of concept…
> >
> > > Well, process specific information is stripped down at FS leve. There
> > > might be a crooked way though, I am not aware of it. Writing a two
> > > stage driver, attaching one at the FS level that passes the driver
> > > attached to the disk the correct PIDs as IOCTLs…may be might work,
> > > not sure.
> >
> > That was pretty much what I was thinking. It does look messy but it might
> > be
> > a candidate if there is no other way…
> >
> > Eric
> >
> > —
> > Questions? First check the IFS FAQ at
> > https://www.osronline.com/article.cfm?id=17
> >
> > You are currently subscribed to ntfsd as: xxxxx@stonestreetone.com To
> > unsubscribe send a blank email to xxxxx@lists.osr.com
> >
> >
> >
> >
> > —
> > Questions? First check the IFS FAQ at
> > https://www.osronline.com/article.cfm?id=17
> >
> > You are currently subscribed to ntfsd as: xxxxx@dungorm.co.uk
> > To unsubscribe send a blank email to xxxxx@lists.osr.com
>
> —
> Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
>
> You are currently subscribed to ntfsd as: xxxxx@gmail.com
> To unsubscribe send a blank email to xxxxx@lists.osr.com
>
–
- Developer
—
Questions? First check the IFS FAQ at
https://www.osronline.com/article.cfm?id=17
You are currently subscribed to ntfsd as: unknown lmsubst tag argument: ‘’
To unsubscribe send a blank email to xxxxx@lists.osr.com