Disk filter driver synchronization issue

OSR_Community_User · April 23, 2003, 1:59pm

I have a HD upper filter driver for real-time ciphering/unciphering and in
the background, I have a system thread which opens the disk and begins the
initial ciphering process concurrently with normal OS operation. I need to
synchronize both processes to avoid garbaging disk and I’m using ERESOURCE
(lite version). I acquire the ERESOURCE every time I receive and process a
IRP_MJ_READ or IRP_MJ_WRITE and I’m a little bit confused about reentrancy
(re-acquiring ERESOURCE). Is it safe to use this mechanism? Could I have
any problem? I’m using both Windows 2000 & XP.

Thanks a lot.

Jose Vicente.

OSR_Community_User · April 23, 2003, 2:15pm

Storage drivers must be prepared for read and write requests to come in at DISPATCH_LEVEL. You cannot use a resource at dispatch level so you will have problems with this mechanism.

You need to use another mechanism for your synchronization, or you need to push your work off to a dedicated worker thread which cannot cause page faults itself (by using paged-pool, by allowing itself to get paged out during a wait, by calling code you’ve made pagable, etc…)

-p

This posting is provided “AS IS” with no warranties, and confers no rights

-----Original Message-----
From: Jos? Vicente S?nchez Ortega [mailto:xxxxx@terra.es]
Sent: Wednesday, April 23, 2003 11:06 AM
To: NT Developers Interest List

I have a HD upper filter driver for real-time ciphering/unciphering and in the background, I have a system thread which opens the disk and begins the initial ciphering process concurrently with normal OS operation. I need to synchronize both processes to avoid garbaging disk and I’m using ERESOURCE (lite version). I acquire the ERESOURCE every time I receive and process a IRP_MJ_READ or IRP_MJ_WRITE and I’m a little bit confused about reentrancy (re-acquiring ERESOURCE). Is it safe to use this mechanism? Could I have any problem? I’m using both Windows 2000 & XP.

Thanks a lot.

Jose Vicente.

You are currently subscribed to ntdev as: xxxxx@microsoft.com To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · April 23, 2003, 2:25pm

I suspect you may have problems if the filter gets called with a
raised IRQL. You could use spin locks instead of ERESOURCE, but I
would recommend queuing any requests that come into the filter that
are for parts of the disk you are currently encrypting. You will still
need to use spinlocks to control access to the list (e.g. using the
normal ExInterlockedInsertTailList etc).

To queue the IRP, just put it on your list and return STATUS_PENDING.
After completing the encryption that effects that request, remove it
from the list and continue processing it.

Shaun

Wednesday, April 23, 2003, 7:06:29 PM, you wrote:

JVSO> I have a HD upper filter driver for real-time ciphering/unciphering and in
JVSO> the background, I have a system thread which opens the disk and begins the
JVSO> initial ciphering process concurrently with normal OS operation. I need to
JVSO> synchronize both processes to avoid garbaging disk and I’m using ERESOURCE
JVSO> (lite version). I acquire the ERESOURCE every time I receive and process a
JVSO> IRP_MJ_READ or IRP_MJ_WRITE and I’m a little bit confused about reentrancy
JVSO> (re-acquiring ERESOURCE). Is it safe to use this mechanism? Could I have
JVSO> any problem? I’m using both Windows 2000 & XP.

JVSO> Thanks a lot.

JVSO> Jose Vicente.

JVSO> —
JVSO> You are currently subscribed to ntdev as: xxxxx@sdlabs.net
JVSO> To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · April 24, 2003, 6:21am

I’m not sure that is right. Reading DDK documentation (“Dispatching Routines
and IRLQ” article), both Read and Write dispatch routines can be called at
most at APC_LEVEL (actually at PASSIVE_LEVEL for most drivers). Peter, can
you confirm this issue, please?

Given this additional data, KeAcquireResourceLite can be called ay APC_LEVEL
and KeReleseResourceLite can be called at DISPATCH_LEVEL so I wouldn’t have
problems since I call KeReleseResourceLite in the Completion routine.

Best regards,

Jose Vicente.

-----Mensaje original-----
De: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]En nombre de Peter Wieland
Enviado el: mi?rcoles, 23 de abril de 2003 20:15
Para: NT Developers Interest List
Asunto: [ntdev] RE: Disk filter driver synchronization issue

Storage drivers must be prepared for read and write requests to come in at
DISPATCH_LEVEL. You cannot use a resource at dispatch level so you will
have problems with this mechanism.

You need to use another mechanism for your synchronization, or you need to
push your work off to a dedicated worker thread which cannot cause page
faults itself (by using paged-pool, by allowing itself to get paged out
during a wait, by calling code you’ve made pagable, etc…)

-p

This posting is provided “AS IS” with no warranties, and confers no rights

-----Original Message-----
From: Jos? Vicente S?nchez Ortega [mailto:xxxxx@terra.es]
Sent: Wednesday, April 23, 2003 11:06 AM
To: NT Developers Interest List

I have a HD upper filter driver for real-time ciphering/unciphering and in
the background, I have a system thread which opens the disk and begins the
initial ciphering process concurrently with normal OS operation. I need to
synchronize both processes to avoid garbaging disk and I’m using ERESOURCE
(lite version). I acquire the ERESOURCE every time I receive and process a
IRP_MJ_READ or IRP_MJ_WRITE and I’m a little bit confused about reentrancy
(re-acquiring ERESOURCE). Is it safe to use this mechanism? Could I have any
problem? I’m using both Windows 2000 & XP.

Thanks a lot.

Jose Vicente.

You are currently subscribed to ntdev as: xxxxx@microsoft.com To
unsubscribe send a blank email to xxxxx@lists.osr.com

You are currently subscribed to ntdev as: xxxxx@secuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

Mark_Roddy · April 24, 2003, 9:08am

-----Original Message-----
From: “Jose Vicente Sanchez Ortega”
To: “NT Developers Interest List”
Date: Thu, 24 Apr 2003 12:21:23 +0200
Subject: [ntdev] RE: Disk filter driver synchronization issue

> I’m not sure that is right. Reading DDK documentation (“Dispatching
> Routines
> and IRLQ” article), both Read and Write dispatch routines can be called
> at
> most at APC_LEVEL (actually at PASSIVE_LEVEL for most drivers). Peter,
> can
> you confirm this issue, please?
>

Trust us, that is completely wrong. The storage stack is an obvious and
notable exception to this claim in the ddk, as paging request occur at
and must be processed at DISPATCH_LEVEL.

===========================
Mark Roddy
Consultant, Microsoft DDK MVP
Hollis Technology Solutions
xxxxx@hollistech.com
www.hollistech.com
603-321-1032

OSR_Community_User · April 24, 2003, 11:50am

For most disk drivers, the read and write handlers are called at
DISPATCH_LEVEL because it more than likely the cache manager is doing
the disk IO and not a specific process. Inpage operations and the likes
must be called at some higher dispatch to ensure proper synchronization.

Jamey

-----Original Message-----
From: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com] On Behalf Of Jose Vicente
Sanchez Ortega
Sent: Thursday, April 24, 2003 3:21 AM
To: NT Developers Interest List
Subject: [ntdev] RE: Disk filter driver synchronization issue

I’m not sure that is right. Reading DDK documentation (“Dispatching
Routines
and IRLQ” article), both Read and Write dispatch routines can be called
at
most at APC_LEVEL (actually at PASSIVE_LEVEL for most drivers). Peter,
can
you confirm this issue, please?

Given this additional data, KeAcquireResourceLite can be called ay
APC_LEVEL
and KeReleseResourceLite can be called at DISPATCH_LEVEL so I wouldn’t
have
problems since I call KeReleseResourceLite in the Completion routine.

Best regards,

Jose Vicente.

-----Mensaje original-----
De: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]En nombre de Peter Wieland
Enviado el: mi?rcoles, 23 de abril de 2003 20:15
Para: NT Developers Interest List
Asunto: [ntdev] RE: Disk filter driver synchronization issue

Storage drivers must be prepared for read and write requests to come in
at
DISPATCH_LEVEL. You cannot use a resource at dispatch level so you will
have problems with this mechanism.

You need to use another mechanism for your synchronization, or you need
to
push your work off to a dedicated worker thread which cannot cause page
faults itself (by using paged-pool, by allowing itself to get paged out
during a wait, by calling code you’ve made pagable, etc…)

-p

This posting is provided “AS IS” with no warranties, and confers no
rights

-----Original Message-----
From: Jos? Vicente S?nchez Ortega [mailto:xxxxx@terra.es]
Sent: Wednesday, April 23, 2003 11:06 AM
To: NT Developers Interest List

I have a HD upper filter driver for real-time ciphering/unciphering and
in
the background, I have a system thread which opens the disk and begins
the
initial ciphering process concurrently with normal OS operation. I need
to
synchronize both processes to avoid garbaging disk and I’m using
ERESOURCE
(lite version). I acquire the ERESOURCE every time I receive and process
a
IRP_MJ_READ or IRP_MJ_WRITE and I’m a little bit confused about
reentrancy
(re-acquiring ERESOURCE). Is it safe to use this mechanism? Could I have
any
problem? I’m using both Windows 2000 & XP.

Thanks a lot.

Jose Vicente.

You are currently subscribed to ntdev as: xxxxx@microsoft.com To
unsubscribe send a blank email to xxxxx@lists.osr.com

You are currently subscribed to ntdev as: xxxxx@secuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

You are currently subscribed to ntdev as: xxxxx@storagecraft.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · April 24, 2003, 11:56am

all drivers are equal, but some are more equal than others.

the DDK documentation is probably talking about top-level drivers. Storage drivers are about as bottom-level as you can get in the IO stacks.

file systems occasionally retry request in their completion routines, which can be running at DPC level. Upper-level drivers can do the same. They could also issue new IRPs in timers, DPCs, etc …

additionally, storage drivers which are in the paging path (which could be just about any storage driver) cannot cause page-faults when servicing read or write requests, and cannot contend with any thread in such a path or it risks deadlock. This rules out touching paged pool, blocking, using the registry, etc… It’s also dangerous to use blocking locks since those locks could be acquired by your IOCTL routine which does cause page faults (which then issues a paging read to your driver which attempts to acquire the already acquired lock and you deadlock)

Just use spinlocks. As someone previously suggested, you should use them to protect a data structure which tells you which blocks are in progress and a queue of requests that are waiting for those blocks to be finished. when a read or write comes in you grab the lock, see if the block is busy (ie. encryption in progress) and either (a) queue the operation until the block is no longer busy or (b) service the request using the unencrypted data that you have cached while doing your encryption.

It may be sufficient to ensure your driver thread doesn’t take page faults. However i’m always a bit worried about adding threads into the paging path. You might think about putting in a set of IOCTLs that your driver supports which allow some component to read a disk range and get back a token they can use when writing the encrypted data. If the block range has changed due to a normal write then the corresponding write IOCTL (which took the token supplied by the read) would fail indiciating to the encryptor that it should redo the encryption.

Another option woudl be to have one “encrypt block/range” ioctl which your thread sends to the driver which does the read/modify/write in chained completion routines (lock block, send read, in completion routine encrypt it and send write, in completion routine unlock block). This is more complex code split up across multiple routines but would avoid the inevitable case in my first suggestion where a disk block is too frequently written for you to manage to encrypt it. You might look at classpnp/cdrom’s IOCTL_CDROM_RAW_READ support for an example of this sort of chaining. On older CD’s each RAW_READ requires a cd command to switch the mode of the drive into larger sectors, a read and a subsequent command to switch it back.

In the latter case you might not even need a system thread and could have a formatter process which sends the IOCTLS from user-mode.

What do you do if the power to the machine is lost during the encryption process? Do you have metadata somewhere which records which blocks have been encrypted so you don’t reboot after only encrypting half the disk and think you’ve encrypted it all?

sounds like a fun project.

-p

-----Original Message-----
From: Jose Vicente Sanchez Ortega [mailto:xxxxx@secuware.com]
Sent: Thursday, April 24, 2003 3:21 AM
To: NT Developers Interest List

I’m not sure that is right. Reading DDK documentation
(“Dispatching Routines and IRLQ” article), both Read and
Write dispatch routines can be called at most at APC_LEVEL
(actually at PASSIVE_LEVEL for most drivers). Peter, can you
confirm this issue, please?

Given this additional data, KeAcquireResourceLite can be
called ay APC_LEVEL and KeReleseResourceLite can be called at
DISPATCH_LEVEL so I wouldn’t have problems since I call
KeReleseResourceLite in the Completion routine.

Best regards,

Jose Vicente.

-----Mensaje original-----
De: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]En nombre de Peter
Wieland Enviado el: mi?rcoles, 23 de abril de 2003 20:15
Para: NT Developers Interest List
Asunto: [ntdev] RE: Disk filter driver synchronization issue

Storage drivers must be prepared for read and write requests
to come in at DISPATCH_LEVEL. You cannot use a resource at
dispatch level so you will have problems with this mechanism.

You need to use another mechanism for your synchronization,
or you need to push your work off to a dedicated worker
thread which cannot cause page faults itself (by using
paged-pool, by allowing itself to get paged out during a
wait, by calling code you’ve made pagable, etc…)

-p

This posting is provided “AS IS” with no warranties, and
confers no rights

-----Original Message-----
From: Jos? Vicente S?nchez Ortega [mailto:xxxxx@terra.es]
Sent: Wednesday, April 23, 2003 11:06 AM
To: NT Developers Interest List

I have a HD upper filter driver for real-time
ciphering/unciphering and in the background, I have a system
thread which opens the disk and begins the initial ciphering
process concurrently with normal OS operation. I need to
synchronize both processes to avoid garbaging disk and I’m
using ERESOURCE (lite version). I acquire the ERESOURCE every
time I receive and process a IRP_MJ_READ or IRP_MJ_WRITE and
I’m a little bit confused about reentrancy (re-acquiring
ERESOURCE). Is it safe to use this mechanism? Could I have
any problem? I’m using both Windows 2000 & XP.

Thanks a lot.

Jose Vicente.

You are currently subscribed to ntdev as:
xxxxx@microsoft.com To unsubscribe send a blank email to
xxxxx@lists.osr.com

You are currently subscribed to ntdev as:
xxxxx@secuware.com To unsubscribe send a blank email to
xxxxx@lists.osr.com

You are currently subscribed to ntdev as:
xxxxx@microsoft.com To unsubscribe send a blank email to
xxxxx@lists.osr.com

Maxim_S_Shatskih · April 24, 2003, 12:32pm

> Given this additional data, KeAcquireResourceLite can be called ay
APC_LEVEL

and KeReleseResourceLite can be called at DISPATCH_LEVEL so I
wouldn’t have

ExReleaseResourceLite

Max

Maxim_S_Shatskih · April 24, 2003, 12:32pm

> Trust us, that is completely wrong. The storage stack is an obvious
and

notable exception to this claim in the ddk, as paging request occur
at
and must be processed at DISPATCH_LEVEL.

At APC_LEVEL.

Max

OSR_Community_User · April 24, 2003, 2:31pm

I trust you, Mark. My only doubt is that, for example I have developed a
lower-filter driver for CD-ROM drives in order to do the same ciphering I
was talking about and since that driver is above SCSI mini-port driver, it
must process commands in DISPATCH_LEVEL (it process SCSIOP_XXX commands
through IRP_MJ_INTERNAL_DEVICE_CONTROL and IRQL=2) but my filter driver for
HD class is an upper filter and I think it’s not lowest driver whom you are
speaking about. Probably I am wrong but at this moment, my
encrypting/descrypting HD driver is working.

Thank you, Mark.

Jose Vicente.

-----Mensaje original-----
De: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]En nombre de Mark Roddy
Enviado el: jueves, 24 de abril de 2003 15:07
Para: NT Developers Interest List
Asunto: [ntdev] RE: Disk filter driver synchronization issue

-----Original Message-----
From: “Jose Vicente Sanchez Ortega”
To: “NT Developers Interest List”
Date: Thu, 24 Apr 2003 12:21:23 +0200
Subject: [ntdev] RE: Disk filter driver synchronization issue

> I’m not sure that is right. Reading DDK documentation (“Dispatching
> Routines
> and IRLQ” article), both Read and Write dispatch routines can be called
> at
> most at APC_LEVEL (actually at PASSIVE_LEVEL for most drivers). Peter,
> can
> you confirm this issue, please?
>

Trust us, that is completely wrong. The storage stack is an obvious and
notable exception to this claim in the ddk, as paging request occur at
and must be processed at DISPATCH_LEVEL.

===========================
Mark Roddy
Consultant, Microsoft DDK MVP
Hollis Technology Solutions
xxxxx@hollistech.com
www.hollistech.com
603-321-1032

—
You are currently subscribed to ntdev as: xxxxx@secuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · April 24, 2003, 2:40pm

Peter:

I have been thinking about using spin-locks but I have doubts. Using this
mechanism, every READ and WRITE request will raise processing level to
DISPATCH thus making the systemt to respond “heavily” to disk access (I have
not made tests, it’s only a guess). In relation with the posibility of
caching requests, I must say that one of the premises is that initial
ciphering process would be as quick as posible and I feel that it could not
be a great idea to enque requests (again I’m guessing based only in my
experience). The other solution that you describe (that by using an external
app/service which can send IOCTL’s to the driver) I must say that I have
thought about it some time ago but I would like to avoid using external
applications.

Than you very much for your valuable help and your interest in this project.

Regards,

Jose Vicente.

-----Mensaje original-----
De: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]En nombre de Peter Wieland
Enviado el: jueves, 24 de abril de 2003 17:56
Para: NT Developers Interest List
Asunto: [ntdev] RE: Disk filter driver synchronization issue

all drivers are equal, but some are more equal than others.

the DDK documentation is probably talking about top-level drivers. Storage
drivers are about as bottom-level as you can get in the IO stacks.

file systems occasionally retry request in their completion routines, which
can be running at DPC level. Upper-level drivers can do the same. They
could also issue new IRPs in timers, DPCs, etc …

additionally, storage drivers which are in the paging path (which could be
just about any storage driver) cannot cause page-faults when servicing read
or write requests, and cannot contend with any thread in such a path or it
risks deadlock. This rules out touching paged pool, blocking, using the
registry, etc… It’s also dangerous to use blocking locks since those locks
could be acquired by your IOCTL routine which does cause page faults (which
then issues a paging read to your driver which attempts to acquire the
already acquired lock and you deadlock)

Just use spinlocks. As someone previously suggested, you should use them to
protect a data structure which tells you which blocks are in progress and a
queue of requests that are waiting for those blocks to be finished. when a
read or write comes in you grab the lock, see if the block is busy (ie.
encryption in progress) and either (a) queue the operation until the block
is no longer busy or (b) service the request using the unencrypted data that
you have cached while doing your encryption.

It may be sufficient to ensure your driver thread doesn’t take page faults.
However i’m always a bit worried about adding threads into the paging path.
You might think about putting in a set of IOCTLs that your driver supports
which allow some component to read a disk range and get back a token they
can use when writing the encrypted data. If the block range has changed due
to a normal write then the corresponding write IOCTL (which took the token
supplied by the read) would fail indiciating to the encryptor that it should
redo the encryption.

Another option woudl be to have one “encrypt block/range” ioctl which your
thread sends to the driver which does the read/modify/write in chained
completion routines (lock block, send read, in completion routine encrypt it
and send write, in completion routine unlock block). This is more complex
code split up across multiple routines but would avoid the inevitable case
in my first suggestion where a disk block is too frequently written for you
to manage to encrypt it. You might look at classpnp/cdrom’s
IOCTL_CDROM_RAW_READ support for an example of this sort of chaining. On
older CD’s each RAW_READ requires a cd command to switch the mode of the
drive into larger sectors, a read and a subsequent command to switch it
back.

In the latter case you might not even need a system thread and could have a
formatter process which sends the IOCTLS from user-mode.

What do you do if the power to the machine is lost during the encryption
process? Do you have metadata somewhere which records which blocks have
been encrypted so you don’t reboot after only encrypting half the disk and
think you’ve encrypted it all?

sounds like a fun project.

-p

-----Original Message-----
From: Jose Vicente Sanchez Ortega [mailto:xxxxx@secuware.com]
Sent: Thursday, April 24, 2003 3:21 AM
To: NT Developers Interest List

I’m not sure that is right. Reading DDK documentation
(“Dispatching Routines and IRLQ” article), both Read and
Write dispatch routines can be called at most at APC_LEVEL
(actually at PASSIVE_LEVEL for most drivers). Peter, can you
confirm this issue, please?

Given this additional data, KeAcquireResourceLite can be
called ay APC_LEVEL and KeReleseResourceLite can be called at
DISPATCH_LEVEL so I wouldn’t have problems since I call
KeReleseResourceLite in the Completion routine.

Best regards,

Jose Vicente.

-----Mensaje original-----
De: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]En nombre de Peter
Wieland Enviado el: mi?rcoles, 23 de abril de 2003 20:15
Para: NT Developers Interest List
Asunto: [ntdev] RE: Disk filter driver synchronization issue

Storage drivers must be prepared for read and write requests
to come in at DISPATCH_LEVEL. You cannot use a resource at
dispatch level so you will have problems with this mechanism.

You need to use another mechanism for your synchronization,
or you need to push your work off to a dedicated worker
thread which cannot cause page faults itself (by using
paged-pool, by allowing itself to get paged out during a
wait, by calling code you’ve made pagable, etc…)

-p

This posting is provided “AS IS” with no warranties, and
confers no rights

-----Original Message-----
From: Jos? Vicente S?nchez Ortega [mailto:xxxxx@terra.es]
Sent: Wednesday, April 23, 2003 11:06 AM
To: NT Developers Interest List

I have a HD upper filter driver for real-time
ciphering/unciphering and in the background, I have a system
thread which opens the disk and begins the initial ciphering
process concurrently with normal OS operation. I need to
synchronize both processes to avoid garbaging disk and I’m
using ERESOURCE (lite version). I acquire the ERESOURCE every
time I receive and process a IRP_MJ_READ or IRP_MJ_WRITE and
I’m a little bit confused about reentrancy (re-acquiring
ERESOURCE). Is it safe to use this mechanism? Could I have
any problem? I’m using both Windows 2000 & XP.

Thanks a lot.

Jose Vicente.

You are currently subscribed to ntdev as:
xxxxx@microsoft.com To unsubscribe send a blank email to
xxxxx@lists.osr.com

You are currently subscribed to ntdev as:
xxxxx@secuware.com To unsubscribe send a blank email to
xxxxx@lists.osr.com

You are currently subscribed to ntdev as:
xxxxx@microsoft.com To unsubscribe send a blank email to
xxxxx@lists.osr.com

You are currently subscribed to ntdev as: xxxxx@secuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · April 24, 2003, 2:49pm

Excuse me, Max. I liked to say “ExReleaseResourceLite”. This is the service
that I’m using.

Thanks for your comments. What dou you think about my last mail to Mark?
Could I be right about my guessing about the difference between CD-ROm case
and HD?

Regards,

Jose Vicente.

-----Mensaje original-----
De: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]En nombre de Maxim S. Shatskih
Enviado el: jueves, 24 de abril de 2003 18:12
Para: NT Developers Interest List
Asunto: [ntdev] RE: Disk filter driver synchronization issue

Given this additional data, KeAcquireResourceLite can be called ay
APC_LEVEL
and KeReleseResourceLite can be called at DISPATCH_LEVEL so I
wouldn’t have

ExReleaseResourceLite

Max

You are currently subscribed to ntdev as: xxxxx@secuware.com
To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · April 24, 2003, 2:54pm

The cost of the spinlock acquisition is probably going to pale in comparison to your encryption/decryption. And most of the other synchronization mechansims use some sort of spinlock to protect their internal state anyway.

On the note of not wanting external programs - personally I think you should move as much code as possible out of your driver and into kernel mode. I suggest this because all code has the potential for bugs, because the code you’re writing doesn’t sound trivial, and because bugs in user-mode apps have a significantly less drastic effect on system stability.

-p

This posting is provided “AS IS” with no warranties, and confers no rights

-----Original Message-----
From: Jose Vicente Sanchez Ortega [mailto:xxxxx@secuware.com]
Sent: Thursday, April 24, 2003 11:40 AM
To: NT Developers Interest List

Peter:

I have been thinking about using spin-locks but I have doubts. Using this mechanism, every READ and WRITE request will raise processing level to DISPATCH thus making the systemt to respond “heavily” to disk access (I have not made tests, it’s only a guess). In relation with the posibility of caching requests, I must say that one of the premises is that initial ciphering process would be as quick as posible and I feel that it could not be a great idea to enque requests (again I’m guessing based only in my experience). The other solution that you describe (that by using an external app/service which can send IOCTL’s to the driver) I must say that I have thought about it some time ago but I would like to avoid using external applications.

Than you very much for your valuable help and your interest in this project.

Regards,

Jose Vicente.

-----Mensaje original-----
De: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]En nombre de Peter Wieland Enviado el: jueves, 24 de abril de 2003 17:56
Para: NT Developers Interest List
Asunto: [ntdev] RE: Disk filter driver synchronization issue

all drivers are equal, but some are more equal than others.

the DDK documentation is probably talking about top-level drivers. Storage drivers are about as bottom-level as you can get in the IO stacks.

file systems occasionally retry request in their completion routines, which can be running at DPC level. Upper-level drivers can do the same. They could also issue new IRPs in timers, DPCs, etc …

additionally, storage drivers which are in the paging path (which could be just about any storage driver) cannot cause page-faults when servicing read or write requests, and cannot contend with any thread in such a path or it risks deadlock. This rules out touching paged pool, blocking, using the registry, etc… It’s also dangerous to use blocking locks since those locks could be acquired by your IOCTL routine which does cause page faults (which then issues a paging read to your driver which attempts to acquire the already acquired lock and you deadlock)

Just use spinlocks. As someone previously suggested, you should use them to protect a data structure which tells you which blocks are in progress and a queue of requests that are waiting for those blocks to be finished. when a read or write comes in you grab the lock, see if the block is busy (ie.
encryption in progress) and either (a) queue the operation until the block is no longer busy or (b) service the request using the unencrypted data that you have cached while doing your encryption.

It may be sufficient to ensure your driver thread doesn’t take page faults.
However i’m always a bit worried about adding threads into the paging path.
You might think about putting in a set of IOCTLs that your driver supports which allow some component to read a disk range and get back a token they can use when writing the encrypted data. If the block range has changed due to a normal write then the corresponding write IOCTL (which took the token supplied by the read) would fail indiciating to the encryptor that it should redo the encryption.

Another option woudl be to have one “encrypt block/range” ioctl which your thread sends to the driver which does the read/modify/write in chained completion routines (lock block, send read, in completion routine encrypt it and send write, in completion routine unlock block). This is more complex code split up across multiple routines but would avoid the inevitable case in my first suggestion where a disk block is too frequently written for you to manage to encrypt it. You might look at classpnp/cdrom’s IOCTL_CDROM_RAW_READ support for an example of this sort of chaining. On older CD’s each RAW_READ requires a cd command to switch the mode of the drive into larger sectors, a read and a subsequent command to switch it back.

In the latter case you might not even need a system thread and could have a formatter process which sends the IOCTLS from user-mode.

What do you do if the power to the machine is lost during the encryption process? Do you have metadata somewhere which records which blocks have been encrypted so you don’t reboot after only encrypting half the disk and think you’ve encrypted it all?

sounds like a fun project.

-p

-----Original Message-----
From: Jose Vicente Sanchez Ortega [mailto:xxxxx@secuware.com]
Sent: Thursday, April 24, 2003 3:21 AM
To: NT Developers Interest List

I’m not sure that is right. Reading DDK documentation (“Dispatching
Routines and IRLQ” article), both Read and Write dispatch routines can
be called at most at APC_LEVEL (actually at PASSIVE_LEVEL for most
drivers). Peter, can you confirm this issue, please?

Given this additional data, KeAcquireResourceLite can be called ay
APC_LEVEL and KeReleseResourceLite can be called at DISPATCH_LEVEL so
I wouldn’t have problems since I call KeReleseResourceLite in the
Completion routine.

Best regards,

Jose Vicente.

-----Mensaje original-----
De: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]En nombre de Peter Wieland
Enviado el: mi?rcoles, 23 de abril de 2003 20:15
Para: NT Developers Interest List
Asunto: [ntdev] RE: Disk filter driver synchronization issue

Storage drivers must be prepared for read and write requests to come
in at DISPATCH_LEVEL. You cannot use a resource at dispatch level so
you will have problems with this mechanism.

You need to use another mechanism for your synchronization, or you
need to push your work off to a dedicated worker thread which cannot
cause page faults itself (by using paged-pool, by allowing itself to
get paged out during a wait, by calling code you’ve made pagable,
etc…)

-p

This posting is provided “AS IS” with no warranties, and confers no
rights

-----Original Message-----
From: Jos? Vicente S?nchez Ortega [mailto:xxxxx@terra.es]
Sent: Wednesday, April 23, 2003 11:06 AM
To: NT Developers Interest List

I have a HD upper filter driver for real-time ciphering/unciphering
and in the background, I have a system thread which opens the disk and
begins the initial ciphering process concurrently with normal OS
operation. I need to synchronize both processes to avoid garbaging
disk and I’m using ERESOURCE (lite version). I acquire the ERESOURCE
every time I receive and process a IRP_MJ_READ or IRP_MJ_WRITE and I’m
a little bit confused about reentrancy (re-acquiring ERESOURCE). Is it
safe to use this mechanism? Could I have any problem? I’m using both
Windows 2000 & XP.

Thanks a lot.

Jose Vicente.

You are currently subscribed to ntdev as:
xxxxx@microsoft.com To unsubscribe send a blank email to
xxxxx@lists.osr.com

You are currently subscribed to ntdev as:
xxxxx@secuware.com To unsubscribe send a blank email to
xxxxx@lists.osr.com

You are currently subscribed to ntdev as:
xxxxx@microsoft.com To unsubscribe send a blank email to
xxxxx@lists.osr.com

You are currently subscribed to ntdev as: xxxxx@secuware.com To unsubscribe send a blank email to xxxxx@lists.osr.com

You are currently subscribed to ntdev as: xxxxx@microsoft.com To unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · April 24, 2003, 2:56pm

CD-ROM drives are never part of the system paging path so they have less
restrictions on what they can and can’t do than disk drives do.

That’s not to say they don’t get paging IO since any file or image on
the CD could be memory mapped by a process. However they don’t have a
page file which is what makes much of the difference.

That said, the CDFS driver does retry requests in its completion
routines which in turn means that the driver could be called at
DISPATCH_LEVEL.

-p

This posting is provided “AS IS” with no warranties, and confers no
rights

-----Original Message-----
From: Jose Vicente Sanchez Ortega [mailto:xxxxx@secuware.com]
Sent: Thursday, April 24, 2003 11:49 AM
To: NT Developers Interest List

Excuse me, Max. I liked to say “ExReleaseResourceLite”. This is the
service that I’m using.

Thanks for your comments. What dou you think about my last mail to Mark?
Could I be right about my guessing about the difference between CD-ROm
case and HD?

Regards,

Jose Vicente.

-----Mensaje original-----
De: xxxxx@lists.osr.com
[mailto:xxxxx@lists.osr.com]En nombre de Maxim S. Shatskih
Enviado el: jueves, 24 de abril de 2003 18:12
Para: NT Developers Interest List
Asunto: [ntdev] RE: Disk filter driver synchronization issue

Given this additional data, KeAcquireResourceLite can be called ay
APC_LEVEL
and KeReleseResourceLite can be called at DISPATCH_LEVEL so I
wouldn’t have

ExReleaseResourceLite

Max

You are currently subscribed to ntdev as: xxxxx@secuware.com To
unsubscribe send a blank email to xxxxx@lists.osr.com

You are currently subscribed to ntdev as: xxxxx@microsoft.com To
unsubscribe send a blank email to xxxxx@lists.osr.com

OSR_Community_User · April 24, 2003, 3:04pm

Make that “out of your driver and into user mode”.

-p

This posting is provided “AS IS” with no warranties, and confers no rights

-----Original Message-----
From: Peter Wieland [mailto:xxxxx@windows.microsoft.com]
Sent: Thursday, April 24, 2003 11:54 AM
To: NT Developers Interest List